Business system development management system based on big data analysis

Abstract

The present invention discloses a business system development and management system based on big data analysis, which specifically relates to the field of business system development, including a data collection time determination module, a development data entry module, a development data analysis module, an abnormality analysis module, and a management module. The present invention significantly improves the efficiency and accuracy of business system development management by integrating big data analysis technology. The various modules of the present invention work together to achieve a closed loop of data collection, analysis, abnormality detection, and automated management, ensuring that business data is comprehensive, real-time, and accurate, and a comprehensive abnormality index model scientifically evaluates the system status, timely discovers and warns of potential problems, and provides strong support for management decisions. At the same time, the emergency response mechanism ensures that the system can be quickly restored when abnormal, ensuring business continuity. Overall, the present invention greatly enhances the controllability and stability of business system development.

Description

Technical Field

[0001] The present invention relates to the technical field of business system development, and more specifically, to a business system development management system based on big data analysis. Background Art

[0002] With the rapid development of information technology, the development and management of business systems have increasingly become key links in corporate operations. Under the current background of digital transformation, how to efficiently and accurately manage and optimize business systems has become a common challenge faced by all industries, especially in the fields of software development, operation and maintenance deployment, and data visualization. Technological innovation and optimization of management models have become important driving forces for business development.

[0003] Existing business system development and management systems usually follow a fixed set of operating processes. These systems usually include data collection, data storage, data processing and data analysis. They collect various types of data from business systems through a combination of automated tools and manual operations, including code submission records, development process indicators, operation and maintenance logs, etc. Subsequently, these data are stored in the database and undergo a series of processing and analysis to support the monitoring and management of the business system. However, this traditional model has obvious deficiencies in the real-time, accuracy and intelligence of data processing.

[0004] Although the existing business system development management system has improved management efficiency to a certain extent, there are still many deficiencies in practical applications. First, the real-time and comprehensiveness of data collection is not enough, and it often cannot cover all key data points of the business system, resulting in deviations in the analysis results. Secondly, the data analysis method is relatively single, lacking the ability to deeply mine and analyze big data, making it difficult to discover potential problems and risks. In addition, the existing system also has obvious shortcomings in anomaly detection and early warning, and cannot respond to and handle abnormal situations in the business system in a timely manner. These problems seriously restrict the efficient operation and sustainable development of the business system, and a more advanced and intelligent management system is urgently needed to solve them. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a business system development management system based on big data analysis, and solves the problems raised in the above-mentioned background technology through the following scheme.

[0006] To achieve the above-mentioned object, the present invention provides the following technical solutions: a business system development management system based on big data analysis, including a system operation database, a system central processing module and a user information terminal, and also including a data collection time determination module, a development data entry module, a development data analysis module, an abnormality analysis module and a management module;

[0007] The data collection time determination module is used to determine the data collection time points of the development data collection module, and the data collection time points of the development data collection module are determined in the same time interval manner, and are marked as 1, 2...n in sequence;

[0008] The development data entry module is used to collect code infrastructure data, development process data, visualization tool data, and operation and deployment detail data of the target business system at each time point, and transmit the collected data to the development data analysis module;

[0009] The development data analysis module is used to analyze the data transmitted by the development data entry module and transmit the analysis results to the abnormality analysis module;

[0010] The abnormality analysis module is used to perform a comprehensive analysis on the data transmitted by the development data analysis module, and determine the abnormal state of the target business system according to the analysis results, and transmit the abnormal state of the target business system to the management module;

[0011] The management module is used to issue management instructions according to the abnormal state of the target business system.

[0012] Preferably, the system operation database includes all data texts of the business system development and management system, and collects information texts output by each module in real time. The system central processing module is used to control the information text instructions output by each module, and the user information terminal is an information output device for receiving the business system development and management system.

[0013] Preferably, the data collection time determination module realizes automatic determination of time points in the Linux system through Cron jobs. At each predetermined time point, the system will automatically trigger the data collection process, perform specific data collection tasks by developing a data entry module, and allow the time interval to be adjusted or the data collection time points to be temporarily added or deleted as needed when the system is running.

[0014] Preferably, the development data entry module includes a data collection unit and a first data transmission unit, the data collection unit is used to collect code infrastructure data, development process data, visualization tool data, and operation and maintenance and deployment detail data of the target business system at each time point, and the first data transmission unit is used to transmit the data collected by the data collection unit to the development data analysis module.

[0015] Preferably, the code infrastructure data includes the rate of change of the number of code submissions for development tasks, the number of intermediate products of code compilation, the time difference of code base snapshots, and the frequency of magic numbers in the code, which are marked as Cr, Cb, Cs, and Cm, respectively; the development process data includes the frequency of requirement changes, the Pull Request feedback delay time, the number of code merge conflicts between developers, and the degree of task slicing, which are marked as Rc, Rp, Rm, and Rt, respectively; the visualization tool data includes the graphical interface response delay, the number of interactive errors of the visualization tool, the visualization completeness of the unit test results, and the visualization participation in the code review process, which are marked as Gr, Ge, Gt, and Gv, respectively; the operation and maintenance and deployment detail data includes the number of revisions of the configuration file, the number of log entries at service startup, the success rate of automated script execution, and the stabilization time after system restart, which are marked as Fc, Fl, Fs, and Fr, respectively.

[0016] Preferably, the data collection unit counts the number of code submissions for each development task through the submission history of the version control system Git, and collects the rate of change of the number of code submissions for the development task by calculating the rate of change between tasks in combination with JIRA; enables logging during the compilation process and counts the number of intermediate product files generated; can obtain the number of intermediate products of code compilation through log parsing of the build tool Maven; takes snapshots of the code base regularly and calculates the time difference between adjacent snapshots; uses scripts in combination with the CI tool Jenkins to implement regular snapshots and difference calculations to obtain the time difference of code base snapshots; uses the static code analysis tool SonarQube to scan the code, and counts the frequency of specific values ​​of magic numbers used directly in the code instead of constants to obtain the frequency of magic number occurrence.

[0017] Preferably, the data collection unit counts the frequency of requirement changes through the change log of the requirement management system JIRA, obtains the creation time and the first comment time of each Pull Request through the API of the code hosting platform GitLab, calculates the time difference between the two to obtain the Pull Request feedback delay time, counts the number of conflicts occurring during the merging process through the merge log of the version control system to obtain the number of code merge conflicts between developers, and obtains the degree of task slicing through the task division of the project management tool JIRA. The specific method is to count the complexity of development tasks and calculate the average task granularity. The task granularity can be measured by the number of subtasks of the task.

[0018] Preferably, the data collection unit simulates user operations through the performance testing tool Selenium and records the interface response time to obtain the graphical interface response delay, adds an error log function to the visualization tool, counts the number of errors occurring in the user's operations to obtain the number of interactive errors of the visualization tool, checks the output of the test report generation tool JUnit to count whether all unit test results are fully displayed, and calculates the integrity ratio to obtain the visual integrity of the unit test results, the visualization participation in the code review process is obtained through the activity log of the code review tool Crucible, and counts the usage and participation of the visualization tool in each code review, specifically the number of graphics attached to the comments.

[0019] Preferably, the data collection unit counts the submission history of the configuration file through the version control system Git, calculates the number of revisions of the configuration file, obtains the number of log entries at service startup by adding a log statistics function to the service startup script, records the number of log entries generated from the execution of the startup command to the full startup of the service, obtains the success rate of the automation script execution by recording the execution result after the automation script is executed, and obtains and counts the success rate through the CI / CD tool Jenkins, and obtains the stability time after the system restart is obtained by recording the system restart completion time and the system reaches a stable state, specifically the time required for no abnormal logs and no high load, and obtains the stability time after the system restart through the script combined with the monitoring tool Grafana.

[0020] Preferably, the development data analysis module includes a first receiving unit, a data analysis unit and a second data transmission unit. The first receiving unit is used to receive data transmitted by the first data transmission unit and pre-process the data, specifically including data cleaning, data integration and data transformation. The data analysis unit is used to establish a code infrastructure data analysis model, a development process data analysis model, a visualization tool data analysis model and an operation and maintenance and deployment details data analysis model, which is used to analyze the data pre-processed by the first receiving unit. The second data transmission unit is used to transmit the analysis results of the data analysis unit to the exception analysis module.

[0021] Preferably, the code infrastructure data analysis model is used to analyze the code infrastructure data transmitted by the development data entry module, which is specifically expressed as:

[0022] ,

[0023] KC i represents the dynamic evaluation value of the code at the i-th time point, Cr i represents the rate of change in the number of code submissions for the development task at time i, Cb i Indicates the number of intermediate products of code compilation at time point i, Cs iIndicates the time difference of the code base snapshot at the i-th time point, Cm i Represents the frequency of magic number in the code at the i-th time point, and n represents the number of time points.

[0024] Preferably, the development process data analysis model is used to analyze the development process data transmitted by the development data entry module, which is specifically expressed as follows:

[0025] ,

[0026] KR i represents the efficiency value of the development process at the i-th time point, Rc i Indicates the frequency of demand change at the i-th time point, Rp i Indicates the Pull Request feedback delay at the i-th time point, Rm i represents the number of code merge conflicts between developers at the i-th time point, Rt i It represents the degree of task slicing at the i-th time point, and n represents the number of time points.

[0027] Preferably, the visualization tool data analysis model is used to analyze the visualization tool data transmitted by the development data entry module, which is specifically expressed as follows:

[0028] ,

[0029] KG i represents the interactive visualization performance evaluation value at the i-th time point, Gr i represents the graphical interface response delay at the i-th time point, Ge i represents the number of interactive errors of the visualization tool at the i-th time point, Gt i Indicates the visual completeness of the unit test results at the i-th time point, Gv i represents the visual participation in the code review process at the i-th time point, and n represents the number of time points.

[0030] Preferably, the operation and maintenance and deployment details data analysis model is used to analyze the operation and maintenance and deployment details data transmitted by the development data entry module, which is specifically expressed as follows:

[0031] ,

[0032] KF i represents the operation and maintenance efficiency value at the i-th time point, Fc i Indicates the revision count of the configuration file at the i-th time point, Fl i Indicates the number of log entries when the service is started at the i-th time point, Fs i represents the success rate of the automated script execution at the i-th time point, Fr iIt represents the stable time after the system restart at the i-th time point, and n represents the number of time points.

[0033] Preferably, the abnormality analysis module includes a second receiving unit, a comprehensive analysis unit, an abnormality judgment unit and a third transmission unit. The second receiving unit is used to receive data transmitted by the second transmission unit. The comprehensive analysis unit is used to establish a comprehensive analysis model, import the data transmitted by the second transmission unit into the comprehensive analysis model, and calculate the comprehensive abnormality index of the target business system. The abnormality judgment unit is used to judge the abnormal state of the target business system according to the comprehensive abnormality index threshold. The third transmission unit is used to transmit the abnormal state of the target business system to the management module.

[0034] Preferably, the comprehensive analysis model is specifically expressed as:

[0035] ,η represents the comprehensive abnormality index of the target business system, KC i ′ represents the normalized code dynamic evaluation value at the i-th time point, KR i ′ represents the standardized development process efficiency value at the i-th time point, KG i ′ represents the standardized interactive visualization performance evaluation value at the i-th time point, KF i ′ represents the normalized operation and maintenance performance value at the i-th time point, w1 represents the weight of the code infrastructure data, w2 represents the weight of the development process data, w3 represents the weight of the visualization tool data, and w4 represents the weight of the operation and maintenance and deployment detail data.

[0036] Preferably, the standardization method in the comprehensive analysis model is the Min-Max standardization method, and w1, w2, w3 and w4 are obtained by fitting experimental data, and w1+w2+w3+w4=1.

[0037] Preferably, the comprehensive abnormality index threshold includes a first threshold and a second threshold, respectively marked as T normal and T warning , and T normal <T warning , when η≤T normal When T normal <η≤T warning When T warning <η, the state of the target business system is marked as abnormal.

[0038] Preferably, the management module includes a third receiving unit, a first control unit, a second control unit and a third control unit, the third receiving unit is used to receive information transmitted by the third transmission unit, the first control unit is used to issue control instructions for a normal state, the second control unit is used to issue control instructions for an alert state, and the third control unit is used to issue control instructions for an abnormal state.

[0039] Preferably, the first control unit keeps acquiring and analyzing the business data when the state of the target business system is normal, and records a normal state log of the target business system.

[0040] Preferably, the second control unit performs wide-coverage health checks twice a day when the target business system is in an alert state, reserves additional computing and network resources, prepares for expansion, and sends a yellow warning signal to management personnel.

[0041] Preferably, the third control unit sets up an emergency response team when the status of the target business system is abnormal, starts troubleshooting within 15 minutes, limits all system traffic, suspends non-critical functions, activates redundant backup systems, performs business switching, and performs health checks every hour until the status is restored.

[0042] Technical effects and advantages of the present invention:

[0043] The present invention determines the time point of development data collection through the data collection time determination module, ensures the standardization and consistency of data collection, marks the data collection time point in the same time interval, and provides an accurate time dimension for subsequent data analysis and anomaly detection; collects multi-dimensional data of the target business system at various time points through the development data entry module, including code infrastructure data, development process data, visualization tool data, and operation and maintenance and deployment detail data. These data provide a rich data source for subsequent development data analysis, which helps to fully understand the development status and operation efficiency of the business system; pre-processes and deeply analyzes the data transmitted by the development data entry module through the development data analysis module, establishes multiple data analysis models, and can Quantitatively evaluate the development efficiency and effectiveness of the business system, providing a scientific basis for anomaly detection and management decisions; conduct a comprehensive analysis of the data transmitted by the development data analysis module through the anomaly analysis module, calculate the comprehensive anomaly index of the target business system, and judge the abnormal state of the business system according to the set threshold, so as to timely discover and warn of potential problems, providing an important basis for the management module to issue corresponding management instructions; the management module issues corresponding management instructions according to the abnormal state of the target business system transmitted by the anomaly analysis module, and can take corresponding control measures for different states to ensure the stable operation and efficient development of the business system. Under abnormal circumstances, the management module can quickly activate the emergency response mechanism to reduce system risks and restore the system status as soon as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0045] Figure 2 It is a schematic diagram of the structure of the development data analysis module of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] refer to Figure 1 The business system development and management system based on big data analysis shown includes a system operation database, a system central processing module and a user information terminal, and also includes a data collection time determination module, a development data entry module, a development data analysis module, an abnormality analysis module and a management module.

[0048] The system operation database includes all data texts of the business system development and management system, and collects the information text output by each module in real time. The system central processing module is used to control the information text instructions output by each module, and the user information terminal is an information output device for receiving the business system development and management system.

[0049] The data collection time determination module is used to determine the data collection time points of the development data collection module, and the data collection time points of the development data collection module are determined in the same time interval manner, which are marked as 1, 2...n in sequence.

[0050] The data collection time determination module realizes automatic determination of time points in the Linux system through Cron jobs. At each predetermined time point, the system will automatically trigger the data collection process, perform specific data collection tasks by developing a data entry module, and allow the time interval to be adjusted or the data collection time points to be temporarily added or deleted as needed when the system is running.

[0051] It should be specifically explained in this embodiment that the first time point represents the initial state of the target business system and does not participate in the actual analysis.

[0052] The development data entry module is used to collect code infrastructure data, development process data, visualization tool data, and operation and maintenance and deployment detail data of the target business system at each time point, and transmit the collected data to the development data analysis module.

[0053] The development data entry module includes a data collection unit and a first data transmission unit. The data collection unit is used to collect code infrastructure data, development process data, visualization tool data, and operation and deployment detail data of the target business system at each time point. The first data transmission unit is used to transmit the data collected by the data collection unit to the development data analysis module.

[0054] The code infrastructure data includes the rate of change of the number of code submissions for development tasks, the number of intermediate products of code compilation, the time difference of code base snapshots, and the frequency of magic numbers in the code, which are marked as Cr, Cb, Cs, and Cm respectively. The development process data includes the frequency of requirement changes, the delay time of Pull Request feedback, the number of code merge conflicts between developers, and the degree of task slicing, which are marked as Rc, Rp, Rm, and Rt respectively. The visualization tool data includes the response delay of the graphical interface, the number of interactive errors of the visualization tool, the visualization completeness of the unit test results, and the visualization participation in the code review process, which are marked as Gr, Ge, Gt, and Gv respectively. The operation and maintenance and deployment detail data includes the number of revisions of the configuration file, the number of log entries at service startup, the success rate of automated script execution, and the stabilization time after system restart, which are marked as Fc, Fl, Fs, and Fr respectively.

[0055] The data collection unit counts the number of code submissions for each development task through the submission history of the version control system Git, and collects the rate of change of the number of code submissions for the development task by calculating the rate of change between tasks in combination with JIRA. By enabling logging during the compilation process, the number of intermediate product files generated is counted, and the number of intermediate products of code compilation can be obtained through log parsing of the build tool Maven. By regularly taking snapshots of the code base and calculating the time difference between adjacent snapshots, a script is used in combination with the CI tool Jenkins to implement regular snapshots and difference calculation to obtain the time difference of code base snapshots. The static code analysis tool SonarQube is used to scan the code, and the frequency of the magic number appearance is obtained by counting the specific numerical values ​​of the magic number directly used in the code instead of the constant frequency.

[0056] The data collection unit counts the frequency of requirement changes through the change log of the requirement management system JIRA, obtains the creation time and the first comment time of each Pull Request through the API of the code hosting platform GitLab, calculates the time difference between the two to obtain the Pull Request feedback delay time, counts the number of conflicts occurring during the merge process through the merge log of the version control system to obtain the number of code merge conflicts between developers, and obtains the degree of task slicing through the task division of the project management tool JIRA. The specific method is to count the complexity of the development tasks and calculate the average task granularity. The task granularity can be measured by the number of subtasks of the task.

[0057] The data collection unit simulates user operations through the performance testing tool Selenium and records the interface response time to obtain the graphical interface response delay. By adding an error log function to the visualization tool, the number of errors occurring in the user's operations is counted to obtain the number of interactive errors of the visualization tool. By checking the output of the test report generation tool JUnit, it is counted whether all unit test results are fully displayed, and the integrity ratio is calculated to obtain the visual integrity of the unit test results. The visualization participation in the code review process is obtained through the activity log of the code review tool Crucible, and the usage and participation of the visualization tool in each code review are counted, specifically the number of graphics attached to the comments.

[0058] The data collection unit counts the submission history of the configuration file through the version control system Git, calculates the number of revisions of the configuration file, adds a log statistics function to the service startup script, records the number of log entries generated from the start of the startup command execution to the full startup of the service, obtains the number of log entries when the service is started, records the execution results after the automation script is executed, obtains and counts the success rate through the CI / CD tool Jenkins to obtain the success rate of the automation script execution, records the system restart completion time and the system reaches a stable state, specifically the time required for no abnormal logs and no high load, and obtains the stable time after the system restart through the script combined with the monitoring tool Grafana.

[0059] The development data analysis module is used to analyze the data transmitted by the development data entry module and transmit the analysis results to the abnormality analysis module.

[0060] refer to Figure 2The development data analysis module includes a first receiving unit, a data analysis unit and a second data transmission unit. The first receiving unit is used to receive data transmitted by the first data transmission unit and pre-process the data, specifically including data cleaning, data integration and data transformation. The data analysis unit is used to establish a code infrastructure data analysis model, a development process data analysis model, a visualization tool data analysis model and an operation and maintenance and deployment details data analysis model, which is used to analyze the data pre-processed by the first receiving unit. The second data transmission unit is used to transmit the analysis results of the data analysis unit to the abnormality analysis module.

[0061] The code infrastructure data analysis model is used to analyze the code infrastructure data transmitted by the development data entry module, which is specifically expressed as follows:

[0062] ,

[0063] KC i represents the dynamic evaluation value of the code at the i-th time point, Cr i represents the rate of change in the number of code submissions for the development task at time i, Cb i Indicates the number of intermediate products of code compilation at time point i, Cs i Indicates the time difference of the code base snapshot at the i-th time point, Cm i Represents the frequency of magic number in the code at the i-th time point, and n represents the number of time points.

[0064] The development process data analysis model is used to analyze the development process data transmitted by the development data entry module, which is specifically expressed as follows:

[0065] ,

[0066] KR i represents the efficiency value of the development process at the i-th time point, Rc i Indicates the frequency of demand change at the i-th time point, Rp i Indicates the Pull Request feedback delay at the i-th time point, Rm i represents the number of code merge conflicts between developers at the i-th time point, Rt i It represents the degree of task slicing at the i-th time point, and n represents the number of time points.

[0067] The visualization tool data analysis model is used to analyze the visualization tool data transmitted by the development data entry module, which is specifically expressed as follows:

[0068] ,

[0069] KG irepresents the interactive visualization performance evaluation value at the i-th time point, Gr i represents the graphical interface response delay at the i-th time point, Ge i represents the number of interactive errors of the visualization tool at the i-th time point, Gt i Indicates the visual completeness of the unit test results at the i-th time point, Gv i represents the visual participation in the code review process at the i-th time point, and n represents the number of time points.

[0070] The operation and maintenance and deployment details data analysis model is used to analyze the operation and maintenance and deployment details data transmitted by the development data entry module, which is specifically expressed as follows:

[0071] ,

[0072] KF i represents the operation and maintenance efficiency value at the i-th time point, Fc i Indicates the revision count of the configuration file at the i-th time point, Fl i Indicates the number of log entries when the service is started at the i-th time point, Fs i represents the success rate of the automated script execution at the i-th time point, Fr i It represents the stable time after the system restart at the i-th time point, and n represents the number of time points.

[0073] The abnormality analysis module is used to perform a comprehensive analysis on the data transmitted by the development data analysis module, and determine the abnormal state of the target business system according to the analysis results, and transmit the abnormal state of the target business system to the management module.

[0074] The abnormality analysis module includes a second receiving unit, a comprehensive analysis unit, an abnormality judgment unit and a third transmission unit. The second receiving unit is used to receive data transmitted by the second transmission unit. The comprehensive analysis unit is used to establish a comprehensive analysis model, import the data transmitted by the second transmission unit into the comprehensive analysis model, and calculate the comprehensive abnormality index of the target business system. The abnormality judgment unit is used to judge the abnormal state of the target business system according to the comprehensive abnormality index threshold. The third transmission unit is used to transmit the abnormal state of the target business system to the management module.

[0075] The comprehensive analysis model is specifically expressed as:

[0076] ,η represents the comprehensive abnormality index of the target business system, KC i ′ represents the normalized code dynamic evaluation value at the i-th time point, KR i ′ represents the standardized development process efficiency value at the i-th time point, KG i′ represents the standardized interactive visualization performance evaluation value at the i-th time point, KF i ′ represents the normalized operation and maintenance performance value at the i-th time point, w1 represents the weight of the code infrastructure data, w2 represents the weight of the development process data, w3 represents the weight of the visualization tool data, and w4 represents the weight of the operation and maintenance and deployment detail data.

[0077] It should be specifically explained in this embodiment that the standardization method in the comprehensive analysis model is the Min-Max standardization method, and w1, w2, w3 and w4 are obtained by fitting experimental data, and w1+w2+w3+w4=1.

[0078] The comprehensive abnormal index threshold includes a first threshold and a second threshold, respectively marked as T normal and T warning , and T normal <T warning , when η≤T normal When T normal <η≤T warning When T warning <η, the state of the target business system is marked as abnormal.

[0079] The management module is used to issue management instructions according to the abnormal state of the target business system.

[0080] The management module includes a third receiving unit, a first control unit, a second control unit and a third control unit. The third receiving unit is used to receive information transmitted by the third transmission unit. The first control unit is used to issue control instructions for a normal state, the second control unit is used to issue control instructions for an alert state, and the third control unit is used to issue control instructions for an abnormal state.

[0081] The first control unit keeps acquiring and analyzing the business data when the state of the target business system is normal, and records a normal state log of the target business system;

[0082] The second control unit performs a wide coverage health check twice a day when the target business system is in an alert state, reserves additional computing and network resources, prepares for expansion, and sends a yellow warning signal to management personnel;

[0083] The third control unit sets up an emergency response team when the status of the target business system is abnormal, starts troubleshooting within 15 minutes, limits all system traffic, suspends non-critical functions, activates redundant backup systems, performs business switching, and performs health checks every hour until the status is restored.

[0084] The present invention determines the time point of development data collection through the data collection time determination module, ensures the standardization and consistency of data collection, marks the data collection time point in the same time interval, and provides an accurate time dimension for subsequent data analysis and anomaly detection; collects multi-dimensional data of the target business system at various time points through the development data entry module, including code infrastructure data, development process data, visualization tool data, and operation and maintenance and deployment detail data. These data provide a rich data source for subsequent development data analysis, which helps to fully understand the development status and operation efficiency of the business system; pre-processes and deeply analyzes the data transmitted by the development data entry module through the development data analysis module, establishes multiple data analysis models, and can Quantitatively evaluate the development efficiency and effectiveness of the business system, providing a scientific basis for anomaly detection and management decisions; conduct a comprehensive analysis of the data transmitted by the development data analysis module through the anomaly analysis module, calculate the comprehensive anomaly index of the target business system, and judge the abnormal state of the business system according to the set threshold, so as to timely discover and warn of potential problems, providing an important basis for the management module to issue corresponding management instructions; the management module issues corresponding management instructions according to the abnormal state of the target business system transmitted by the anomaly analysis module, and can take corresponding control measures for different states to ensure the stable operation and efficient development of the business system. Under abnormal circumstances, the management module can quickly activate the emergency response mechanism to reduce system risks and restore the system status as soon as possible.

[0085] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0086] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A business system development and management system based on big data analysis, including a system operation database, a system central processing module and a user information terminal, characterized in that: It also includes a data collection time determination module, a development data entry module, a development data analysis module, an abnormality analysis module and a management module; The data collection time determination module is used to determine the data collection time points of the development data collection module, and the data collection time points of the development data collection module are determined in the same time interval manner, and are marked as 1, 2...n in sequence; The development data entry module is used to collect code infrastructure data, development process data, visualization tool data, and operation and deployment detail data of the target business system at each time point, and transmit the collected data to the development data analysis module; The development data entry module includes a data collection unit and a first data transmission unit, the data collection unit is used to collect code infrastructure data, development process data, visualization tool data, and operation and deployment detail data of the target business system at each time point, and the first data transmission unit is used to transmit the data collected by the data collection unit to the development data analysis module; The development data analysis module is used to analyze the data transmitted by the development data entry module and transmit the analysis results to the abnormality analysis module; The development data analysis module includes a first receiving unit, a data analysis unit and a second data transmission unit. The first receiving unit is used to receive data transmitted by the first data transmission unit and pre-process the data, specifically including data cleaning, data integration and data transformation. The data analysis unit is used to establish a code infrastructure data analysis model, a development process data analysis model, a visualization tool data analysis model and an operation and maintenance and deployment details data analysis model, which are used to analyze the data pre-processed by the first receiving unit. The second data transmission unit is used to transmit the analysis result of the data analysis unit to the abnormality analysis module; The code infrastructure data analysis model is used to analyze the code infrastructure data transmitted by the development data entry module, which is specifically expressed as follows: ， KC i represents the dynamic evaluation value of the code at the i-th time point, Cr i represents the rate of change in the number of code submissions for the development task at time i, Cb i Indicates the number of intermediate products of code compilation at time point i, Cs i Indicates the time difference of the code base snapshot at the i-th time point, Cm i represents the frequency of magic number in the code at the i-th time point, and n represents the number of time points; The development process data analysis model is used to analyze the development process data transmitted by the development data entry module, which is specifically expressed as follows: ， KR i represents the efficiency value of the development process at the i-th time point, Rc i Indicates the frequency of demand change at the i-th time point, Rp i Indicates the Pull Request feedback delay at the i-th time point, Rm i represents the number of code merge conflicts between developers at the i-th time point, Rt i represents the degree of task slicing at the i-th time point, and n represents the number of time points; The visualization tool data analysis model is used to analyze the visualization tool data transmitted by the development data entry module, which is specifically expressed as follows: ， KG i represents the interactive visualization performance evaluation value at the i-th time point, Gr i represents the graphical interface response delay at the i-th time point, Ge i represents the number of interactive errors of the visualization tool at the i-th time point, Gt i Indicates the visual completeness of the unit test results at the i-th time point, Gv i represents the visual participation in the code review process at the i-th time point, and n represents the number of time points; The operation and maintenance and deployment details data analysis model is used to analyze the operation and maintenance and deployment details data transmitted by the development data entry module, which is specifically expressed as follows: ， KF i represents the operation and maintenance efficiency value at the i-th time point, Fc i Indicates the revision count of the configuration file at the i-th time point, Fl i Indicates the number of log entries when the service is started at the i-th time point, Fs i represents the success rate of the automated script execution at the i-th time point, Fr i represents the stable time after the system restart at the i-th time point, n represents the number of time points, and Fs max Indicates the maximum success rate of automated script execution of the target business system; The abnormality analysis module is used to perform a comprehensive analysis on the data transmitted by the development data analysis module, and determine the abnormal state of the target business system according to the analysis results, and transmit the abnormal state of the target business system to the management module; The abnormality analysis module includes a second receiving unit, a comprehensive analysis unit, an abnormality judgment unit and a third transmission unit, the second receiving unit is used to receive data transmitted by the second transmission unit, the comprehensive analysis unit is used to establish a comprehensive analysis model, import the data transmitted by the second transmission unit into the comprehensive analysis model, calculate the comprehensive abnormality index of the target business system, the abnormality judgment unit is used to judge the abnormal state of the target business system according to the comprehensive abnormality index threshold, and the third transmission unit is used to transmit the abnormal state of the target business system to the management module; The comprehensive analysis model is specifically expressed as: ,η represents the comprehensive abnormality index of the target business system, KC i ′ represents the normalized code dynamic evaluation value at the i-th time point, KR i ′ represents the standardized development process efficiency value at the i-th time point, KG i ′ represents the standardized interactive visualization performance evaluation value at the i-th time point, KF i ′ represents the normalized O&M performance value at the i-th time point, w1 represents the weight of the code infrastructure data, w2 represents the weight of the development process data, w3 represents the weight of the visualization tool data, and w4 represents the weight of the O&M and deployment detail data; The comprehensive abnormal index threshold includes a first threshold and a second threshold, respectively marked as T normal and T warning , and T normal <T warning , when η≤T normal When T normal <η≤T warning When T warning <η, the state of the target business system is marked as abnormal; The management module is used to issue management instructions according to the abnormal state of the target business system; The management module includes a third receiving unit, a first control unit, a second control unit and a third control unit. The third receiving unit is used to receive information transmitted by the third transmission unit. The first control unit is used to issue control instructions for a normal state, the second control unit is used to issue control instructions for an alert state, and the third control unit is used to issue control instructions for an abnormal state.

2. The business system development and management system based on big data analysis according to claim 1, characterized in that: The code infrastructure data includes the rate of change of the number of code submissions for development tasks, the number of intermediate products of code compilation, the time difference of code base snapshots, and the frequency of magic numbers in the code, which are marked as Cr, Cb, Cs, and Cm respectively. The development process data includes the frequency of requirement changes, the delay time of Pull Request feedback, the number of code merge conflicts between developers, and the degree of task slicing, which are marked as Rc, Rp, Rm, and Rt respectively. The visualization tool data includes the response delay of the graphical interface, the number of interactive errors of the visualization tool, the visualization completeness of the unit test results, and the visualization participation in the code review process, which are marked as Gr, Ge, Gt, and Gv respectively. The operation and maintenance and deployment detail data includes the number of revisions of the configuration file, the number of log entries at service startup, the success rate of automated script execution, and the stabilization time after system restart, which are marked as Fc, Fl, Fs, and Fr respectively.

3. The business system development and management system based on big data analysis according to claim 1, characterized in that: The first control unit keeps acquiring and analyzing the business data when the state of the target business system is normal, and records a normal state log of the target business system; The second control unit performs a wide coverage health check twice a day when the target business system is in an alert state, reserves additional computing and network resources, prepares for expansion, and sends a yellow warning signal to management personnel; The third control unit sets up an emergency response team when the status of the target business system is abnormal, starts troubleshooting within 15 minutes, limits all system traffic, suspends non-critical functions, activates redundant backup systems, performs business switching, and performs health checks every hour until the status is restored.

Images