Method and apparatus for determining software running state

Abstract

This application discloses a method and apparatus for determining the running state of software, relating to the field of software technology. The method includes: acquiring multiple running data points of a target software during its operation; inputting the multiple running data points into a target model to obtain the processing result of the target model on the multiple running data points, wherein the processing result includes the target software type of the target software and at least one target indicator data point corresponding to the target software type, the target indicator data point being data corresponding to an indicator used to evaluate the running state of the target software; and determining the running state of the target software based on each target indicator data point and the target threshold corresponding to that target indicator data point. This application solves the technical problem of low efficiency in determining the running state of software in the prior art.

Description

Technical Field

[0001] This application relates to the field of software technology, and more specifically, to a method and apparatus for determining the running state of software. Background Technology

[0002] In existing technologies, different types of software use different metrics to determine their operating status. For example, for mobile payment software, metrics such as user click-through rate and number of online users are important indicators for measuring the software's operating status. However, for office software used internally by enterprises, since the number of users is relatively fixed, the number of online users and user click-through rate are no longer important indicators for measuring the operating status of office software. Instead, metrics such as the number of real-time services and the number of batch services are used to measure the operating status of office software.

[0003] In existing technologies, the software type is usually identified manually, and the corresponding indicator data is extracted. Then, the operating status of the software is determined based on the indicator data. Since the whole process requires a lot of manual labor and time, the efficiency of determining the operating status of the software is low.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and apparatus for determining the running state of software, so as to at least solve the technical problem of low efficiency in determining the running state of software in the prior art.

[0006] According to one aspect of the embodiments of this application, a method for determining the running state of software is provided, comprising: acquiring multiple running data of a target software during its operation; inputting the multiple running data into a target model to obtain the processing result of the target model on the multiple running data, wherein the processing result includes the target software type of the target software and at least one target indicator data corresponding to the target software type, the target indicator data being data corresponding to an indicator used to evaluate the running state of the target software; and determining the running state of the target software based on each target indicator data and a target threshold corresponding to the target indicator data.

[0007] Furthermore, the target software type includes any one of the following software types: a first software type, wherein the software of the first software type is used to serve a preset user group; a second software type, wherein the software of the second software type is used to perform batch data calculation; and a third software type, wherein the software of the third software type is used to process the data.

[0008] Furthermore, the target model described above is trained through the following process: acquiring multiple historical operating data, wherein the multiple historical operating data include at least the first operating data of the first software, the second operating data of the second software, and the third operating data of the third software, wherein the software type of the first software is the first software type, the software type of the second software is the second software type, and the software type of the third software is the third software type; and training the target model based on the multiple historical operating data.

[0009] Furthermore, the method for determining the software's operating state also includes: identifying at least one first indicator data in the first operating data, wherein each first indicator data corresponds to a first indicator, and the first indicator is an indicator used to evaluate the operating state of the first software based on the application scenario of the first software; identifying at least one second indicator data in the second operating data, wherein each second indicator data corresponds to a second indicator, and the second indicator is an indicator used to evaluate the operating state of the second software based on the application scenario of the second software; identifying at least one third indicator data in the third operating data, wherein each third indicator data corresponds to a third indicator, and the third indicator is an indicator used to evaluate the operating state of the third software based on the application scenario of the third software; and training a target model based on each first indicator data, each second indicator data, and each third indicator data.

[0010] Furthermore, the method for determining the software operating status also includes: generating a first label for each first indicator data based on the first indicator and the first software type corresponding to each first indicator data; generating a second label for each second indicator data based on the second indicator and the second software type corresponding to each second indicator data; generating a third label for each third indicator data based on the third indicator and the third software type corresponding to each third indicator data; and training a target model based on each first indicator data, each second indicator data, each third indicator data, and the target label, wherein the target label includes the first label, the second label, and the third label.

[0011] Furthermore, the method for determining the software operating status also includes: when there is abnormal indicator data in at least one target indicator data, the operating status of the target software is determined to be abnormal, wherein the abnormal indicator data is the target indicator data that is less than the corresponding target threshold; when each target indicator data is greater than or equal to the target threshold corresponding to the target indicator data, the operating status of the target software is determined to be normal.

[0012] Furthermore, the method for determining the software operating status also includes: after acquiring multiple operating data of the target software during operation, identifying at least one target operating data among the multiple operating data, wherein the at least one target operating data includes at least the number of failures of the target software within a preset time period, the number of functions that the target software can provide, the response speed of the target software, and the user activity of the target software within the preset time period; and determining the operating status of the target software based on the at least one target operating data.

[0013] Furthermore, the method for determining the software's operating status also includes: determining the target interval corresponding to each target operating data from a set of preset numerical intervals, wherein each numerical interval corresponds to a preset score; determining the score of each target operating data based on the preset score corresponding to the target interval, thereby obtaining at least one score; obtaining the weight value corresponding to each target operating data; performing a weighted summation on the at least one score based on the weight value to obtain the target score; and determining the operating status of the target software based on the target score.

[0014] Furthermore, the method for determining the software's operating status also includes: if the target score is less than the preset score, the operating status of the target software is determined to be abnormal; if the target score is greater than or equal to the preset score, the operating status of the target software is determined to be normal.

[0015] According to another aspect of the embodiments of this application, a device for determining the running state of software is also provided, comprising: an acquisition module for acquiring multiple running data of a target software during its operation; an input module for inputting the multiple running data into a target model to obtain the processing result of the target model on the multiple running data, wherein the processing result includes the target software type of the target software and at least one target indicator data corresponding to the target software type, the target indicator data being data corresponding to an indicator used to evaluate the running state of the target software; and a determination module for determining the running state of the target software based on each target indicator data and a target threshold corresponding to the target indicator data.

[0016] In this application, a method is adopted to identify the target software type and at least one target indicator data corresponding to the target software type through a target model. First, multiple operational data points of the target software during operation are acquired. Then, these operational data points are input into the target model to obtain the processing results of the target model on the multiple operational data points. The processing results include the target software type and at least one target indicator data corresponding to the target software type. The target indicator data are the data corresponding to the indicators used to evaluate the operational status of the target software. Finally, the operational status of the target software is determined based on each target indicator data point and the target threshold corresponding to that target indicator data.

[0017] As described above, this application identifies software types using a pre-trained target model and identifies target indicator data from multiple operational data sets based on the software type. This allows for the automatic acquisition of indicator data corresponding to different software types when processing operational data, and the automatic detection of the software's operational status based on this indicator data. Because this application avoids the need for manual classification of software and identification of indicator data for each type, as required by existing technologies, it saves significant labor costs and improves the efficiency of determining software operational status.

[0018] Therefore, the technical solution of this application achieves the purpose of automatically identifying the software type and determining the target index data of the software based on the software type, thereby reducing the cost of manual identification and solving the technical problem of low efficiency in determining the software running status in the prior art. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a flowchart of an optional method for determining the software running state according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of an optional data tracking module according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of an optional product operation indicator module according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an optional indicator information synchronization module according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an optional data processing module according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of an optional operation push module according to an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of an optional model training process according to an embodiment of this application;

[0027] Figure 8 This is a flowchart of another method for determining the software running state according to an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of an optional software operation system according to an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of an optional software running state determination device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] As competition intensifies in the software industry, operational analysis plays a crucial role in creating value through technology. To continuously develop excellent software products and enhance industry competitiveness, it's essential to analyze software operation status to determine its application value and propose optimization and feature retirement suggestions. Furthermore, to meet market demands, software iterations are becoming increasingly frequent. Therefore, software developers must continuously identify software shortcomings and competitive advantages to enhance their core competitiveness.

[0034] However, different types of software use different metrics to determine their operational status. For example, for mobile payment software launched by banks for all users, metrics such as user click-through rate and number of online users are important indicators for measuring the software's operational status. If the user click-through rate and number of online users are both high, it indicates that the software has high user acceptance and is highly competitive. If the user click-through rate and number of online users are low, it indicates that the software has low user acceptance and is less competitive.

[0035] For internal management software used within banks, such as attendance software, since the number of users is relatively fixed, the number of online users and the click-through rate are no longer important indicators for measuring the software's operational status. Instead, metrics such as real-time service volume and batch service volume are needed to measure the software's operational status. Higher real-time service volume and batch service volume indicate that the attendance software can support a large number of users accessing it in a short period of time, giving it a competitive edge among similar attendance software.

[0036] Based on the above, it is clear that different types of software use different indicators to determine their operational status. Therefore, determining the corresponding indicator data for each software type is crucial to improving the accuracy of software operational status determination. Currently, existing technologies typically employ manual methods to identify software types and determine the corresponding indicator data. This method incurs significant labor costs, resulting in low efficiency in determining software operational status.

[0037] To address the aforementioned problems, this application provides an embodiment of a method for determining the running state of software. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] In addition, it should be noted that a software operation system can serve as the execution subject of the method for determining the software running state in the embodiments of this application.

[0039] Figure 1 This is a flowchart of an optional method for determining the software running state according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0040] Step S101: Obtain multiple runtime data of the target software during its operation.

[0041] In step S101, the software operating system includes at least a data tracking module. This data tracking module acquires multiple operational data points of the target software during its operation using tracking technology.

[0042] Specifically, Figure 2 This is a schematic diagram of an optional data tracking module according to an embodiment of this application, as shown below. Figure 2 As shown, the data tracking module includes a tracking field setting unit, a classification unit, a tracking method setting unit, a tracking type setting unit, a data upload unit, a data preprocessing unit, and a data lake unit.

[0043] The tracking field design unit is used to design tracking fields and retrieve tracking records based on these fields. Specifically, there is a one-to-one correspondence between tracking records and user actions; one tracking record corresponds to one user action. Furthermore, tracking records should be complete, allowing the parsing of multiple pieces of software runtime data, including but not limited to user information, business information, interaction behavior information, and device environment information. In addition, the naming of tracking fields needs to be complete, accurate, and self-descriptive.

[0044] The classification unit is used to categorize the obtained operational data. Specifically, the classification unit can categorize user information by user ID; it can categorize business information by unique element identifiers in the business information, including but not limited to page ID and function ID; the classification unit can also categorize related fields in the interaction behavior information by the behavior source information contained in the interaction behavior information, where the behavior source information includes but is not limited to session ID, previous page number or path, source channel identifier, interaction time, etc. In addition, the classification unit can also divide the above-mentioned device environment information into device information and client information, where device information includes device unique ID, operating system, mobile phone model, etc., and client information includes client version number, client current wake-up time, client first startup time, etc.

[0045] The event tracking method setting unit is used to set the event tracking method. Event tracking can be divided into code-based event tracking and code-free event tracking.

[0046] The event tracking type setting unit is used to set the event tracking type. Event tracking types include page-based event tracking and control-based event tracking. Page-based event tracking is mainly used for internal management software with explicit page operation information, while control-based event tracking is mainly used for data platform software, which provides underlying service support.

[0047] The data upload unit is used to upload the collected operational data via the standard HTTP protocol. During upload, the operational data needs to be compressed to control the size of the data packets. Compression can be performed based on dimensions such as time and number of records.

[0048] The data preprocessing unit is used to process fields with special requirements in the uploaded running data. For example, during the upload process, the data preprocessing unit performs URL encoding on Chinese fields.

[0049] The data lake unit is used to receive and store operational data.

[0050] Step S102: Input multiple running data into the target model to obtain the processing results of the target model on the multiple running data.

[0051] In step S102, the processing result includes the target software type of the target software and at least one target indicator data corresponding to the target software type. The target indicator data is the data corresponding to the indicator used to evaluate the operating status of the target software.

[0052] Optionally, the target model mentioned above is a model trained using a machine learning algorithm, and the target software type mentioned above includes any one of the following software types: a first software type, wherein the software of the first software type is used to serve a preset user group; a second software type, wherein the software of the second software type is used to perform batch data calculation; and a third software type, wherein the software of the third software type is used to process the data.

[0053] Specifically, the first software type is internal management software, such as attendance software, payroll software, contract management software, email software, and other non-corporate software used internally by banks. Internal management software only serves a predetermined user group, such as internal employees. Furthermore, the corresponding metrics for internal management software include, but are not limited to, daily active user rate, click-through rate at application entry points, number of registered users, and usage of core functions.

[0054] The second software type is data platform software. This type of software provides underlying service support and is mainly used for batch data computation. The corresponding metrics for data platform software include, but are not limited to, the number of data models deployed, the number of data model training iterations, the number of real-time services, the number of batch services, the number of integrated algorithms, the number of platform users, the number of deployment scenarios, and the number of user operations.

[0055] The third type of software is data asset software. This type of software is used to process data, such as generating knowledge graphs based on the data. The corresponding indicator data for data asset software includes, but is not limited to, the amount of source data, the amount of topic-aggregated data, statistical indicator data, knowledge graphs, and data services.

[0056] Step S103: Determine the operating status of the target software based on each target indicator data and the target threshold corresponding to that target indicator data.

[0057] In step S103, each target metric data has a corresponding target threshold. For example, if the target software is a software of a first software type, its corresponding at least one metric data includes daily active user rate, number of clicks on the application entry point, number of registered users, and core function usage. The target threshold corresponding to the daily active user rate is threshold A, the target threshold corresponding to the number of clicks on the application entry point is threshold B, the target threshold corresponding to the number of registered users is threshold C, and the target threshold corresponding to the core function usage is threshold D.

[0058] In addition, the software operation system will detect the relationship between each target indicator data and the corresponding target threshold, and determine the operating status of the target software based on the detection results. For example, if any target indicator data is less than the corresponding target threshold, the software operation system will determine that the operating status of the target software is abnormal, that is, the operating status of the target software does not meet the requirements of the operators, and the software value of the target software is low.

[0059] Based on steps S101 to S103, this application employs a method of identifying the target software type and at least one target indicator data corresponding to the target software type through a target model. First, multiple operational data points of the target software during operation are acquired. Then, these multiple operational data points are input into the target model to obtain the processing results of the target model on the multiple operational data points. The processing results include the target software type of the target software and at least one target indicator data corresponding to the target software type. The target indicator data refers to the data corresponding to the indicators used to evaluate the operational status of the target software. Finally, the operational status of the target software is determined based on each target indicator data point and the target threshold corresponding to that target indicator data.

[0060] As described above, this application identifies software types using a pre-trained target model and identifies target indicator data from multiple operational data sets based on the software type. This allows for the automatic acquisition of indicator data corresponding to different software types when processing operational data, and the automatic detection of the software's operational status based on this indicator data. Because this application avoids the need for manual classification of software and identification of indicator data for each type, as required by existing technologies, it saves significant labor costs and improves the efficiency of determining software operational status.

[0061] Therefore, the technical solution of this application achieves the purpose of automatically identifying the software type and determining the target index data of the software based on the software type, thereby reducing the cost of manual identification and solving the technical problem of low efficiency in determining the software running status in the prior art.

[0062] In an optional embodiment, the target model in this application can be trained through the following process:

[0063] First, the software operation system acquires multiple historical operation data points through the data tracking module. These historical operation data points include at least the first operation data of the first software, the second operation data of the second software, and the third operation data of the third software, where the software type of the first software is designated as first software type, the software type of the second software is designated as second software type, and the software type of the third software is designated as third software type. Then, the software operation system trains the target model based on these historical operation data points.

[0064] Specifically, the software operation system identifies at least one first indicator data point in the first operational data, at least one second indicator data point in the second operational data, and at least one third indicator data point in the third operational data. Each first indicator data point corresponds to a first indicator, which is an indicator used to evaluate the operational status of the first software based on its application scenario; each second indicator data point corresponds to a second indicator, which is an indicator used to evaluate the operational status of the second software based on its application scenario; and each third indicator data point corresponds to a third indicator, which is an indicator used to evaluate the operational status of the third software based on its application scenario. Then, the software operation system trains a target model based on each of the first, second, and third indicator data points.

[0065] Optionally, the software operation system needs to call the product operation indicator module when identifying indicator data in the operational data. For example, Figure 3As shown, the product operation metrics module includes a product operation metric setting unit and an information storage unit. Specifically, users can use the product operation metric setting unit to analyze user needs, user satisfaction feedback, key functionalities, etc., to obtain metrics corresponding to different software types. For example, based on software type, software can be divided into internal management software, data platform software, and data asset software. The metrics for each type of software are as follows:

[0066] The primary metrics for internal management software (software type 1) are: daily active users, number of clicks on the application entry point, number of registered users, and usage of core functions (e.g., number of instant messaging messages, number of mobile emails, number of to-do items, etc.).

[0067] The second indicator for data platform software (software type is the second software type) includes: number of models deployed, number of model training sessions, number of real-time services, number of batch services, number of integrated algorithms, number of platform users, number of deployment scenarios, and number of user operations.

[0068] The third indicator for data asset software (software type 2) includes: source data volume, topic aggregated data volume, statistical indicator data, knowledge graph, data services, etc.

[0069] In addition, the indicator information storage unit is used to store the indicator information formulated by the product operation indicator formulation unit. The indicator information includes, but is not limited to, the software name corresponding to the indicator, the system name corresponding to the indicator, the system module name corresponding to the indicator, the indicator type, and the indicator name.

[0070] Furthermore, the software operation system also includes a module for synchronizing indicator information. For example... Figure 4 As shown, the indicator information synchronization module includes an information acquisition unit, an information parsing unit, and an information storage unit. When multiple historical running data sets are needed to train the target model, the information acquisition unit first obtains running data for different software types from the data lake unit. Then, the information parsing unit calls the aforementioned indicator information storage unit and parses the running data for different software types to obtain indicator data and non-indicator data from the running data. Specifically, the information acquisition unit acquires at least the first running data of the first software, the second running data of the second software, and the third running data of the third software. The information parsing unit can obtain at least one first indicator data set from the first running data, at least one second indicator data set from the second running data, and at least one third indicator data set from the third running data. Additionally, the information storage unit is used to classify and store the indicator data and non-indicator data identified by the information parsing unit.

[0071] In an optional embodiment, the software operation system further generates a first label for each first indicator data point based on the first indicator and the first software type corresponding to each first indicator data point; generates a second label for each second indicator data point based on the second indicator and the second software type corresponding to each second indicator data point; and generates a third label for each third indicator data point based on the third indicator and the third software type corresponding to each third indicator data point. Finally, the software operation system trains a target model based on each first indicator data point, each second indicator data point, each third indicator data point, and the target label, wherein the target label includes the first label, the second label, and the third label.

[0072] Optional, such as Figure 5 As shown, the software operation system also includes a data processing module, which includes an indicator configuration unit, a clustering unit, an indicator processing unit, and a data middle platform unit.

[0073] Specifically, the indicator configuration unit is used to manage the indicators used in the model training process, such as deleting or adding indicators. The indicator configuration unit also configures the indicator processing logic to facilitate the processing of indicators, such as modifying indicator names, setting indicator processing frequency, processing dimensions, and other information.

[0074] The clustering unit is used to generate labels for each metric data point. Specifically, the clustering unit can first randomly select 20% of the metric data stored in the information storage unit for classification, and then generate corresponding labels for each metric data point. The clustering unit will determine the software type and metric corresponding to each metric data point. For example, if the software type corresponding to a metric data point is the first software type, and the metric data describes the number of registered users, then the clustering unit will generate a label for the metric data point as "first software type, number of registered users metric".

[0075] In addition, the clustering unit will extract features for each indicator data. The extracted features can characterize the software type and indicator corresponding to the indicator data. For example, the first indicator data includes features such as the number of registered users, the number of communication messages, and the number of mobile emails; the second indicator data includes features such as the number of models launched, the number of API calls, and the number of platform users; and the third indicator data includes features such as the number of post sources and the number of topic aggregations.

[0076] Finally, the clustering unit continuously trains based on the feature data extracted from each indicator data and the corresponding label for each indicator data. It then randomly selects 60% of the indicator data from the information storage unit for label validation. Of this, 70% of the label validation data is used to train and construct the target model, 20% is used to test the trained target model, and 10% is used to perform final validation by adjusting parameters. This process continuously improves the accuracy and robustness of the target model, ultimately resulting in a target model that can accurately identify the software type of the input indicator data and determine which indicator it corresponds to.

[0077] In addition, the indicator processing unit is used to obtain specific indicator data through the clustering unit, and in combination with the indicator processing logic configured in the indicator configuration unit, process the indicator data, and store the processing results in the data platform unit.

[0078] In one optional embodiment, when there is abnormal indicator data in at least one target indicator data, the software operation system determines that the operating state of the target software is abnormal, wherein the abnormal indicator data is the target indicator data that is less than the corresponding target threshold; when each target indicator data is greater than or equal to the target threshold corresponding to the target indicator data, the software operation system determines that the operating state of the target software is normal.

[0079] Optionally, the software operation system also includes an operation push module, wherein, Figure 6 A schematic diagram of an optional operation push module according to an embodiment of this application is shown, such as... Figure 6 As shown, the operation push module includes an indicator monitoring unit, an indicator threshold configuration unit, an early warning result push unit, and a result storage unit.

[0080] The indicator monitoring unit is used to monitor at least one target indicator data identified by the target model and store the monitored data.

[0081] The indicator threshold configuration unit is used to configure a corresponding threshold for each indicator. For example, each target indicator corresponding to the target indicator data has a corresponding target threshold. It should be noted that the indicator threshold configuration unit can configure single-indicator thresholds and multi-indicator thresholds. A single-indicator threshold means that a threshold can be set for each indicator. For example, for at least one target indicator data output by the target model, if any one of the at least one target indicator data is less than its corresponding target threshold, the software operation system determines that the target software's operating status is abnormal and generates a warning message. If each of the at least one target indicator data is greater than or equal to its corresponding target threshold, the software operation system determines that the target software's operating status is normal.

[0082] The early warning result push unit is used to generate the above-mentioned early warning information and push the early warning information to product managers and other personnel, so that product managers can understand the reasons for the poor current software operation status in a timely manner and give reasonable measures to improve the value of the software.

[0083] The results storage unit is used to store the early warning results so that subsequent problem tracing can be completed.

[0084] In one optional embodiment, after acquiring multiple operational data points of the target software during its operation, the software operation system further identifies at least one target operational data point from among the multiple operational data points. This at least one target operational data point includes at least the number of failures of the target software within a preset time period, the number of functions the target software can provide, the response speed of the target software, and the user activity level of the target software within the preset time period. Then, the software operation system determines the operational status of the target software based on the at least one target operational data point.

[0085] Optionally, the multiple operational data may also include at least product characteristic data and product development characteristic data. Among them, product characteristic data includes, but is not limited to, software function data, software performance data, software security data, software stability data, and software usability data of the target software.

[0086] Specifically, software functionality data refers to the number of functions the target software can provide; software performance data refers to the response speed of the target software; software security data refers to the number of security vulnerabilities in the target software; software stability data refers to the number of failures of the target software within a preset time period; and software usability data refers to the number of user complaints received by the target software within a preset time period.

[0087] In addition, product development characteristic data includes, but is not limited to, user activity, product release frequency, access volume, and user satisfaction of the target software within a preset time period.

[0088] Among these metrics, product activity refers to the online time of users within a preset time period for the target software. Product release frequency refers to the monthly update frequency of the target software. Page views refer to the daily page views of the target software. User satisfaction refers to user satisfaction ratings when using the target software.

[0089] In one alternative embodiment, such as Figure 7As shown, during the training process of the target model, the aforementioned product feature data and product R&D feature data can also be added to the model training process. Thus, after the software operation system inputs multiple operational data into the target model, the target model can output at least one target indicator data point, as well as at least one target operational data point. This at least one target operational data point includes at least the number of failures of the target software within a preset time period, the number of functions provided by the target software, the response speed of the target software, the user activity level of the target software within the preset time period, the number of security vulnerabilities in the target software, the number of user complaints received by the target software within the preset time period, the product release frequency of the target software within the preset time period, the access volume of the target software within the preset time period, and the user satisfaction level of the target software within the preset time period.

[0090] In addition, after obtaining at least one target operating data, the software operation system can detect whether each target operating data is less than the preset threshold corresponding to each target operating data, obtain the detection result, and determine the operating status of the target software based on the detection result.

[0091] For example, when there is abnormal operating data in at least one target operating data, the operating state of the target software is determined to be abnormal, wherein the abnormal operating data is the target operating data that is less than the corresponding preset threshold.

[0092] In an optional embodiment, the software operation system can further determine the operating status of the target software by comprehensively evaluating at least one target operating data point. Specifically, the software operation system determines a target interval corresponding to each target operating data point from a plurality of preset numerical intervals, wherein each numerical interval corresponds to a preset score. Then, the software operation system determines a score for each target operating data point based on the preset score corresponding to the target interval, obtaining at least one score. Further, the software operation system also obtains a weight value corresponding to each target operating data point, and performs a weighted summation of the at least one score based on the weight value to obtain a target score. Finally, the software operation system determines the operating status of the target software based on the target score.

[0093] If the target score is less than the preset score, the software operation system determines the target software's operating status as abnormal; if the target score is greater than or equal to the preset score, the software operation system determines the target software's operating status as normal.

[0094] Optionally, Table 1 shows a target software scoring table for comprehensive evaluation based on at least one target operating data, as shown in Table 1:

[0095] Table 1

[0096]

[0097]

[0098] In one alternative embodiment, Figure 8 A flowchart illustrating another method for determining software operating state according to an embodiment of this application is shown, such as... Figure 8 As shown, it includes the following steps:

[0099] Step S801: Analyze valuable metrics based on user product usage and implement tracking points in the software based on these metrics.

[0100] Step S802: Collect indicator data through embedded points.

[0101] Step S803: The acquired indicator data is automatically classified according to rule matching and machine learning algorithms, and indicator data corresponding to different software types are filtered out.

[0102] Step S804: Obtain product feature data and product R&D feature data, and train the target model based on the product feature data, product R&D feature data and indicator data.

[0103] Step S805: Use the target model to identify at least one target indicator data and at least one target operation data of the target software, and then visualize them.

[0104] In one alternative embodiment. Figure 9 A schematic diagram of an optional software operation system according to an embodiment of this application is shown, such as... Figure 9 As shown, the software operating system includes:

[0105] The Product Operation Metrics (MOM) module is used to obtain metric information. Product managers can use this module to analyze metrics for customized enterprise products, such as internal management products, data platform products, and data asset products, by combining user needs, user satisfaction feedback, and key functionalities, and obtain operational metrics that can measure the value of the product.

[0106] The data tracking module is used to maintain tracking-related information, including tracking field design, tracking field classification, tracking method, tracking type, data upload, data preprocessing, and other related processing.

[0107] The indicator information synchronization module is used to acquire indicator data, including product name, functional module, indicator type, indicator name, and other indicator data, providing a data source for indicator processing.

[0108] The data processing module is used to process indicator data according to certain logic.

[0109] The model analysis module is used to determine the running status of the target software based on the processing results output by the target model.

[0110] The support system module is used to display at least one target indicator data and at least one target operation data output by the target model through visual charts, so that solution users can more intuitively understand the operating status of the target software.

[0111] The operation push module is used to push the running status of the target software to the terminal device corresponding to the target object, so that the target object can improve the target software.

[0112] As described above, this application identifies software types using a pre-trained target model and identifies target indicator data from multiple operational data sets based on the software type. This allows for the automatic acquisition of indicator data corresponding to different software types when processing operational data, and the automatic detection of the software's operational status based on this indicator data. Because this application avoids the need for manual classification of software and identification of indicator data for each type, as required by existing technologies, it saves significant labor costs and improves the efficiency of determining software operational status.

[0113] Example 2

[0114] According to an embodiment of this application, an embodiment of a software running state determination device is also provided, such as... Figure 10 As shown, the device includes: an acquisition module 1001, used to acquire multiple operational data of the target software during its operation; an input module 1002, used to input the multiple operational data into a target model to obtain the processing result of the target model on the multiple operational data, wherein the processing result includes the target software type of the target software and at least one target indicator data corresponding to the target software type, the target indicator data being the data corresponding to an indicator used to evaluate the operational status of the target software; and a determination module 1003, used to determine the operational status of the target software based on each target indicator data and the target threshold corresponding to the target indicator data.

[0115] Optionally, the target software type includes any one of the following software types: a first software type, wherein the software of the first software type is used to serve a preset user group; a second software type, wherein the software of the second software type is used to perform batch data calculation; and a third software type, wherein the software of the third software type is used to process the data.

[0116] Optionally, the software running status determination device further includes: a first acquisition module and a training module. The first acquisition module is used to acquire multiple historical running data sets, wherein the multiple historical running data sets include at least first running data of a first software, second running data of a second software, and third running data of a third software, wherein the software type of the first software is the first software type, the software type of the second software is the second software type, and the software type of the third software is the third software type; the training module is used to train a target model based on the multiple historical running data sets.

[0117] Optionally, the training module further includes: a first identification unit, a second identification unit, a third identification unit, and a first training unit. The first identification unit is used to identify at least one first indicator data in the first running data, wherein each first indicator data corresponds to a first indicator, and the first indicator is an indicator used to evaluate the running status of the first software based on the application scenario of the first software; the second identification unit is used to identify at least one second indicator data in the second running data, wherein each second indicator data corresponds to a second indicator, and the second indicator is an indicator used to evaluate the running status of the second software based on the application scenario of the second software; the third identification unit is used to identify at least one third indicator data in the third running data, wherein each third indicator data corresponds to a third indicator, and the third indicator is an indicator used to evaluate the running status of the third software based on the application scenario of the third software; and the first training unit is used to train a target model based on each first indicator data, each second indicator data, and each third indicator data.

[0118] Optionally, the first training unit further includes: a first generation subunit, a second generation subunit, a third generation subunit, and a first training subunit. The first generation subunit is used to generate a first label for each first indicator data point based on the first indicator and the first software type corresponding to each first indicator data point; the second generation subunit is used to generate a second label for each second indicator data point based on the second indicator and the second software type corresponding to each second indicator data point; the third generation subunit is used to generate a third label for each third indicator data point based on the third indicator and the third software type corresponding to each third indicator data point; and the first training subunit is used to train a target model based on each first indicator data point, each second indicator data point, each third indicator data point, and the target label, wherein the target label includes the first label, the second label, and the third label.

[0119] Optionally, the detection module further includes: a first determining unit and a second determining unit. The first determining unit is used to determine that the operating state of the target software is abnormal when at least one target indicator data contains abnormal indicator data, wherein the abnormal indicator data is target indicator data that is less than the corresponding target threshold. The second determining unit is used to determine that the operating state of the target software is normal when each target indicator data is greater than or equal to the target threshold corresponding to that target indicator data.

[0120] Optionally, the device for determining the software operating status further includes: an identification module and a first determination module. The identification module is used to identify at least one target operating data among multiple operating data, wherein the at least one target operating data includes at least the number of failures of the target software within a preset time period, the number of functions that the target software can provide, the response speed of the target software, and the user activity level of the target software within the preset time period; the first determination module is used to determine the operating status of the target software based on the at least one target operating data.

[0121] Optionally, the first determining module further includes: a third determining unit, a fourth determining unit, a first acquiring unit, a summing unit, and a fifth determining unit. The third determining unit is used to determine the target interval corresponding to each target running data from a plurality of preset numerical intervals, wherein each numerical interval corresponds to a preset score; the fourth determining unit is used to determine the score of each target running data according to the preset score corresponding to the target interval, obtaining at least one score; the first acquiring unit is used to acquire the weight value corresponding to each target running data; the summing unit is used to perform a weighted summation of the at least one score according to the weight value, obtaining the target score; and the fifth determining unit is used to determine the running status of the target software based on the target score.

[0122] Optionally, the fifth determining unit further includes: a first determining subunit and a second determining subunit. The first determining subunit is used to determine the target software's operating state as abnormal if the target score is less than a preset score; the second determining subunit is used to determine the target software's operating state as normal if the target score is greater than or equal to the preset score.

[0123] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0124] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0129] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of determining a software running state, characterized by, include: Obtain multiple runtime data points of the target software during its operation; The multiple operational data are input into the target model to obtain the processing result of the target model on the multiple operational data. The processing result includes the target software type of the target software and at least one target indicator data corresponding to the target software type. The target indicator data is the data corresponding to the indicator used to evaluate the operational status of the target software. The operating status of the target software is determined based on each target indicator data and the target threshold corresponding to that target indicator data; The target software type includes any one of the following software types: A first software type, wherein the software of the first software type is used to serve a preset user group; The second software type, wherein the software of the second software type is used for batch calculation of data; The third software type, wherein the software of the third software type is used for data processing.

2. The method of claim 1, wherein, The target model is trained through the following process: Acquire multiple historical running data, wherein the multiple historical running data include at least the first running data of the first software, the second running data of the second software, and the third running data of the third software, wherein the software type of the first software is the first software type, the software type of the second software is the second software type, and the software type of the third software is the third software type; The target model is obtained by training based on the aforementioned historical operational data.

3. The method of claim 2, wherein, The target model is trained based on the aforementioned historical operational data, including: Identify at least one first indicator data in the first running data, wherein each first indicator data corresponds to a first indicator, and the first indicator is an indicator for evaluating the running status of the first software based on the application scenario of the first software. Identify at least one second indicator data in the second running data, wherein each second indicator data corresponds to a second indicator, the second indicator being an indicator for evaluating the running status of the second software based on the application scenario of the second software; Identify at least one third indicator data in the third operating data, wherein each third indicator data corresponds to a third indicator, and the third indicator is an indicator for evaluating the operating status of the third software based on the application scenario of the third software. The target model is trained based on each of the first indicator data, each of the second indicator data, and each of the third indicator data.

4. The method of claim 3, wherein, The target model is trained based on the first indicator data, the second indicator data, and the third indicator data, including: A first label is generated for each first indicator data based on the first indicator and the first software type corresponding to each first indicator data. A second label is generated for each second indicator data based on the second indicator and the second software type corresponding to each second indicator data. A third label is generated for each third indicator data based on the third indicator and the third software type corresponding to each third indicator data. The target model is trained based on each first indicator data, each second indicator data, each third indicator data, and the target label, wherein the target label includes the first label, the second label, and the third label.

5. The method of claim 1, wherein, The operating status of the target software is determined based on each target indicator data and the corresponding target threshold, including: When abnormal indicator data exists in at least one of the target indicator data, the operating state of the target software is determined to be abnormal, wherein the abnormal indicator data is the target indicator data that is less than the corresponding target threshold. When each target indicator data is greater than or equal to the target threshold corresponding to that target indicator data, the operating state of the target software is determined to be normal.

6. The method of claim 1, wherein, After acquiring multiple runtime data points of the target software during its operation, the method further includes: Identify at least one target operational data among the plurality of operational data, wherein the at least one target operational data includes at least the number of failures of the target software within a preset time period, the number of functions that the target software can provide, the response speed of the target software, and the user activity of the target software within the preset time period; The operating status of the target software is determined based on the at least one target operating data.

7. The method of claim 6, wherein, Determining the operating status of the target software based on the at least one target operating data includes: The target interval corresponding to each target running data is determined from a set of multiple preset numerical intervals, wherein each numerical interval corresponds to a preset score; A score for each target running data is determined based on a preset score corresponding to the target interval, resulting in at least one score; Obtain the weight value corresponding to each target's running data; The target score is obtained by weighting and summing the at least one score according to the weight value. The operating status of the target software is determined based on the target score.

8. The method of claim 7, wherein, Determining the operating status of the target software based on the target score includes: If the target score is less than the preset score, the operating state of the target software is determined to be abnormal. If the target score is greater than or equal to the preset score, the target software is determined to be in a normal operating state.

9. A device for determining the running state of software, characterized in that, include: The acquisition module is used to acquire multiple runtime data of the target software during its operation. An input module is used to input the plurality of running data into a target model to obtain the processing result of the target model on the plurality of running data. The processing result includes the target software type of the target software and at least one target indicator data corresponding to the target software type. The target indicator data is the data corresponding to the indicator used to evaluate the running status of the target software. The determination module is used to determine the running status of the target software based on each target indicator data and the target threshold corresponding to that target indicator data; The target software type includes any one of the following software types: a first software type, wherein the software of the first software type is used to serve a preset user group; a second software type, wherein the software of the second software type is used to perform batch data calculation; and a third software type, wherein the software of the third software type is used to process the data.

Images