Software business component evaluation methods, devices, terminals and media
By transforming the raw data of software business components into a business benefit sequence, generating intervention variables, and using a regression model to calculate the goodness of fit, the problem of being unable to quantify component contributions in scenarios without a control group is solved, enabling accurate evaluation and intuitive display of the attribution contributions of components.
Patent Information
- Application Number
- CN202611072463.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-20
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies cannot accurately evaluate the impact of intervention on business benefits in continuous operation scenarios without a control group. They cannot distinguish whether the benefit improvement stems from component optimization or the natural improvement trend of the business itself. They cannot construct a benefit baseline assuming no intervention, which makes it impossible to attribute the net contribution brought by component intervention.
By transforming the raw data of software business components into a business benefit sequence, intervention variables are generated, and a pre-set regression model is used to calculate the goodness of fit. Based on the goodness of fit, the target evaluation results are obtained, including the target comprehensive score and the target dimension contribution score. Counterfactual baselines and target display diagrams are generated to intuitively show the attribution contribution.
It enables accurate evaluation of the impact of intervention on business benefits of software business components in continuous operation scenarios without a control group, and intuitively displays the attribution contribution of the components, solving the problem that the contribution of components cannot be quantified in existing technologies.
Smart Images

Figure CN123068239A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, terminal and medium for evaluating software business components. Background Technology
[0002] With the development of intelligent systems, business systems have evolved into complex systems composed of numerous software components, algorithm models, and data services. The intervention and optimization of these components have a significant impact on business benefits such as the accuracy of functions and the timeliness of early warnings.
[0003] However, existing software business components require continuous operation 24 / 7, making it impossible to pause services and set up a blank control group. This makes it difficult to quantify and assess the true business contribution after component intervention. The main shortcomings are as follows: Firstly, an older version cannot be maintained in parallel for evaluation as a control; existing technology fails in this scenario, meaning it cannot handle situations without a control group. Secondly, even when comparing indicators before and after intervention, it is impossible to distinguish whether the improvement in efficiency stems from component optimization or from the natural improvement trend of the business itself over time, making it difficult to separate the natural trend from the intervention effect. Thirdly, it is impossible to construct a baseline of efficiency assuming no intervention, thus making it impossible to attribute the net contribution brought about by component intervention. Summary of the Invention
[0004] The main purpose of this application is to provide a method, apparatus, terminal and medium for evaluating software business components, which aims to accurately evaluate the impact of intervention on business benefits of software business components in continuous operation scenarios without a control group, and intuitively display the attribution contribution of software business components.
[0005] To achieve the above objectives, this application provides a method for evaluating software business components, the method comprising: The original data corresponding to the software business components are transformed into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point. Based on the business benefit sequence, intervention variables are generated, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after the intervention; Based on the business benefit sequence and the intervention variables, the goodness of fit of the preset regression model is calculated using a preset regression model. Based on the business benefit sequence and the goodness of fit, the target evaluation results corresponding to the software business components are obtained.
[0006] Specifically, the raw data includes temperature forecast values and temperature observation values, and the operational benefit sequence is a temperature accuracy sequence, which includes at least one temperature accuracy. The process of transforming the raw data corresponding to the software business components into a business benefit sequence includes: By using a preset regular expression, the numerical part corresponding to the temperature forecast value and the numerical part corresponding to the temperature observation value are obtained from the temperature forecast value and the temperature observation value. The absolute error value is calculated based on the numerical part corresponding to the temperature forecast value and the numerical part corresponding to the temperature observation value. Based on the absolute error value, the temperature accuracy sequence is generated using a preset piecewise function.
[0007] Specifically, the step of calculating the goodness of fit of the preset regression model based on the business benefit sequence and the intervention variables includes: Using the intervention variable as the independent variable corresponding to the preset regression model, and using the business benefit index value in the business benefit sequence as the dependent variable corresponding to the preset regression model, a fitting operation is performed to obtain the coefficient value corresponding to the preset regression model. Substitute the coefficient values and the intervention variables into the preset regression model to calculate the predicted values corresponding to the preset regression model. The goodness of fit is calculated based on the predicted value corresponding to the preset regression model and the business benefit index value.
[0008] Specifically, the intervention variables include an intervention design matrix and intervention points. The intervention design matrix includes a time series, an intervention instruction sequence, and an interaction sequence. The time series is used to characterize the inherent trend of changes in business benefit indicators over time before the intervention. The intervention instruction sequence is used to characterize the immediate jump in benefits brought about by the launch of software business components. The interaction sequence is used to characterize the change in the slope of changes in business benefit indicators after the intervention relative to the change before the intervention. The intervention point is used to characterize the time boundary point where the intervention occurs. The preset regression model is represented by the following calculation expression:
[0009] in, This represents the predicted value corresponding to the preset regression model. This represents the intercept coefficient, which characterizes the baseline benefit corresponding to the starting point before intervention by the software business component. This represents the time series. This indicates the intervention instruction sequence. This represents the interaction sequence. , as well as The first coefficient, the second coefficient, and the third coefficient are respectively represented, and the first coefficient, the second coefficient, and the third coefficient correspond to the time series, the intervention indication sequence, and the interaction sequence, respectively.
[0010] Specifically, the target evaluation results include the target comprehensive score; The step of obtaining the target evaluation results corresponding to the software business components based on the business benefit sequence and the goodness of fit includes: Based on the portion of the business benefit sequence before the intervention point, the average value of the business benefit index before the intervention is calculated; The target comprehensive score is calculated based on the average value of the business efficiency indicators before the intervention and the goodness of fit.
[0011] Specifically, the target evaluation result also includes a target dimension contribution score, which is used to characterize the evaluation of the software business component on various preset dimensions, including accuracy, stability and performance. The method further includes: Based on the target comprehensive score and the weights corresponding to the preset dimensions, the target dimension contribution score is calculated.
[0012] Specifically, the method further includes: Based on the intercept coefficient and the first coefficient, a counterfactual baseline is generated, wherein the counterfactual baseline is used to characterize the natural trend curve corresponding to the business benefit index value assuming that the software business components have not intervened; Based on the counterfactual baseline, the predicted value corresponding to the preset regression model, the business benefit sequence, the intervention point, and the target dimension contribution score, a target display chart is generated, wherein the target display chart is used to characterize the trend comparison of whether or not the software business component has an attribution contribution.
[0013] To achieve the above objectives, this application also provides a software business component evaluation apparatus, the apparatus comprising: The first unit is used to transform the raw data corresponding to the software business components into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point. The second unit is used to generate intervention variables based on the business benefit sequence, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after intervention; The third unit is used to calculate the goodness of fit of the preset regression model based on the business benefit sequence and the intervention variables. The fourth unit is used to obtain the target evaluation results corresponding to the software business components based on the business benefit sequence and the goodness of fit.
[0014] To achieve the above objectives, this application also provides a terminal, including a memory storing multiple instructions; the processor loads instructions from the memory to execute the steps in any of the methods provided in this application.
[0015] To achieve the above objectives, this application also provides a medium storing a plurality of instructions adapted for loading by a processor to execute the steps in any of the methods provided in this application.
[0016] This application provides a method, apparatus, terminal, and medium for evaluating software business components. The method first transforms the raw data corresponding to the software business components into a business benefit sequence; based on the business benefit sequence, an intervention variable is generated; based on the business benefit sequence and the intervention variable, the goodness of fit of the preset regression model is calculated; based on the business benefit sequence and the goodness of fit, the target evaluation result corresponding to the software business component is obtained. This allows for accurate evaluation of the impact of intervention on business benefits of software business components in a continuous operation scenario without a control group, and intuitively displays the attribution contribution of the software business components. Attached Figure Description
[0017] Figure 1 A flowchart illustrating the method provided in the embodiments of this application; Figure 2 A business benefit attribution trend comparison chart provided for embodiments of this application; Figure 3 This is a schematic diagram of the device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the terminal structure provided in an embodiment of this application. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] Existing software business components require continuous operation 24 / 7, making it impossible to pause services and set up a blank control group. This makes it difficult to quantify and assess the true business contribution after component intervention. The main shortcomings are as follows: First, an older version cannot be maintained in parallel for evaluation as a control; existing technology fails in this scenario, meaning it cannot handle situations without a control group. Second, even when comparing indicators before and after intervention, it is impossible to distinguish whether the improvement in efficiency stems from component optimization or from the natural improvement trend of the business itself over time; third, it is difficult to separate the natural trend from the intervention effect. Fourth, it is impossible to construct a baseline of efficiency assuming no intervention, thus making it impossible to attribute the net contribution brought about by component intervention.
[0020] Therefore, embodiments of this application provide a software business component evaluation method, apparatus, terminal, and medium to solve practical technical problems.
[0021] In some embodiments, the device may be integrated into an electronic device, such as a terminal or server.
[0022] In some embodiments, the server may also be implemented as a terminal.
[0023] The server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
[0024] The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein.
[0025] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.
[0026] This application provides a method for evaluating software business components. The method can accurately evaluate the impact of intervention on business benefits of software business components in a continuous operation scenario without a control group, and intuitively display the attribution contribution of software business components.
[0027] like Figure 1 The specific process of the method can be as follows: S110. The original data corresponding to the software business components is transformed into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point.
[0028] In some embodiments, the raw data includes temperature forecasts and temperature observations, and the operational benefit sequence is a temperature accuracy sequence, which includes at least one temperature accuracy.
[0029] Specifically, the process of transforming the raw data corresponding to the software business components into a business benefit sequence includes the steps A1 to A3 shown below: A1. Using a preset regular expression, extract the numerical part corresponding to the temperature forecast value and the numerical part corresponding to the temperature observation value from the temperature forecast value and the temperature observation value. A2. Based on the numerical part corresponding to the temperature forecast value and the numerical part corresponding to the temperature observation value, the absolute error value is calculated; A3. Based on the absolute error value, the temperature accuracy sequence is generated using a preset piecewise function.
[0030] Specifically, the raw data may include the contents shown in Table 1 below:
[0031] Table 1. Original Data Illustration Specifically, calculation rules are automatically matched based on the component type of the software business components. Taking temperature forecast accuracy as an example, a preset regular expression is first used:
[0032] Extract the numerical portions (removing unit symbols such as ℃) from the temperature forecast and observed value strings, then calculate the absolute error. Calculate the accuracy score for each time point using the following piecewise function, ultimately generating a column Y, which represents the business benefit sequence (an integer sequence from 0 to 100, stored in...). middle).
[0033] Accuracy calculation formula:
[0034] Regular expressions:
[0035] in: : Matches optional negative signs (supports negative temperatures, such as -5.2℃); : Matches one or more digits (integer part); : Matches optional decimal points; : Matches zero or more digits (decimal part).
[0036] S120. Based on the business benefit sequence, generate intervention variables, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during, and after intervention.
[0037] In some embodiments, the data is sorted in ascending order of observation time to generate a continuous time series. 1, 2, 3, ... Based on the "Intervention Status" column in Table 1 of the original data: if the status is not "Before Intervention", then... 1, otherwise 0. Automatically locate the first intervention point T0 (i.e., the first line). 1 corresponds to the time sequence number. Construct the interaction sequence corresponding to the interaction item. The final intervention design matrix is obtained as follows:
[0038] This process does not require users to manually mark intervention points; three sequences are generated: , , .
[0039] S130. Based on the business benefit sequence and the intervention variables, the goodness of fit of the preset regression model is calculated using a preset regression model.
[0040] In some embodiments, the intervention variables include an intervention design matrix and intervention points. The intervention design matrix includes a time series, an intervention instruction sequence, and an interaction sequence. The time series is used to characterize the inherent trend of changes in business benefit indicators over time before the intervention. The intervention instruction sequence is used to characterize the immediate jump in benefits brought about by the launch of software business components. The interaction sequence is used to characterize the change in the slope of business benefit indicators after the intervention relative to the change before the intervention. The intervention point is used to characterize the time boundary point at which the intervention occurs.
[0041] The preset regression model is represented by the following calculation expression:
[0042] in, This represents the predicted value corresponding to the preset regression model. This represents the intercept coefficient, which characterizes the baseline benefit corresponding to the starting point before intervention by the software business component. This represents the time series. This indicates the intervention instruction sequence. This represents the interaction sequence. , as well as The first coefficient, the second coefficient, and the third coefficient are respectively represented, and the first coefficient, the second coefficient, and the third coefficient correspond to the time series, the intervention indication sequence, and the interaction sequence, respectively.
[0043] In some embodiments, the step of calculating the goodness of fit of the preset regression model based on the business benefit sequence and the intervention variable includes the steps B1 to B3 shown below: B1. Using the intervention variable as the independent variable corresponding to the preset regression model, and the business benefit index value in the business benefit sequence as the dependent variable corresponding to the preset regression model, perform a fitting operation to obtain the coefficient value corresponding to the preset regression model. B2. Substitute the coefficient values and the intervention variables into the preset regression model to calculate the predicted values corresponding to the preset regression model; B3. Based on the predicted values corresponding to the preset regression model and the business benefit index values, the goodness of fit is calculated.
[0044] In some embodiments, the generated intervention design matrix As the independent variable, with That is, the business benefit index value in the business benefit sequence is the dependent variable, and the call... Performing ordinary least squares fitting yields four coefficients: (Intercept coefficient): Represents the baseline benefit at the starting point before intervention; ( The coefficient, i.e., the first coefficient, represents the slope of the natural trend before intervention; ( The coefficient, i.e., the second coefficient, represents the immediate effect of intervention (instantaneous jump). ( The coefficient, i.e. the third coefficient, represents the duration of the intervention (the change in slope).
[0045] After fitting, the independent variable values at each time point are substituted into the equation of the preset regression model to obtain the predicted value of the preset regression model at that time point. :
[0046] Alternatively, in matrix form:
[0047] Therefore, the goodness of fit is calculated. .
[0048] In some embodiments, it can also be set to obtain and the maximum time number in the time series Calculate the length of the observation period after intervention. This leads to the cumulative attribution benefit:
[0049] The cumulative attribution benefit represents the total benefit improvement generated by the intervention of the software business component from the start of the intervention until the end of the observation period.
[0050] S140. Based on the business benefit sequence and the goodness of fit, obtain the target evaluation results corresponding to the software business components.
[0051] In some embodiments, the target evaluation result includes a target comprehensive score; The step of obtaining the target evaluation result corresponding to the software business component based on the business benefit sequence and the goodness of fit includes the following steps C1 to C2: C1. Based on the partial sequence before the intervention point in the business benefit sequence, calculate the average value of the business benefit index before the intervention.
[0052] In some embodiments, from Extracting the pre-intervention portion from the sequence (i.e. (Records), calculate the average value of business performance indicators before intervention. .
[0053] C2. Based on the average value of the business benefit indicators before the intervention and the goodness of fit, the target comprehensive score is calculated.
[0054] In some embodiments, the target comprehensive score Calculated using the following formula and rounded to the nearest integer:
[0055] In some embodiments, the target evaluation result further includes a target dimension contribution score, which is used to characterize the evaluation of the software business component on various preset dimensions, including accuracy, stability, and performance; the method further includes the following specific implementation process: Based on the target comprehensive score and the weights corresponding to the preset dimensions, the target dimension contribution score is calculated.
[0056] Specifically, assuming the weights corresponding to the preset dimensions are: accuracy 35%, stability 30%, and performance 35%, then the target dimension contribution score is calculated as follows: × The weights corresponding to the preset dimensions.
[0057] In some embodiments, the method further includes steps D1 to D2 as shown below: D1. Based on the intercept coefficient and the first coefficient, generate a counterfactual baseline, wherein the counterfactual baseline is used to characterize the natural trend curve corresponding to the business benefit index value assuming that the software business components have not intervened.
[0058] Specifically, the counterfactual baseline is calculated using the following formula. (Assuming benefits without upgrades):
[0059] D2. Based on the counterfactual baseline, the predicted value corresponding to the preset regression model, the business benefit sequence, the intervention point, and the target dimension contribution score, generate a target display chart, wherein the target display chart is used to characterize the trend comparison of whether or not the software business component has an attribution contribution.
[0060] Specifically, the front-end uses ECharts to draw line charts: actual observed values (i.e., the Y sequence, represented by a dotted line), and model-predicted values (i.e., the Y-series, represented by a dotted line). Sequence (solid line), natural growth extrapolation curve (i.e., counterfactual baseline, dashed line). At the intervention point Mark the vertical line at the point; the shaded area between the actual observation and the counterfactual baseline represents the attribution contribution. Figure 2 As shown. Additionally, radar maps can be drawn based on the target dimension contribution score.
[0061] Specifically, such as Figure 2 As shown in the figure: The core message in the diagram: Horizontal axis: Time sequence (days), from day 1 to day 30.
[0062] Vertical axis: Forecast accuracy (points, 0-100 points), representing the level of operational efficiency.
[0063] Three curves: Actual observation (blue dotted line): Real changes in benefits.
[0064] Model fitting (red dashed line): Considers the fitting trend after component intervention.
[0065] Natural growth extrapolation (green dotted line): The counterfactual baseline that continues to extrapolate the trend before the intervention, assuming no component upgrades.
[0066] Two shaded areas: Gray shading (baseline period): Data prior to intervention, used to establish natural trends.
[0067] Light blue shading (observation period): Post-intervention data; the difference between the actual value and the counterfactual baseline represents the attribution contribution of the software business components.
[0068] Black vertical line: Component launch time The left side shows the old system, and the right side shows the new components.
[0069] Purpose: To visually demonstrate the difference in benefits between "having software business components" and "not having software business components", with the attribution contribution presented in shaded areas, so that even non-professionals can understand the results at a glance.
[0070] In summary, this application provides a method for evaluating software business components, which enables accurate evaluation of the impact of intervention on business benefits in a continuous operation scenario without a control group, and intuitively displays the attribution contribution of the software business components.
[0071] To better implement the above methods, this application also provides a software business component evaluation device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.
[0072] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the software business component evaluation device specifically integrated into the terminal as an example.
[0073] For example, such as Figure 3 As shown, the software business component evaluation device 300 may include a first unit 301, a second unit 302, a third unit 303, and a fourth unit 304. The device includes: The first unit 301 is used to convert the original data corresponding to the software business component into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business component at each time point. The second unit 302 is used to generate intervention variables based on the business benefit sequence, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after intervention. The third unit 303 is used to calculate the goodness of fit of the preset regression model based on the business benefit sequence and the intervention variables through the preset regression model. The fourth unit 304 is used to obtain the target evaluation results corresponding to the software business components based on the business benefit sequence and the goodness of fit.
[0074] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.
[0075] As can be seen from the above, the embodiments of this application can accurately evaluate the impact of the intervention of software business components on business benefits in a continuous operation scenario without a control group, and intuitively demonstrate the attribution contribution of the software business components.
[0076] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.
[0077] In some embodiments, the product processing device may also be integrated into multiple electronic devices, such as multiple servers, with the software business component evaluation method of this application being implemented by the multiple servers.
[0078] In this embodiment, the electronic device will be described in detail as a terminal, for example, such as... Figure 4 As shown, it illustrates a structural schematic diagram of the terminal 400 involved in an embodiment of this application. Specifically: The terminal 400 may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The terminal 400 structure shown does not constitute a limitation on the terminal 400, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 401 is the control center of the terminal 400. It connects various parts of the terminal 400 via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the terminal 400. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.
[0079] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal 400, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
[0080] The terminal 400 also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0081] The terminal 400 may also include an input module 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
[0082] The terminal 400 may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module. The terminal 400 can perform short-range wireless transmission through the wireless module of the communication module 405, thereby providing users with wireless broadband Internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.
[0083] Although not shown, terminal 400 may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, processor 401 in terminal 400 loads the executable files corresponding to the processes of one or more applications into memory 402 according to the following instructions, and processor 401 runs the applications stored in memory 402 to realize various functions, as follows: The original data corresponding to the software business components are transformed into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point. Based on the business benefit sequence, intervention variables are generated, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after the intervention; Based on the business benefit sequence and the intervention variables, the goodness of fit of the preset regression model is calculated using a preset regression model. Based on the business benefit sequence and the goodness of fit, the target evaluation results corresponding to the software business components are obtained.
[0084] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0085] As can be seen from the above, the embodiments of this application can accurately evaluate the impact of the intervention of software business components on business benefits in a continuous operation scenario without a control group, and intuitively demonstrate the attribution contribution of the software business components.
[0086] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be accomplished by instructions, or by instructions controlling related hardware. These instructions can be stored in a medium and loaded and executed by a processor.
[0087] Therefore, embodiments of this application provide a medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the software business component evaluation methods provided in embodiments of this application. For example, the instructions can execute the following steps: The original data corresponding to the software business components are transformed into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point. Based on the business benefit sequence, intervention variables are generated, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after the intervention; Based on the business benefit sequence and the intervention variables, the goodness of fit of the preset regression model is calculated using a preset regression model. Based on the business benefit sequence and the goodness of fit, the target evaluation results corresponding to the software business components are obtained.
[0088] The medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0089] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a medium. A processor of a computer device reads the computer instructions from the medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.
[0090] Since the instructions stored in the medium can execute the steps in any of the software business component evaluation methods provided in the embodiments of this application, the beneficial effects that any of the software business component evaluation methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.
[0091] The above provides a detailed description of a software business component evaluation method, apparatus, terminal, and medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for evaluating software business components, characterized in that, The method includes: The original data corresponding to the software business components are transformed into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point. Based on the business benefit sequence, intervention variables are generated, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after the intervention; Based on the business benefit sequence and the intervention variables, the goodness of fit of the preset regression model is calculated using a preset regression model. Based on the business benefit sequence and the goodness of fit, the target evaluation results corresponding to the software business components are obtained.
2. The method as described in claim 1, characterized in that, The raw data includes temperature forecast values and temperature observation values, and the operational benefit sequence is a temperature accuracy sequence, which includes at least one temperature accuracy. The process of transforming the raw data corresponding to the software business components into a business benefit sequence includes: By using a preset regular expression, the numerical part corresponding to the temperature forecast value and the numerical part corresponding to the temperature observation value are obtained from the temperature forecast value and the temperature observation value. The absolute error value is calculated based on the numerical part corresponding to the temperature forecast value and the numerical part corresponding to the temperature observation value. Based on the absolute error value, the temperature accuracy sequence is generated using a preset piecewise function.
3. The method as described in claim 1, characterized in that, The step of calculating the goodness of fit of the preset regression model based on the business benefit sequence and the intervention variables includes: Using the intervention variable as the independent variable corresponding to the preset regression model, and using the business benefit index value in the business benefit sequence as the dependent variable corresponding to the preset regression model, a fitting operation is performed to obtain the coefficient value corresponding to the preset regression model. Substitute the coefficient values and the intervention variables into the preset regression model to calculate the predicted values corresponding to the preset regression model. The goodness of fit is calculated based on the predicted value corresponding to the preset regression model and the business benefit index value.
4. The method as described in claim 1, characterized in that, The intervention variables include an intervention design matrix and intervention points. The intervention design matrix includes a time series, an intervention instruction sequence, and an interaction sequence. The time series is used to characterize the inherent trend of business benefit indicators before intervention over time. The intervention instruction sequence is used to characterize the instantaneous jump in benefits brought about by the launch of software business components. The interaction sequence is used to characterize the change in the slope of business benefit indicators after intervention relative to the change before intervention. The intervention point is used to characterize the time boundary point where intervention occurs. The preset regression model is represented by the following calculation expression: in, This represents the predicted value corresponding to the preset regression model. This represents the intercept coefficient, which characterizes the baseline benefit corresponding to the starting point before intervention by the software business component. This represents the time series. This indicates the intervention instruction sequence. This represents the interaction sequence. , as well as These represent the first coefficient, the second coefficient, and the third coefficient, respectively, which correspond to the time series, the intervention indication sequence, and the interaction sequence, respectively.
5. The method as described in claim 4, characterized in that, The target evaluation results include the target comprehensive score; The step of obtaining the target evaluation results corresponding to the software business components based on the business benefit sequence and the goodness of fit includes: Based on the portion of the business benefit sequence before the intervention point, the average value of the business benefit index before the intervention is calculated; The target comprehensive score is calculated based on the average value of the business efficiency indicators before the intervention and the goodness of fit.
6. The method as described in claim 5, characterized in that, The target evaluation result also includes target dimension contribution score, which is used to characterize the evaluation of software business components on various preset dimensions, including accuracy, stability and performance; The method further includes: Based on the target comprehensive score and the weights corresponding to the preset dimensions, the target dimension contribution score is calculated.
7. The method as described in claim 6, characterized in that, The method further includes: Based on the intercept coefficient and the first coefficient, a counterfactual baseline is generated, wherein the counterfactual baseline is used to characterize the natural trend curve corresponding to the business benefit index value assuming that the software business components have not intervened; Based on the counterfactual baseline, the predicted value corresponding to the preset regression model, the business benefit sequence, the intervention point, and the target dimension contribution score, a target display chart is generated, wherein the target display chart is used to characterize the trend comparison of whether or not the software business component has an attribution contribution.
8. A software business component evaluation device, characterized in that, The device includes: The first unit is used to transform the raw data corresponding to the software business components into a business benefit sequence, wherein the business benefit sequence is used to characterize the business benefit index value obtained by the software business components at each time point. The second unit is used to generate intervention variables based on the business benefit sequence, wherein the intervention variables are used to characterize the impact of software business components on the business scenario before, during and after intervention; The third unit is used to calculate the goodness of fit of the preset regression model based on the business benefit sequence and the intervention variables. The fourth unit is used to obtain the target evaluation results corresponding to the software business components based on the business benefit sequence and the goodness of fit.
9. A terminal, characterized in that, The method includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the method as described in any one of claims 1 to 7.
10. A medium, characterized in that, The medium stores a plurality of instructions adapted for loading by a processor to execute the steps of the method according to any one of claims 1 to 7.
