MOM software system evaluation method, electronic device and storage medium based on multiple scenarios

Abstract

The MOM software system evaluation method, electronic device and storage medium based on multiple scenarios include: obtaining parameter information of the software system running in different scenarios to form a data set, constructing a multi-scenario dynamic adaptation evaluation model, training and testing the constructed multi-scenario dynamic adaptation evaluation model through the data set; using the trained multi-scenario dynamic adaptation evaluation model to evaluate the performance of the software system in different scenarios, statistically analyzing the performance of different scenarios to obtain the performance index of the software system; constructing a fuzzy evaluation model, and using fuzzy comprehensive evaluation to evaluate the software system based on the performance index of the software system. The present invention can perceive scene changes in real time and perform performance evaluation accordingly, and the evaluation is more comprehensive and accurate.

Description

Technical Field

[0001] The present invention relates to the fields of industry software and artificial intelligence, and in particular to a MOM software system evaluation method based on multiple scenarios, an electronic device and a storage medium. Background Art

[0002] With the rapid development and widespread application of information technology, various software systems play a vital role in all fields of modern society. However, different application scenarios have different requirements for the quality and performance of software systems, and traditional static evaluation methods can no longer meet such diverse needs. In actual applications, the system faces diverse scenario requirements, such as high concurrency scenarios, low latency scenarios, security-sensitive scenarios, etc. These scenarios pose different challenges and requirements to the performance and quality of software systems.

[0003] At present, traditional system evaluation methods of MOM software are often based on static indicators and cannot perceive and adapt to changes in scenarios in real time. They usually only focus on the average performance of the system and ignore the performance differences in different scenarios. In order to meet the needs of multiple scenarios, it is necessary to develop an evaluation method that can perceive scenario changes in real time and dynamically adjust system configuration and parameters. Summary of the invention

[0004] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and propose a MOM software system evaluation method based on multiple scenarios, an electronic device and a storage medium, which can perceive scene changes in real time and perform performance evaluation accordingly.

[0005] The present invention adopts the following technical solution:

[0006] A MOM software system evaluation method based on multiple scenarios, including:

[0007] Obtain parameter information of the software system running in different scenarios to form a data set, build a multi-scenario dynamic adaptation evaluation model, and train and test the constructed multi-scenario dynamic adaptation evaluation model through the data set; use the trained multi-scenario dynamic adaptation evaluation model to evaluate the performance of the software system in different scenarios, and perform statistical analysis on the performance of different scenarios to obtain the performance indicators of the software system;

[0008] A fuzzy evaluation model is constructed, and the software system is evaluated by using fuzzy comprehensive evaluation based on the performance indicators of the software system.

[0009] Furthermore, the performance evaluation of the software system in a given scenario specifically includes the following:

[0010] The parameter information of the software system in a given scenario is defined as a parameter set, where each parameter in the parameter set represents a variable in the scenario;

[0011] Extract sample values ​​from the distribution of each parameter, and combine the sampled sample values ​​into a scene parameter vector;

[0012] The scenario parameter vector is input into the trained multi-scenario dynamic adaptation evaluation model to obtain the status and performance indicators of the software system in a given scenario:

[0013] State j ,Performance j , R j =G(S j )

[0014] Wherein, G(.) represents the multi-scenario dynamic adaptation evaluation model, S j is a scene parameter vector, State j It is the software system in a given scenario S j The state of performance j is the system in a given scenario S j The cost indicator under R j It is the response time performance indicator.

[0015] Furthermore, the multi-scenario dynamic adaptation evaluation model evaluates the performance of the software system, specifically:

[0016] Task division is regarded as dividing the set of m scene parameter vectors into several subsets, each subset is assigned to a working node;

[0017] Let X ij is a binary variable indicating whether the scene parameter vector is assigned to the worker node N j , X ij =1 means allocation, X ij =0 means no allocation, define W ij To transform the parameter vector S i Assign to N j The cost is calculated as follows:

[0018] Each scene parameter vector is assigned to exactly one node:

[0019]

[0020] Each worker node can receive multiple scene parameter vectors:

[0021]

[0022] Output the objective function that minimizes the total cost:

[0023]

[0024] The parameter information includes a time series, and a multi-scenario dynamic adaptation evaluation model evaluates the performance of the software system. Specifically, the ARMA (p, q) model is used for prediction, and the observation value X of each time point t and the previous p time points in the time series of a given scenario is t-1 ,X t-2 ,·····,X t-p As input to the autoregressive part, the output is the response time for a given scenario:

[0025]

[0026] Where p is the autoregressive order, q is the moving average order, and φ i is the autoregressive coefficient, θ j is the moving average coefficient, ∈ t is the error term; R t is the response time for a given scenario.

[0027] Furthermore, statistical analysis is performed on the performance of different scenarios to obtain the performance indicators of the software system, which specifically includes the following calculations:

[0028] Response time mean calculation:

[0029]

[0030] Response time variance calculation:

[0031]

[0032] Resource utilization calculation:

[0033]

[0034] Among them, R j is the response time performance indicator of the jth scenario, u(t) is the resource utilization at time t; T represents the length of time considered for calculating resource utilization.

[0035] Furthermore, the fuzzy evaluation model uses a membership function to describe the fuzzy concept of software quality, and the formula is as follows:

[0036] The performance indicators of the software system are combined into an evaluation factor set U. For each factor u in the evaluation factor set, i Comments for ratings v j The set of is the comment set V:

[0037] V = {v1, v2, v3, v4} = {excellent, good, qualified, unqualified};

[0038] For each factor u i , through expert scoring or questionnaire survey, the membership of each level in the comment set is obtained to form a fuzzy relationship matrix R, which is expressed as follows:

[0039]

[0040] The element r in the fuzzy relationship matrix R ij Indicator u i Comments j The membership degree of m is the number of levels in the comment set, and n is the number of factors in the evaluation factor set;

[0041] Let weight vector A = (a1, a2, ...., a n ) indicates the importance of each evaluation factor in the evaluation process; and meets the following requirements:

[0042]

[0043] The weight vector A and the fuzzy relationship matrix R are synthesized using the fuzzy synthesis operator to obtain the fuzzy comprehensive evaluation result vector B:

[0044]

[0045] in, Represents the fuzzy composition operator.

[0046] Furthermore, the specific calculation formula of the fuzzy comprehensive evaluation result vector B is:

[0047]

[0048] According to the fuzzy comprehensive evaluation result vector, the final evaluation level of the system software is determined according to the maximum membership principle.

[0049] An electronic device, characterized in that it includes at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the MOM software system evaluation method based on multiple scenarios.

[0050] A storage medium, characterized in that the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the MOM software system evaluation method based on multiple scenarios.

[0051] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. In the present invention, a multi-scenario dynamic adaptation evaluation model is used to evaluate the performance of the software system in different scenarios, and statistical analysis is performed on the performance of different scenarios to obtain the performance indicators of the software system. It can perceive the scene changes in real time and perform performance evaluation accordingly, and the evaluation is more comprehensive and accurate.

[0053] 2. In the present invention, the scene dynamic adaptation evaluation model regards task division as dividing the set composed of scene parameter vectors into several subsets, each subset is assigned to a working node, and an objective function of minimizing the allocated cost is set. The output minimized total cost is one of the performances, and the response time is predicted based on the time series. Through the performance obtained by evaluation, various resources in the system, such as computing resources, storage resources, network resources, etc., can be reasonably allocated and utilized, reducing resource waste and improving resource utilization.

[0054] 3. In the present invention, fuzzy comprehensive evaluation is used to evaluate the software system based on the performance indicators of the software system. Through real-time monitoring and regular evaluation of the system, the performance of the software in different scenarios can be fully reflected, and the performance bottlenecks and potential problems of the system and the level of the system can be discovered in time, providing a basis for optimization and improvement of the system according to the level.

[0055] 4. In the present invention, through a dynamic testing environment and diversified test cases, the performance and quality of the software can be comprehensively evaluated, and optimization suggestions can be provided to help software developers promptly discover and solve problems during the software development process, thereby improving the quality and performance of the software; at the same time, this method is not only applicable to MOM systems, but can also be extended to other manufacturing-related software systems, providing strong support for the digital transformation of the manufacturing industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The figure is a flow chart of the method of the present invention.

[0057] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0058] The present invention is further described below through specific implementation modes.

[0059] See also Figure 1, a multi-scenario based MOM software system evaluation method,MOM software system is a manufacturing operation management system, a core tool for digital manufacturing, used to coordinate and manage all aspects of manufacturing operations, including production planning, quality management, equipment maintenance, and inventory management. The MOM system is rich in functions and can improve production efficiency, ensure quality, reduce costs, and optimize resources. The MOM system is widely used in various industries, including automobile manufacturing, food processing, pharmaceutical manufacturing, electronic manufacturing, and semiconductor manufacturing. Different industries can configure and customize the MOM system according to their own needs to meet specific manufacturing processes and standards.

[0060] The evaluation method of the present invention specifically includes the following:

[0061] 1) Obtain parameter information of the software system running in different scenarios to form a data set, build a multi-scenario dynamic adaptation evaluation model, and train and test the constructed multi-scenario dynamic adaptation evaluation model through the data set; use the trained multi-scenario dynamic adaptation evaluation model to evaluate the performance of the software system in different scenarios, and perform statistical analysis on the performance of different scenarios to obtain the performance indicators of the software system;

[0062] In this step, the parameter information of operation in different scenarios includes system operation status, user behavior and other data. By integrating real-time monitoring and data acquisition modules in the MOM software system, system operation parameter information can be collected in real time. The data set is divided into a training set and a test set, and the model is trained using the training set to ensure the accuracy and reliability of the model. Then, the multi-scenario dynamic adaptation evaluation model is tested with the test set to verify the effectiveness and applicability of the model.

[0063] When using the trained multi-scenario dynamic adaptation evaluation model to evaluate the performance of the software system in different scenarios, the parameter information of the software system in a given scenario is defined as the parameter set {X1, X2, .....X n}, n is the number of parameter information, each parameter X in the parameter set i Represents a variable in the scene, i = 1, 2, ... n. Different parameters in the scene correspond to different variables, including random parameters, which are random variables X i , assuming it follows a normal distribution:

[0064]

[0065] Among them, μ i is the mean, is the variance.

[0066] Taking randomness parameters as an example, sample values ​​are drawn from the distribution of each parameter Where j represents the scene number:

[0067]

[0068] For continuous random variables, f i (x) is a probability density function; for discrete random variables, f i (x) is the probability mass function, which indicates the parameter The possible values ​​of and their corresponding probabilities.

[0069] Combine the sampled values ​​into a scene parameter vector:

[0070]

[0071] Input the scenario parameter vector into the trained multi-scenario dynamic adaptation evaluation model to obtain the status and performance indicators of the software system in a given scenario:

[0072] State j ,Performance j , R j =G(S j );

[0073] Among them, G(.) represents the multi-scenario dynamic adaptation evaluation model, S j is a scene parameter vector, State j It is the software system in a given scenario S j The state of performance j is the system in a given scenario S j The cost indicator under R j It is the response time performance indicator.

[0074] In this embodiment, the multi-scenario dynamic adaptation evaluation model evaluates the performance of the software system, specifically:

[0075] Task division is regarded as dividing the set of m scene parameter vectors into several subsets, each of which is assigned to a working node. Specifically, let X ij is a binary variable indicating whether the scene parameter vector is assigned to the worker node N j , X ij =1 means allocation, X ij =0 means no allocation, define W ij To transform the parameter vector S i Assign to N j The cost is calculated as follows:

[0076] Each scene parameter vector is assigned to exactly one node:

[0077]

[0078] Each worker node can receive multiple scene parameter vectors:

[0079]

[0080] Output the objective function that minimizes the total cost:

[0081]

[0082] According to the objective function of minimizing the total cost, a cost index can be calculated, which is used as one of the indicators for evaluating the performance of the software system.

[0083] In this embodiment, the parameter information also includes time series, the performance of the multi-scenario dynamic adaptation evaluation model for evaluating the software system, and the response time. Specifically, the ARMA (p, q) model is used for prediction, and the observation value X at each time point t and the previous p time points in the time series of a given scenario is calculated. t-1 ,X t-2 ,·····,X t-p As input to the autoregressive part, the output is the response time for a given scenario:

[0084]

[0085] Where p is the autoregressive order, q is the moving average order, and φ i is the autoregressive coefficient, θ j is the moving average coefficient, ∈ t is the error term; R t is the response time for a given scenario.

[0086] In this embodiment, different scenarios can obtain the performance of corresponding scenarios, and statistical analysis is performed on the performance of different scenarios to obtain the performance index of the software system, which specifically includes the following calculations:

[0087] Response time mean calculation:

[0088]

[0089] Response time variance calculation:

[0090]

[0091] Resource utilization calculation:

[0092]

[0093] Among them, R jis the response time performance indicator of the jth scenario, u(t) is the resource utilization at time t; T represents the length of time considered for calculating resource utilization.

[0094] Through statistical analysis of the performance of different scenarios, comprehensive evaluation results can be obtained.

[0095] In practical applications, for a given scenario, after obtaining the performance through the multi-scenario dynamic adaptation evaluation model, the software system in the given scenario can be dynamically adjusted and optimized according to the obtained performance. Multiple adjustment plans can be executed using a parallelization strategy, and then evaluated using the multi-scenario dynamic adaptation evaluation model. According to the effects of multiple adjustment plans, the best plan can be selected for application, so as to quickly respond to environmental changes and ensure the efficient and stable operation of the software. According to the obtained performance, problems and deficiencies in the software can also be discovered and fixed, prompting software developers to improve and perfect it, thereby improving the overall quality of the software.

[0096] 2) Construct a fuzzy evaluation model and use fuzzy comprehensive evaluation to evaluate the software system based on the performance indicators of the software system.

[0097] In the present invention, a dynamic evaluation index system of software quality and performance is defined, a fuzzy evaluation model of a dynamic evaluation model algorithm is adopted, and a comprehensive evaluation of software quality is obtained through fuzzy comprehensive evaluation.

[0098] The fuzzy evaluation model uses membership functions to describe the fuzzy concept of software quality, as follows:

[0099] The performance indicators of the software system are combined into an evaluation factor set U, U = {u1,u2,....,u n}, including cost indicators, response time mean, response time variance and resource utilization, etc., suppose that for each factor u in the evaluation factor set i Comments for ratings v j The set of is the comment set V:

[0100] V = {v1, v2, v3, v4} = {excellent, good, qualified, unqualified};

[0101] For each factor u i , through expert scoring or questionnaire survey, the membership of each level in the comment set is obtained to form a fuzzy relationship matrix R, which is expressed as follows:

[0102]

[0103] The element r in the fuzzy relationship matrix R ij Indicator u i Comments j The membership degree of m is the number of levels in the comment set, and n is the number of factors in the evaluation factor set;

[0104] Let weight vector A = (a1, a2, ...., a n ) indicates the importance of each evaluation factor in the evaluation process; and meets

[0105]

[0106] The weight vector A and the fuzzy relationship matrix R are synthesized using the fuzzy synthesis operator to obtain the fuzzy comprehensive evaluation result vector B:

[0107]

[0108] in, Represents a fuzzy synthesis operator, which includes weighted average method and main factor determination type. For example, (M(·,+)) is used.

[0109] The specific calculation formula of the fuzzy comprehensive evaluation result vector B is:

[0110]

[0111] According to the fuzzy comprehensive evaluation result vector, the final evaluation level of the system software is determined according to the maximum membership principle.

[0112] In actual applications, through real-time monitoring and regular evaluation of software systems, we can promptly discover the performance bottlenecks and potential problems of the software system and the level of the system, and provide a basis for the optimization and improvement of the software system according to the level.

[0113] Based on this, the present invention also proposes an electronic device, comprising at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the above-mentioned multi-scenario based MOM software system evaluation method.

[0114] The present invention also proposes a storage medium, which stores one or more computer instructions, and the one or more computer instructions are used to implement the above-mentioned MOM software system evaluation method based on multiple scenarios.

[0115] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A MOM software system evaluation method based on multiple scenarios, characterized in that: include: Obtain parameter information of the software system running in different scenarios to form a data set, build a multi-scenario dynamic adaptation evaluation model, and train and test the constructed multi-scenario dynamic adaptation evaluation model through the data set; use the trained multi-scenario dynamic adaptation evaluation model to evaluate the performance of the software system in different scenarios, and perform statistical analysis on the performance of different scenarios to obtain the performance indicators of the software system; A fuzzy evaluation model is constructed, and the software system is evaluated by using fuzzy comprehensive evaluation based on the performance indicators of the software system.

2. A MOM software system evaluation method based on multiple scenarios as described in claim 1, characterized in that: The performance evaluation of the software system in a given scenario specifically includes the following: The parameter information of the software system in a given scenario is defined as a parameter set, where each parameter in the parameter set represents a variable in the scenario; Extract sample values ​​from the distribution of each parameter, and combine the sampled sample values ​​into a scene parameter vector; The scenario parameter vector is input into the trained multi-scenario dynamic adaptation evaluation model to obtain the status and performance indicators of the software system in a given scenario: State j ,Performance j ,R j =G(S j ); Wherein, G(.) represents the multi-scenario dynamic adaptation evaluation model, S j is a scene parameter vector, State j It is the software system in a given scenario S j The state of performance j is the system in a given scenario S j The cost indicator under R j It is the response time performance indicator.

3. A MOM software system evaluation method based on multiple scenarios as described in claim 2, characterized in that: The multi-scenario dynamic adaptation evaluation model evaluates the performance of the software system, specifically: Task division is regarded as dividing the set of m scene parameter vectors into several subsets, each subset is assigned to a working node; Let X ij is a binary variable indicating whether the scene parameter vector is assigned to the worker node N j , X ij =1 means allocation, X ij =0 means no allocation, define W ij To transform the parameter vector S i Assign to N j The cost is calculated as follows: Each scene parameter vector is assigned to exactly one node: Each worker node can receive multiple scene parameter vectors: Output the objective function that minimizes the total cost:

4. A MOM software system evaluation method based on multiple scenarios as described in claim 2, characterized in that: The parameter information includes a time series, and a multi-scenario dynamic adaptation evaluation model evaluates the performance of the software system. Specifically, the ARMA (p, q) model is used for prediction, and the observation value X of each time point t and the previous p time points in the time series of a given scenario is t-1 ,X t-2 ,·····,X t-p As input to the autoregressive part, the output is the response time for a given scenario: Where p is the autoregressive order, q is the moving average order, and φ i is the autoregressive coefficient, θ j is the moving average coefficient, ∈ t is the error term; R t is the response time for a given scenario.

5. A MOM software system evaluation method based on multiple scenarios as described in claim 1, characterized in that: The performance indicators of the software system are obtained by statistically analyzing the performance of different scenarios, including the following calculations: Response time mean calculation: Response time variance calculation: Resource utilization calculation: Among them, R j is the response time performance indicator of the jth scenario, u(t) is the resource utilization at time t; T represents the length of time considered for calculating resource utilization.

6. A MOM software system evaluation method based on multiple scenarios as described in claim 1, characterized in that: The fuzzy evaluation model uses a membership function to describe the fuzzy concept of software quality. The formula is as follows: The performance indicators of the software system are combined into an evaluation factor set U. For each factor u in the evaluation factor set, i Comments for ratings v j The set of is the comment set V: V = {v1, v2, v3, v4} = {excellent, good, qualified, unqualified}; For each factor u i , through expert scoring or questionnaire survey, the membership of each level in the comment set is obtained to form a fuzzy relationship matrix R, which is expressed as follows: The element r in the fuzzy relationship matrix R ij Indicator u i Comments j The membership degree of m is the number of levels in the comment set, and n is the number of factors in the evaluation factor set; Let weight vector A = (a1, a2, ...., a n ) indicates the importance of each evaluation factor in the evaluation process; and meets the following requirements: The weight vector A and the fuzzy relationship matrix R are synthesized using the fuzzy synthesis operator to obtain the fuzzy comprehensive evaluation result vector B: in, Represents the fuzzy composition operator.

7. A MOM software system evaluation method based on multiple scenarios as described in claim 6, characterized in that: The specific calculation formula of the fuzzy comprehensive evaluation result vector B is: According to the fuzzy comprehensive evaluation result vector, the final evaluation level of the system software is determined according to the maximum membership principle.

8. An electronic device, characterized in that: The method comprises at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a MOM software system evaluation method based on multiple scenarios as described in any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement a multi-scenario based MOM software system evaluation method as described in any one of claims 1 to 7.

Images