Method and system for quickly constructing quantitative evaluation model for deduction

By quickly building a quantitative evaluation model, the problem of difficult to quantify qualitative content in strategic decision-making is solved, quantitative evaluation is realized, computing efficiency and model adaptability are improved, and strategic game deduction is suitable for complex systems.

CN119089128APending Publication Date: 2024-12-06INST OF SOFTWARE - CHINESE ACAD OF SCI +2
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310693094.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-12
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Strategic decision-making issues are complex and qualitative content is difficult to quantify, and existing technologies are difficult to effectively solve quantitative evaluation problems in complex systems.

Method used

Using the method of quickly building a quantitative evaluation model, through three steps of data configuration, adversarial deduction and evaluation application, an intelligent model between resources, measures and indicators is established, and computational experiment methods and exploratory methods are used, combined with contribution function analysis measures. The influence of indicators is adjusted to fit the real world.

Benefits of technology

The quantitative relationship evaluation between measures and resources and measures and indicators in the process of strategic game is realized. It is versatile and flexible, and can quickly build an evaluation model and improve computing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089128A_ABST
    Figure CN119089128A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of efficiency evaluation, provides a method and a system for quickly constructing a quantitative evaluation model for deduction, and establishes an intelligent model representing the relationship among resources, measures and effects. The system mainly comprises three modules: a data configuration module used for establishing an index system and establishing an evaluation model used for deduction according to three elements of resources, measures and indexes; the confrontation deduction module is used for interacting with the user, inputting various measures input in the deduction process into the evaluation model, and calculating the influence of the measures on the indexes; and the evaluation application module is used for displaying the measure in the deduction process and the influence result of the measure on the index calculated by the evaluation model, analyzing the influence result of the measure on the index in combination with the actual situation of the real world, and adjusting the setting of the evaluation model. According to the method, effectiveness evaluation of measures in the game deduction process is realized, flexible adjustment can be performed according to the deduction field and object, and an evaluation model can be quickly constructed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of performance evaluation, and in particular relates to a method and system for rapidly constructing a quantitative evaluation model for deduction. Background Art

[0002] Strategic decision-making problems are typical complex system problems that cannot be solved by analytical calculations, modeling and simulation techniques. Using the idea of ​​"comprehensive integration seminar hall" proposed by Comrade Qian Xuesen, organizing experts in the field to study strategic decision-making problems through strategic game deduction is a method that has been proven to be effective in solving complex system problems by a large number of practices at home and abroad.

[0003] It has become an important core work for national think tanks to provide strategic decision-making measures and suggestions for serving various important units. In the face of an increasingly complex international situation, the use of strategic game deduction to solve strategic problems has become an indispensable research method. In the research process, data is an indispensable reference content when studying problems, and strategic problems are mostly qualitative content and difficult to quantify. Therefore, this invention draws on computational experimental methods and exploratory methods to conduct in-depth research to solve the problem that qualitative content is difficult to quantify.

[0004] Computational Experiments is a method for studying complex systems. Computational Experiments treats the "simulation" results as an alternative version of reality, or a possible reality; at the same time, the actual system is also regarded as a possible reality, "equivalent" to the simulation results, realizing the ideological shift from computational simulation to computational experiments. In computational experiments, traditional computational simulation becomes an "experimental" process in a "computational laboratory", a means of "growing and nurturing" various complex systems, and the actual system is only a possible result of this "computational experiment". For the evaluation of complex systems, computational experiments are considered to be a very vital method, or at least a beneficial attempt.

[0005] The exploratory method is one of the hot methods for studying complex war systems at home and abroad. It is an analytical method based on the integrity and uncertainty of the system. The goal of the exploratory method is to understand the impact of uncertainty factors on the problem being studied, and at the same time explore the various capabilities and strategies of the system that can complete the corresponding task requirements, so as to fully grasp various key factors, obtain flexible, efficient and adaptable problem-solving solutions, and achieve the purpose of capability planning and solution optimization. The biggest difficulty in implementing the exploratory method is the contradiction between large computing scale and limited computing power. How to improve computing efficiency is also the research focus of exploratory analysis.

[0006] After in-depth research on computational experimental methods and exploratory methods, the present invention has invented a method for quickly constructing a quantitative evaluation model for strategic game deduction in combination with actual strategic problems and research fields, which effectively solves the problem that qualitative content is difficult to quantify in the process of strategic game. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for quickly constructing a quantitative evaluation model for deduction, and to establish an intelligent model that can represent the relationship between resources, measures and levels.

[0008] The technical solution adopted by the present invention is:

[0009] A method for quickly building a quantitative evaluation model for deduction, mainly including three steps: data configuration step, adversarial deduction step and evaluation application step;

[0010] Data configuration steps: Establish an indicator system through the data configuration module, and establish an evaluation model for deduction based on the three elements of resources, measures, and indicators;

[0011] Adversarial simulation step: The adversarial simulation module interacts with the user, inputs various measures input during the simulation into the evaluation model, and calculates the impact of the measures on the indicators;

[0012] Evaluation application steps: The evaluation application module is used to display the measures used in the simulation process and the impact of the measures on the indicators calculated by the evaluation model. The impact of the measures on the indicators is analyzed in combination with the actual situation in the real world, and the settings of the evaluation model are adjusted.

[0013] Furthermore, the evaluation indicators in the indicator system are divided into three levels from coarse-grained to fine-grained, among which the first-level indicators are the overall level, which is a reflection of the level of each party. The second-level indicators include five indicators: funds, results, talents, institutions, and industries. Each second-level indicator contains at least one third-level indicator.

[0014] Furthermore, the evaluation model includes: given an initial trend forecast value of an indicator, each party consumes resources and implements measures based on the initial trend forecast value, and affects its own and other parties' indicator values; the indicator value is settled based on the initial trend forecast value and the impact of the measures on the indicator; the implemented measures will affect the available resources in the following period of time; and a review, summary and evaluation are conducted after the deduction is completed.

[0015] Furthermore, the impact of the measures on the indicators is represented by an impact matrix on the indicators at each moment after the measures are implemented.

[0016] Furthermore, after conducting strategic game simulations, the contribution function is used to analyze the effects of each measure type, showing the proportion of each type of measure's impact on the indicators. The contribution of the measure type is compared with the real world to evaluate whether the simulation and calculation model are consistent with the real world.

[0017] Furthermore, the deduction process includes the following stages:

[0018] In the simulation preparation stage, the simulation objectives are determined, simulation scenarios are edited, and corresponding simulation models are selected. If the model components do not meet the requirements, the model components need to be developed.

[0019] During the simulation implementation phase, the initial situation is generated based on the simulation scenario, and the participating parties formulate countermeasures based on the initial situation. The countermeasures of the participating parties are displayed in the comprehensive situation and saved in the database.

[0020] During the simulation and evaluation phase, the effectiveness of countermeasures is evaluated based on the preset model, that is, the impact function of the measures on the indicators is calculated, and the countermeasures formulated in the simulation implementation phase are reviewed and replayed based on the evaluation results. The model is adjusted based on the evaluation results and the actual situation in the real world. If the model cannot meet the requirements, feedback is provided to the system operation and maintenance personnel for development.

[0021] A system for quickly building a quantitative evaluation model for deduction, comprising:

[0022] The data configuration module is used to establish an indicator system and build an evaluation model for deduction based on the three elements of resources, measures and indicators;

[0023] The adversarial simulation module is used to interact with the user, input various measures input during the simulation into the evaluation model, and calculate the impact of the measures on the indicators;

[0024] The evaluation application module is used to display the measures in the simulation process and the impact of the measures on the indicators calculated by the evaluation model, analyze the impact of the measures on the indicators in combination with the actual situation in the real world, and adjust the settings of the evaluation model.

[0025] As mentioned above, a method for quickly building a quantitative evaluation model for deduction is used, in which: an evaluation index system is established in the data configuration module, and the relationship between resources, measures and indicators in the science and technology field during the deduction process is clarified; after using the system to conduct scientific and technological strategic game deduction, the contribution of each measure is calculated and analyzed.

[0026] The beneficial effects of the present invention are:

[0027] (1) The present invention realizes the effectiveness evaluation of measures in the game simulation process, and can effectively reflect the quantitative relationship between measures and resources, and between measures and indicators in the simulation process.

[0028] (2) The present invention is applicable to different deductions and can be flexibly adjusted according to the fields and objects of deduction, so as to quickly construct an evaluation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a framework diagram of the technology deduction system;

[0030] Figure 2 It is a relationship diagram of each part in the deduction process;

[0031] Figure 3 To simulate the system workflow diagram;

[0032] Figure 4 The contribution of different types of measures in the simulation process;

[0033] Figure 5 It is the contribution of different types of measures in different time periods during the simulation process. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0035] like Figure 1 As shown, a method for quickly constructing a quantitative evaluation model for deduction of the present invention mainly includes three modules: a data configuration module, a confrontation deduction module and an evaluation application module. Among them, the data configuration module establishes the corresponding relationship between the measure category and the indicator system through the management tool configured by the deduction model and selects the calculation function of each indicator. Before organizing the deduction, the director group selects the corresponding evaluation model, and generates the evaluation model of this deduction by fine-tuning some parameters, such as the indicator type affected by the measure, the function of the indicator calculation and the parameters of the function. The confrontation deduction module is the part that interacts with the user. The various measures input during the expert deduction process are input into the evaluation model to calculate the impact of the measures on the evaluation indicators. The evaluation application module displays the expert's measure data during the deduction process and the impact results of the measures calculated by the model on the indicators through charts, and gives an analysis report by analyzing the measures and the impact results. The director group analyzes the impact results of the measures on the indicators in combination with the actual situation in the real world and adjusts the settings of the evaluation model.

[0036] 1. Establish an indicator system

[0037] Table 1

[0038]

[0039] The establishment of the index system is completed by the data configuration module. Before retrieving data through the data configuration module, it is first necessary to establish a target level evaluation index system. The present invention mainly establishes an index system for measuring the levels of all parties in the deduction, and divides the system indicators into three levels, where the first-level indicators are the overall level, which is a reflection of the level of each party itself. Taking the field of science and technology as an example, it mainly includes five secondary indicators such as funds, achievements, talents, institutions, and industries. Each secondary indicator contains at least one tertiary indicator. The performance evaluation method in the present invention is to explore how system parameters affect system performance indicators based on data mining. When applied to different deduction industries and goals, the tertiary indicators themselves will change accordingly.

[0040] Taking the field of science and technology as an example, the specific indicator system is shown in Table 1, where the third-level indicators can be regarded as specific sub-field indicators, and the first-level indicators correspond to overall level indicators.

[0041] On the basis of the science and technology indicator system, the three-level indicators are used to calculate the impact of measures on the indicators. The present invention uses the analytic hierarchy process (AHP) to determine the weight of each indicator. Each evaluation indicator occupies a certain importance in the program evaluation, and its importance is reflected by the indicator weight, which is related to the guiding ideology of decision makers in formulating evaluation programs. For example, when evaluating the level of science and technology, is it focused on long-term development or short-term technical evaluation? If it focuses on the former, then the indicators that reflect long-term conditions, such as talents, are considered. The specific size of the indicator weight needs to be calculated and determined by consulting the decision-making body and users, and then calculated by a certain weight calculation method.

[0042] 2. Build a deduction model

[0043] After the indicator system is established, it is also necessary to design a deduction model, that is, an evaluation model for deduction, which is completed by the data configuration module. The three elements of resources, measures, and indicators are used to abstractly describe the operating mechanism of a multinational science and technology system with multiple game opponents. Indicators reflect the status of each party, and each party uses measures to change indicators. The release of measures consumes the resources of each party. Resources are determined by indicators. The relationship between the three is as follows: Figure 2 Among them, measures refer to the countermeasures taken by the participants in response to the current situation, such as issuing policies to increase investment in a certain field; resources refer to the human, financial, and material resources that can be used by the party during this exercise, such as the amount of money that can be invested in the measures.

[0044] The deduction model is a model that represents the relationship between the three elements of measures, resources, and indicators. Considering the dynamic changes in the science and technology system and the impact caused by measures, the deduction model is designed as follows: given the initial trend forecast value of the indicator, each party consumes resources and implements measures based on the initial trend forecast value, and affects its own and other parties' indicator values. The indicator value needs to be settled in combination with the initial trend forecast value and the impact of measures on the indicator. The implemented measures will affect the available resources for a period of time. After the deduction is completed, a review summary and evaluation will be conducted.

[0045] Assume that the deduction time T = {t0, t1, ..., t r ,…,t N}, is a discrete time; only the indicators and resource values ​​at discrete moments are considered. That is, the predicted indicator value at time t0 for:

[0046]

[0047] Each row represents a sub-indicator, and each column represents a moment in the simulation. r The indicator value at the moment is:

[0048]

[0049] t0,……,t r-1 The index values ​​at the moment are fixed, r ,……,t N The value at that moment is the predicted value.

[0050] The impact of measures on indicators is completed by the impact matrix of indicators at each moment after the implementation of measures. r If it is implemented at all times, the impact matrix on the indicators is:

[0051]

[0052] in, Indicates the impact value of the nth indicator affected by the measure at time r. The impact matrix of the measure on the indicator is calculated through the influence function (see below).

[0053] Afterwards, Δ M The impact is superimposed on On, get That is, t after the measures have an impact r+1 The indicator value at the moment.

[0054] In the present invention, a fuzzy algorithm is used to determine the effect of each measure and the affected object (i.e., the affected index). Due to the different types of measures in the deduction process, they affect different indicators, and even if they affect the same indicator, the impact of different measures is different, which makes it impossible to compare the measures. Therefore, it is not realistic to make accurate measurements. In this case, using the knowledge of experts in the field will help to unify the measurement of the effect of the measures. The index system in the present invention is all quantitative indicators, and the specific method of obtaining its influence function is as follows.

[0055] The influence function is given by the empirical function:

[0056]

[0057]

[0058]

[0059]

[0060] Where x represents the xth time point of the impact, n represents the parameter of the measure, and Γ represents the gamma function.

[0061] The influence function formula is determined by experts after they determine the maximum and minimum influence moments of the measure, and determine that the function value at the maximum influence moment reaches the maximum value, while the function value at the minimum influence moment is 0, and the function graph at other moments is a monotonous smooth curve, and then the influence function is determined through experience; after the definition domain is given, the function value is normalized to obtain the influence value acting on the indicator matrix. After determining the influence function, the influence function is calibrated by fitting the deduced results with real-world data.

[0062] The indicators and influence functions of the measures are listed in Table 2.

[0063] Considering the resource limitation, let the intensity of the measure be F, the duration of the measure be T, the resource input of the measure be M, and the impact function of the measure be f, then:

[0064] M=F∫0 T f(x)dx

[0065] After applying the system in practice and inviting experts to conduct scientific and technological strategic game deduction, the contribution function is used to analyze the effect of each measure category. The contribution calculation method of the measure category is as follows:

[0066]

[0067] For measure c i The contribution of the class in the deduction process; E sis the total effect of all measures in the simulation process, To remove measure c from the simulation process i The total effect of all other measures outside the class. By limiting the time, the contribution of the measure class before a certain moment can be observed. The purpose of calculating the contribution of the measure class is to show the proportion of the impact of various measures on the indicators, and to compare the contribution of the measure class with the real world, so as to better evaluate whether this deduction and calculation model conforms to the real world.

[0068] Table 2

[0069]

[0070] Figure 3 The following is the workflow of the simulation system, as shown in the figure, which specifically includes the following steps:

[0071] 1) Game preparation stage. In the game preparation stage, the goal of the game needs to be determined, the game scenario needs to be edited, and the corresponding simulation model (i.e., game model) needs to be selected. If the model component does not meet the requirements, the model component needs to be developed. The game scenario refers to the current basic situation, confrontation intention, and development situation of the parties involved in the exercise. It is written according to the game topic and purpose to create an immersive game environment for all parties involved.

[0072] 2) Simulation implementation stage. In the simulation implementation stage, the system generates an initial situation based on the simulation assumptions, and the participating parties formulate countermeasures based on the initial situation. The system displays the countermeasures of the participating parties in the comprehensive situation and saves them in the database.

[0073] 3) Simulation and evaluation stage. In the simulation and evaluation stage, the system evaluates the effectiveness of countermeasures based on the preset model (i.e., calculates the impact function of measures on indicators), and replays the countermeasures formulated during the simulation implementation stage based on the evaluation results. The director team analyzes the effectiveness evaluation results and the actual situation in the real world and adjusts the model. If the model does not meet the needs, it needs to be fed back to the system operation and maintenance personnel for development. The director team refers to the management personnel of the simulation process. The director team can control the progress of the exercise, such as starting, pausing, and ending the exercise; it can also release crisis events.

[0074] The overall contribution of different measures is as follows: Figure 4 As shown in the figure, the contribution intervals of different measures are as follows Figure 5 shown.

[0075] From the overall deduction, the two types of measures, self-development and sanctions and blockades, have the largest contribution. This is because in real life, one's own technological development and external sanctions and blockades have the greatest impact on the level of science and technology; the contribution of technical alliance measures is also high, and security review is low, but it is impossible to fully distinguish the impact of measures at different times. Therefore, it is necessary to calculate the contribution of measures by time period.

[0076] Judging from the contribution and classification of the measures, the Red team tends to use "self-development" and "technical alliance" measures, while sanctions measures are more aimed at countering the Blue team's censorship measures; while the Blue team's measures are mainly aimed at imposing sanctions on the Red team.

[0077] Judging from the overall effects of measures, development measures have the greatest long-term impact on scientific and technological levels, because development measures are the core and foundation of scientific and technological growth; the impact of joint measures at different stages is similar, because they are more flexible and can play a certain role at different stages; the use of sanctions measures is mainly focused on suppressing China, and the effect is more concentrated in the early stages, and its effect is limited in the later stages; review measures are very ineffective when the reviewed party is prepared, and the system contribution is relatively small.

[0078] Another embodiment of the present invention provides a system for quickly constructing a quantitative evaluation model for deduction, which includes:

[0079] The data configuration module is used to establish an indicator system and build an evaluation model for deduction based on the three elements of resources, measures and indicators;

[0080] The adversarial simulation module is used to interact with the user, input various measures input during the simulation into the evaluation model, and calculate the impact of the measures on the indicators;

[0081] The evaluation application module is used to display the measures in the simulation process and the impact of the measures on the indicators calculated by the evaluation model, analyze the impact of the measures on the indicators in combination with the actual situation in the real world, and adjust the settings of the evaluation model.

[0082] The specific implementation process of each module refers to the above description of the method of the present invention.

[0083] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0084] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0085] The present invention does not elaborate on some of the well-known technologies that belong to those skilled in the art.

[0086] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for quickly constructing a quantitative evaluation model for deduction, characterized in that: The following steps are involved: Establish an indicator system through the data configuration module, and establish an evaluation model for deduction based on the three elements of resources, measures and indicators; Through the interaction with users in the adversarial simulation module, various measures input during the simulation are input into the evaluation model to calculate the impact of the measures on the indicators; The evaluation application module displays the measures used in the simulation process and the impact of the measures on the indicators calculated by the evaluation model. The impact of the measures on the indicators is analyzed in combination with the actual situation in the real world, and the settings of the evaluation model are adjusted.

2. The method according to claim 1, characterized in that The evaluation indicators in the indicator system are divided into three levels from coarse-grained to fine-grained. The first-level indicators are the overall level, which reflects the level of each party. The second-level indicators include five indicators: funds, results, talents, institutions, and industries. Each second-level indicator contains at least one third-level indicator.

3. The method according to claim 1, characterized in that The evaluation model includes: given the initial trend forecast value of the indicator, each party consumes resources and implements measures based on the initial trend forecast value, and affects its own and other parties' indicator values; the indicator value is settled based on the initial trend forecast value and the impact of the measures on the indicator; the implemented measures will affect the available resources in the next period of time; and a review summary and evaluation is conducted after the deduction is completed.

4. The method according to claim 1, characterized in that: The impact of the measures on the indicators is represented by the impact matrix of the indicators at each moment after the implementation of the measures. Suppose that the measure M is r If it is implemented at all times, the impact matrix on the indicators is: in, It represents the impact value of the nth indicator affected by the measure at time r; the influence matrix of the measure on the indicator is calculated through the influence function.

5. The method according to claim 4, characterized in that The influence function is: Where x represents the xth time point of the impact, n represents the parameter of the measure, and Γ represents the gamma function.

6. The method according to claim 1, characterized in that After the strategic game simulation, the contribution of each measure class is used to analyze the effect of each measure class, showing the proportion of each measure's impact on the indicator. The contribution of the measure class is compared with the real world to evaluate whether the simulation and calculation model are consistent with the real world. The calculation method of the contribution of the measure class is as follows: in, For measure c i The contribution of the class in the deduction process, E s is the total effect of all measures in the simulation process, To remove measure c from the simulation process i The total effect of all other measures outside the category.

7. The method according to claim 1, characterized in that The deduction process includes the following stages: In the simulation preparation stage, the simulation objectives are determined, simulation scenarios are edited, and corresponding simulation models are selected. If the model components do not meet the requirements, the model components need to be developed. During the simulation implementation phase, the initial situation is generated based on the simulation scenario, and the participating parties formulate countermeasures based on the initial situation. The countermeasures of the participating parties are displayed in the comprehensive situation and saved in the database. During the simulation and evaluation phase, the effectiveness of countermeasures is evaluated based on the preset model, that is, the impact function of the measures on the indicators is calculated, and the countermeasures formulated in the simulation implementation phase are reviewed and replayed based on the evaluation results. The model is adjusted based on the evaluation results and the actual situation in the real world. If the model cannot meet the needs, feedback is provided to the system operation and maintenance personnel for development.

8. A system for rapidly constructing a quantitative evaluation model for deduction, characterized in that: include: The data configuration module is used to establish an indicator system and build an evaluation model for deduction based on the three elements of resources, measures and indicators; The adversarial simulation module is used to interact with the user, input various measures input during the simulation into the evaluation model, and calculate the impact of the measures on the indicators; The evaluation application module is used to display the measures in the simulation process and the impact of the measures on the indicators calculated by the evaluation model, analyze the impact of the measures on the indicators in combination with the actual situation in the real world, and adjust the settings of the evaluation model.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Multi-Agent-based dynamic equipment security system efficiency evaluation method and system

    CN105825022A

  • Combat deduction simulation and efficiency evaluation integrated method

    CN112052608A

  • Efficiency evaluation method based on massive adversarial simulation deduction data modeling and analysis

    CN114239228A