Method and device for checking an index

Abstract

The application discloses an index optimization method and device of automatic integrated verification, and the method comprises the following steps: acquiring a first basic index set comprising a plurality of basic indexes for a current test task, wherein the plurality of basic indexes are obtained by summarizing basic indexes in a plurality of basic index lists; performing optimization screening on the plurality of basic indexes in the first basic index set according to the priority of the plurality of basic indexes to obtain a second basic index set, wherein the second basic index set comprises at least part of the plurality of basic indexes; determining a first quantity difference value of the number of basic indexes contained in the first basic index set and the second basic index set; comparing the first quantity difference value with an average value of the first quantity difference value in historical data which is determined according to a plurality of same tests; and in response to the first quantity difference value being smaller than the average value and the difference between the first quantity difference value and the average value being greater than a preset threshold, sending prompt information that an index optimization screening standard is not met.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an automatic integrated verification index optimization method and apparatus, electronic equipment, non-transient computer-readable storage medium, and computer program product. Background Technology

[0002] After the experimental requirements are constructed and the capability requirements and basic indicators are generated, a large amount of basic indicator data is accumulated. Extensive data queries and data correlation operations are required to analyze and filter the required indicators, thereby achieving the goal of basic indicator integration and optimization. Optimizing the basic indicators of capability requirements is a crucial task, verifying issues in the requirement generation, gap analysis, and indicator integration processes, and ensuring the final generation of the capability requirement list.

[0003] After the integration of basic capability requirements indicators is completed, the data is large and complex, covering a wide range of capabilities with varying levels of intensity. Staff need to optimize, filter, and uniformly analyze these indicators across different areas to confirm redundancy, missing data, and whether the overall indicators meet the requirements. This process is labor-intensive, slow, and heavily influenced by subjective factors, potentially leading to errors in judgment. Furthermore, the need for repeated verification of basic indicators across different projects reduces staff motivation. Summary of the Invention

[0004] This disclosure provides an automatic integrated verification index optimization method and apparatus, electronic device, computer-readable storage medium, and computer program product.

[0005] According to one aspect of this disclosure, an automatic integrated verification index optimization method is provided, comprising: for the current experimental task, obtaining a first set of basic indicators including multiple basic indicators, wherein the multiple basic indicators are obtained by summarizing the basic indicators in the multiple basic indicator lists; optimizing and filtering the multiple basic indicators according to the priority of the multiple basic indicators in the first set of basic indicators to obtain a second set of basic indicators, wherein the second set of basic indicators contains at least some of the multiple basic indicators; determining a first quantity difference in the number of basic indicators contained in the first set of basic indicators and the second set of basic indicators; comparing the first quantity difference with the average value of the first quantity difference determined from multiple similar experiments in historical data; and in response to the first quantity difference being less than the average value and the difference between the two being greater than a preset threshold, sending a prompt message indicating that the index optimization and filtering criteria are not met.

[0006] According to another aspect of this disclosure, an automatic integrated verification index optimization device is provided, comprising: an acquisition unit configured to acquire, for the current experimental task, a first set of basic indicators including multiple basic indicators, wherein the multiple basic indicators are obtained by summarizing the basic indicators in the multiple basic indicator lists; an optimization unit configured to optimize and filter the multiple basic indicators according to the priority of the multiple basic indicators in the first set of basic indicators to obtain a second set of basic indicators, wherein the second set of basic indicators contains at least some of the basic indicators; a determination unit configured to determine a first quantity difference in the number of basic indicators contained in the first set of basic indicators and the second set of basic indicators; a comparison unit configured to compare the first quantity difference with the average value of the first quantity difference determined from multiple similar experiments in historical data; and a sending unit configured to send a prompt message indicating that the index optimization screening criteria are not met in response to the first quantity difference being less than the average value and the difference between the two being greater than a preset threshold.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the above-described method.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing a computer program, wherein the computer program implements the above-described method when executed by a processor.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-described method when executed by a processor.

[0010] According to one or more embodiments of the present invention, the optimization level of the current experiment can be evaluated based on historical data. When the optimization level is poor, relevant prompts can be sent to the user to facilitate secondary optimization screening or manual optimization screening. The method of the present invention can ensure that the optimization screening of basic indicators meets the expected standards, so as to minimize the number of basic indicators after aggregation, thereby reducing the difficulty of subsequent manual operation and analysis of basic indicators.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings exemplify embodiments and form part of the specification, serving to explain exemplary implementations of the embodiments together with the textual description. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements. In the drawings:

[0013] Figure 1 A flowchart of an automatic integrated verification index optimization method according to an embodiment of the present disclosure is shown;

[0014] Figure 2 A flowchart of an automatic integrated verification index optimization method according to another embodiment of the present disclosure is shown;

[0015] Figure 3 A flowchart illustrating a method for obtaining a second set of basic indicators according to an embodiment of this disclosure is shown;

[0016] Figure 4 A schematic diagram of an operation interface for performing an application of an index optimization method according to an embodiment of the present disclosure is shown;

[0017] Figure 5 A flowchart of an automatic integrated verification index optimization method according to another embodiment of the present disclosure is shown;

[0018] Figure 6 A structural block diagram of an index optimization apparatus for automatic integrated verification according to an embodiment of the present disclosure is shown;

[0019] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0022] The terminology used in the description of the various examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0023] In order to alleviate, mitigate or eliminate at least one technical problem in the related art, according to one or more embodiments of the present disclosure, an automatic integrated verification index optimization method is proposed.

[0024] Before detailing the exemplary embodiments of this disclosure, a brief introduction to the background technology of this application will be given first. The method of the embodiments of this application can be used to determine the priority order of improving multiple basic indicators of a test subject after conducting experiments on the test subject.

[0025] When testing complex test subjects, it may be necessary to test multiple performance indicators for that subject. These indicators may belong to different levels, with higher-level indicators potentially containing multiple lower-level indicators. In one example, the test subject could be various equipment, such as military equipment. The unit undertaking the equipment testing task could test multiple performance indicators of the military equipment (e.g., a tank), such as the tank's attack capability, mobility, and defensive capabilities. These performance indicators are high-level indicators, representing some of the tank's comprehensive capabilities. High-level indicators further include multiple lower-level indicators; for example, the tank's attack capability further includes attack range, attack frequency, and firepower, etc. These indicators are multiple lower-level indicators contained within the high-level indicator of attack capability.

[0026] Therefore, it's understandable that high-level indicators for a particular experiment can be subdivided into multiple lower-level indicators. During the specific experiment, the experiment requirement data for that experiment can be received first. This data describes the different levels of experimental requirements for the experiment, indicating the expected values ​​for the corresponding level of indicators. Using a tank as an example, the requirement data could be something like requiring the tank to achieve a firing range of 1000m. After the experiment, a tree-structured capability indicator system data corresponding to the experiment requirement data can be constructed to represent the actual capabilities of the experimental subject. This tree-structured capability indicator system data includes a root node and multiple levels of child nodes and leaf nodes belonging to the root node. Leaf nodes represent the lowest level of experimental capability required to complete different experimental tasks. Each leaf node includes a corresponding list of basic indicators, which contains multiple basic indicators representing the lowest level of indicators in this experiment; that is, the basic indicators cannot be further subdivided.

[0027] Figure 1 A flowchart of an automatic integrated verification metric optimization method 100 according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method 100 includes:

[0028] Step 110: For the current experimental task, obtain a first set of basic indicators, which includes multiple basic indicators. The multiple basic indicators are obtained by summarizing the basic indicators in the multiple basic indicator lists.

[0029] Step 120: Optimize and filter multiple basic indicators according to the priority of multiple basic indicators in the first basic indicator set to obtain a second basic indicator set, which contains at least some of the basic indicators among the multiple basic indicators.

[0030] Step 130: Determine the first quantity difference between the number of basic indicators contained in the first basic indicator set and the second basic indicator set;

[0031] Step 140: Compare the first quantity difference with the average of the first quantity differences determined from multiple similar trials in historical data; and

[0032] Step 150: In response to the first quantity difference being less than the average value and the difference between the two being greater than a preset threshold, a prompt message indicating that the indicator optimization screening criteria are not met is sent.

[0033] Basic indicators can have different types. The type of basic indicator can be distinguished based on the type of physical quantity it represents. The type of basic indicator can include time, distance, etc. For example, a basic indicator could be that the test tank's engagement range is 1000m, then the type of basic indicator could be distance. In step 110, the indicator type to which all basic indicators belong is determined, and then they are classified according to the indicator type. For example, there can be multiple basic indicators of the same type, distance. One basic indicator could be that the test tank's engagement range is 1000m, and another basic indicator could be that the test tank's detection range is 800m; both of these basic indicators belong to the "distance" type. As mentioned above, the high-level indicators of the test mission can be subdivided into multiple low-level indicators. Therefore, after the test, multiple leaf nodes of the tree-structured capability indicator system data can be obtained, each leaf node including a corresponding list of basic indicators. In step 110, the first set of basic indicators is obtained by summarizing the basic indicators from the multiple basic indicator lists generated by the current test mission. The above summarization refers to merging all basic indicators from multiple basic indicator lists according to their type.

[0034] Each basic indicator has a predetermined priority, which indicates that the basic indicator must be satisfied first in this experiment. In step 120, the basic indicators in the first set of basic indicators are optimized and filtered according to their priorities to obtain a second set of basic indicators. For example, the basic indicators with higher priority can be retained, while the basic indicators with lower priority can be deleted, thus forming the second set of basic indicators. It can be understood that after the above optimization and filtering, the second set of basic indicators will contain only at least a portion of the basic indicators.

[0035] In step 130, the number of basic indicators in the first basic indicator set is subtracted from the number of basic indicators in the second basic indicator set to obtain a first quantity difference. The first quantity difference represents the degree of optimization screening described above. When the first quantity difference is large, it indicates a high degree of optimization screening; when the first quantity difference is small, it indicates a low degree of optimization screening.

[0036] Prior to this experimental task, there were multiple historical experimental tasks. Each historical experimental task determined a first quantity difference between the number of basic indicators contained in the corresponding first and second basic indicator sets. The average value was obtained by averaging the multiple first quantity differences obtained from the multiple experiments. The above average value represents the average degree of optimization screening in the historical experiments. In step 140, the first quantity difference is compared with the average value of the first quantity differences determined from multiple similar experiments in the historical data to determine whether the degree of optimization screening in this experiment meets the standard.

[0037] In step 150, if the difference between the first quantity and the average value is less than the average value, and the difference between the two is greater than a preset threshold, it indicates that the optimization and screening of the basic indicators in this experiment is insufficient. In this case, a prompt message indicating that the indicator optimization and screening criteria are not met will be sent. The above prompt message can be sent to the relevant operation interface of the terminal being operated by the experimenter, who can then perform filtering and optimization of the displayed basic indicators and other operations through this operation interface.

[0038] According to one or more embodiments of the present invention, the optimization level of the current experiment can be evaluated based on historical data. When the optimization level is poor, relevant prompts can be sent to the user to facilitate secondary optimization screening or manual optimization screening. The method of the present invention can ensure that the optimization screening of basic indicators meets the expected standards, so as to minimize the number of basic indicators after aggregation, thereby reducing the difficulty of subsequent manual operation and analysis of basic indicators.

[0039] Figure 2 A flowchart of an automatic integration verification index optimization method 200 according to another embodiment of the present disclosure is shown.

[0040] like Figure 2 As shown, the method includes:

[0041] Step 210: Obtain the first keyword set related to multiple basic indicators in the second basic indicator set, and the second keyword set extracted according to the experimental requirements of the current experimental task;

[0042] Step 220: Compare the first keyword set and the second keyword set;

[0043] Step 230: In response to determining that the first keyword set is a non-empty subset of the second keyword set, a prompt message indicating missing basic indicators is sent.

[0044] In step 210, for each basic indicator in the second set of basic indicators, a first keyword related to that basic indicator can be obtained, thus forming a first keyword set. The first keyword can be related to the type of the basic indicator. For example, if a basic indicator is a strike distance of 1000m, then the first keyword could be distance. Additionally, second keywords related to the initial experimental requirements can be extracted. In some embodiments, the experimental requirements can be a text document outlining the overall requirements for the entire experiment. The keyword extraction described above can be implemented using a relevant keyword extraction model. The text of the experimental requirements is input into the relevant model, and multiple second keywords are output to generate a second keyword set.

[0045] In step 220, if all second keywords in the second keyword set include all first keywords in the first keyword set, then the first keyword set is determined to be a non-empty subset of the second keyword set. It can be understood that during the optimization and screening of the first set of basic indicators, although the number of basic indicators decreases, the elements or types of basic indicators do not decrease; otherwise, a certain basic indicator would be missing. In step 230, if the first keyword set is determined to be a non-empty subset of the second keyword set, it indicates that a basic indicator has been missing during the optimization and screening process. A prompt message indicating a missing basic indicator can be sent to the relevant operation interface of the terminal being used by the experimenter.

[0046] In this embodiment, the system can verify whether any basic indicators are missing during the optimization and filtering process based on keyword information. If a missing basic indicator is detected, a relevant prompt message is sent to the user so that they can re-optimize and filter the basic indicators. This method prevents the deletion of crucial basic indicators during the optimization process, thereby further ensuring the accuracy of the optimization and filtering.

[0047] Figure 3A flowchart of a method 300 for obtaining a second set of basic indicators according to an embodiment of this disclosure is shown, such as... Figure 3 As shown, the method 300 includes:

[0048] Step 310: Classify the multiple basic indicators in the first set of basic indicators to obtain multiple types of basic indicators;

[0049] Step 320: Prioritize the basic metrics included in each type; and

[0050] Step 330: For each type of basic indicators, select a portion of the basic indicators from the basic indicators of that type according to the priority of the basic indicators to form a second set of basic indicators.

[0051] As mentioned above, the multiple basic indicators in the first set of basic indicators have various types. In step 310, the multiple basic indicators in the first set of basic indicators can be classified to obtain multiple types of basic indicators. Figure 4 A schematic diagram of an operation interface 400 for performing an index optimization method according to embodiments of the present disclosure is shown. Figure 4 As shown, the basic indicators in the first set of basic indicators have been classified into four types: damage rate, strike adaptability, damage strike time, and hit rate. Each type contains at least one basic indicator.

[0052] In step 320, the priority of each basic indicator included in each type can be predetermined based on the relevant parameters of that basic indicator. These relevant parameters include, but are not limited to, the difference between the expected and actual values ​​of the basic indicator, the cost of the test equipment required to test the basic indicator, and the number of all basic indicators contained in the leaf node to which the basic indicator belongs. The priority of each determined basic indicator can be represented by a priority score. A basic indicator with a higher score indicates a higher priority, and thus can be ranked before other lower priority basic indicators in step 320.

[0053] In step 330, for each type of basic indicator, only at least one basic indicator with higher priority can be retained, while lower priority basic indicators can be deleted, thus forming a second set of basic indicators. For example... Figure 4 As shown, taking the basic indicators of the damage strike time type as an example, this type contains 3 basic indicators. In the optimization and screening process, only two of the higher priority basic indicators (the basic indicators that are checked in the figure) are retained, and the other low priority basic indicator (the basic indicator that is not checked in the figure) is deleted.

[0054] Figure 5A flowchart of an index optimization method 500 according to another embodiment of the present disclosure is shown. Figure 5 As shown, the method 500 includes:

[0055] Step 510: Determine whether the user has re-optimized and filtered multiple basic indicators in the first set of basic indicators;

[0056] Step 520: In response to the user re-optimizing and filtering multiple basic indicators in the first basic indicator set, a third basic indicator set obtained through user optimization and filtering is determined. The third basic indicator set includes at least some of the basic indicators in the multiple basic indicators of the first basic indicator set.

[0057] Step 530: Determine the second quantity difference between the number of basic indicators contained in the first basic indicator set and the third basic indicator set;

[0058] Step 540: Store the second quantity difference in historical data for use in calculating the average quantity difference in subsequent trial tasks; and

[0059] Step 550: In response to the user not re-optimizing and filtering multiple basic indicators in the first basic indicator set, the first quantity difference is stored in historical data for use in calculating the average value of the first quantity difference in subsequent experimental tasks.

[0060] In step 510, if the user is dissatisfied with the optimized filtering results or receives a prompt message indicating that the indicator optimization filtering criteria are not met, they can choose to actively re-optimize and filter multiple basic indicators in the first basic indicator set. Based on the user's active optimization filtering results, a third basic indicator set is generated. In step 510, the user can re-add deleted basic indicators or delete filtered and retained basic indicators through operations on the operation interface 400.

[0061] In step 540, if the user re-optimizes and filters the first set of basic indicators, the second quantity difference after re-optimization and filtering will be used as the data to be saved in the historical data. In step 550, if the user does not re-optimize and filter the first set of basic indicators, it means that the result of this optimization and filtering meets the user's expectations. In this case, the aforementioned first quantity difference of this experiment can be directly stored in the historical data.

[0062] In this embodiment, the aforementioned first or second quantity difference will be stored in historical data. When determining the average value of the quantity differences in the historical data in subsequent experiments, the data of the first or second quantity difference recorded in this experiment will be used as a reference. Therefore, the data from this experiment will be used for index optimization in subsequent experiments.

[0063] According to another aspect of this disclosure, an automatic integrated verification index optimization device is also provided. Figure 6 A structural block diagram of an automatic integrated verification index optimization apparatus 600 according to an embodiment of the present disclosure is shown. Figure 6 As shown, the device 600 includes: an acquisition unit 610 configured to acquire a first set of basic indicators including multiple basic indicators for the current experimental task, wherein the multiple basic indicators are obtained by summarizing the basic indicators in the multiple basic indicator lists; an optimization unit 620 configured to optimize and filter the multiple basic indicators according to the priority of the multiple basic indicators in the first set of basic indicators to obtain a second set of basic indicators, wherein the second set of basic indicators contains at least some of the basic indicators; a determination unit 630 configured to determine a first quantity difference in the number of basic indicators contained in the first set of basic indicators and the second set of basic indicators; a comparison unit 640 configured to compare the first quantity difference with the average value of the first quantity difference determined from multiple similar experiments in historical data; and a sending unit 650 configured to send a prompt message indicating that the indicator optimization and filtering criteria are not met in response to the first quantity difference being less than the average value and the difference between the two being greater than a preset threshold.

[0064] According to another aspect of this disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the method described above.

[0065] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing a computer program is also provided, wherein the computer program implements the method described above when executed by a processor.

[0066] According to another aspect of this disclosure, a computer program product is also provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the method described above.

[0067] See Figure 7 The following description serves as a structural block diagram of the electronic device 700 disclosed herein, which is an example of a hardware device applicable to various aspects of this disclosure. The electronic device can be different types of computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the disclosure described and / or claimed herein.

[0068] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. (As follows) Figure 7 As shown, the electronic device 700 may include at least one processor 701, working memory 702, input unit 704, display unit 705, speaker 706, storage unit 707, communication unit 708 and other output units 709 that are capable of communicating with each other via system bus 703.

[0069] Processor 701 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 701 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Processor 701 may be configured to acquire and execute computer-readable instructions stored in working memory 702, storage unit 707, or other computer-readable media, such as program code of operating system 702a, program code of application program 702b, etc.

[0070] Working memory 702 and storage unit 707 are examples of computer-readable storage media for storing instructions that are executed by processor 701 to perform the various functions described above. Working memory 702 may include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, storage unit 707 may include hard disk drives, solid-state drives, removable media including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Working memory 702 and storage unit 707 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code that can be executed by processor 701 as a specific machine configured to perform the operations and functions described in the examples herein.

[0071] Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal input related to user settings and / or function control of electronic device, and can include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone and / or remote control. Output unit can be any type of device capable of presenting information, and can include, but is not limited to, display unit 705, speaker 706 and other output units 709. Other output units 709 can include, but are not limited to, video / audio output terminals, vibrators and / or printers. Communication unit 708 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and can include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and / or chipsets, such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0072] The application program 702b in working register 702 can be loaded to execute the various methods and processes described above. For example, in some embodiments, the image processing methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 707. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 700 via storage unit 707 and / or communication unit 708. When the computer program is loaded and executed by processor 701, one or more steps of the image processing methods described above can be performed. Alternatively, in other embodiments, processor 701 can be configured to execute the image processing methods by any other suitable means (e.g., by means of firmware).

[0073] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0074] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0075] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0076] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0077] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0078] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0079] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0080] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for optimizing metrics through automatic integration and verification, comprising: For the current test mission, a first set of basic indicators including multiple basic indicators is obtained. The multiple basic indicators are obtained by summarizing the basic indicators in the multiple basic indicator lists. The current test mission is a test mission for military equipment, and the multiple basic indicators are the performance indicators of the military equipment. Based on the priority of multiple basic indicators in the first set of basic indicators, the multiple basic indicators are optimized and filtered to obtain a second set of basic indicators, which includes at least some of the multiple basic indicators. Obtain a first set of keywords related to multiple basic indicators in the second set of basic indicators, and a second set of keywords extracted according to the experimental requirements of the current experimental task; Compare the first keyword set and the second keyword set; In response to determining that the first keyword set is a non-empty subset of the second keyword set, a prompt message indicating missing basic indicators is sent; Determine the first quantity difference between the number of basic indicators contained in the first set of basic indicators and the second set of basic indicators; Compare the first quantity difference with the average of the first quantity differences determined from multiple similar experiments in historical data; and In response to the first quantity difference being less than the average value, and the difference between the two being greater than a preset threshold, a prompt message indicating that the indicator optimization screening criteria are not met is sent.

2. The method as described in claim 1, wherein, Based on the priority of multiple basic indicators in the first set of basic indicators, the multiple basic indicators are optimized and filtered to obtain the second set of basic indicators, which includes: The basic indicators in the first set of basic indicators are classified to obtain multiple types of basic indicators; Prioritize the basic metrics included in each type; and For each type of basic indicators, a subset of basic indicators are selected from the basic indicators of that type according to their priority to form the second set of basic indicators.

3. The method of claim 2, further comprising: In response to the user not re-optimizing and filtering the multiple basic indicators in the first basic indicator set, the first quantity difference is stored in the historical data for use in calculating the average value of the first quantity difference in subsequent experimental tasks.

4. The method of claim 2, further comprising: In response to the user re-optimizing and filtering multiple basic indicators in the first basic indicator set, a third basic indicator set obtained through user optimization and filtering is determined, wherein the third basic indicator set includes at least some of the basic indicators in the multiple basic indicators of the first basic indicator set. Determine a second quantity difference between the number of basic indicators contained in the first set of basic indicators and the third set of basic indicators; as well as The second quantity difference is stored in the historical data for use in calculating the average value of the first quantity difference in subsequent experimental tasks.

5. An automatic integrated verification index optimization device, comprising: The acquisition unit is configured to acquire a first set of basic indicators, including multiple basic indicators, for the current test task. The multiple basic indicators are obtained by summarizing the basic indicators in the multiple basic indicator lists. The current test task is a test task for military equipment, and the multiple basic indicators are the performance indicators of the military equipment. The optimization unit is configured to optimize and filter the multiple basic indicators according to the priority of the multiple basic indicators in the first basic indicator set to obtain a second basic indicator set, the second basic indicator set containing at least some of the multiple basic indicators; and to obtain a first keyword set related to the multiple basic indicators in the second basic indicator set, and a second keyword set extracted according to the experimental requirements of the current experimental task. The determining unit is configured to determine a first quantity difference between the number of basic indicators contained in the first set of basic indicators and the second set of basic indicators. The comparison unit is configured to compare the first quantity difference with the average of the first quantity differences determined from multiple similar experiments in historical data; compare the first keyword set and the second keyword set; and The sending unit is configured to send a prompt message indicating that the index optimization screening criteria are not met when the difference between the first quantity and the average value is less than the average value and the difference between the two is greater than a preset threshold; and to send a prompt message indicating that the basic index is missing when the first keyword set is determined to be a non-empty subset of the second keyword set.

6. An electronic device, comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

8. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method of any one of claims 1-4.

Images