Generation method and device of version issuing evaluation document, storage medium and computer equipment

Abstract

The invention discloses a version issuing evaluation document generation method and device, a storage medium and computer equipment, and the method comprises the steps: obtaining an initial version issuing evaluation document, carrying out the typesetting of the initial version issuing evaluation document based on the preset evaluation parameters of the version issuing, obtaining an intermediate version issuing evaluation document, and carrying out the typesetting of the intermediate version issuing evaluation document; the preset evaluation parameter comprises a concerned performance index corresponding to at least one tested model of the present edition; the tested models are tested through a perception evaluation platform, a test link of each tested model is traversed, test data of the tested models are obtained in the test links, and the test data comprise model identifiers and model performance test data of the tested models; and determining attention performance test data of the tested model based on the test data and an attention performance index of the tested model, and writing the attention performance test data into the intermediate edition issuing evaluation document to form a target edition issuing evaluation document of the edition issuing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method and device for generating a release evaluation document, a storage medium, and a computer device. Background Art

[0002] With the rapid development of digital technology, the version update speed of various application programs and intelligent devices has also been continuously accelerating. The version update of application programs and intelligent devices is inseparable from the iterative update of the models they rely on. How to accurately and quickly perform performance testing and test result analysis on the models used in the new version of application programs, intelligent devices, etc. is an important factor affecting the version update speed.

[0003] In the prior art, technicians test each model used in the new version of the application program and intelligent device one by one, and then analyze the test results of the models one by one, with low efficiency, which affects the release efficiency of application programs, intelligent devices, etc. Summary of the Invention

[0004] In view of this, this application provides a method and device for generating a release evaluation document, a storage medium, and a computer device, which improves the automation degree of writing the evaluation document, helps to improve the model testing efficiency and test result analysis efficiency, and helps to further improve the release efficiency.

[0005] According to one aspect of this application, a method for generating a release evaluation document is provided. The method includes:

[0006] Obtain an initial release evaluation document, and perform typesetting on the initial release evaluation document based on the preset evaluation parameters of this release to obtain an intermediate release evaluation document, where the preset evaluation parameters include at least one concerned performance index corresponding to at least one tested model of this release;

[0007] Traverse the test links of each tested model, and obtain the test data of the tested model in the test link, where the test data includes the model identifier and model performance test data of the tested model;

[0008] Based on the test data and the concerned performance index of the tested model, determine the concerned performance test data of the tested model, and write the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document of this release.

[0009] Optionally, each tested model corresponds to at least one version and at least one test data set; the testing of the tested model by the perception evaluation platform includes:

[0010] Through the perception evaluation platform, different versions of the model under test are tested using each test data set corresponding to the model under test, and the test data is saved in a test link corresponding to the test content, where the test data further includes the version number of the model under test and the test data set identifier corresponding to the test data;

[0011] Correspondingly, writing the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release includes:

[0012] Writing the concerned performance test data, the version number of the model under test corresponding to the concerned performance test data, and the test data set identifier into the intermediate release evaluation document to form the target release evaluation document for this release.

[0013] Optionally, writing the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release includes:

[0014] Obtaining the preset test criteria for each model under test, where the preset test criteria include the test passing conditions for the concerned performance indicators of the model under test;

[0015] For each model under test, based on the preset test criteria and the concerned performance test data of the model under test, determining the test results for each version corresponding to the model under test;

[0016] Writing the concerned performance test data of each model under test into the intermediate release evaluation document and marking the test results for each version of each model under test to form the target release evaluation document for this release.

[0017] Optionally, after determining the test results for each version corresponding to the model under test, the method further includes:

[0018] If at least one version of each model under test passes the test, generating the test passing information for this release and / or generating the release model recommendation information for this release, where the test passing information includes the version numbers that pass the test corresponding to each model under test, and the release model recommendation information includes the model version numbers corresponding to the optimal test results for each model under test;

[0019] If none of the versions of any first model under test pass the test, generating the first test failure information for this release, where the first test failure information includes the first model under test identifier.

[0020] Optionally, after determining the test results for each version corresponding to the model under test, the method further includes:

[0021] If all versions of any second tested model in the tested model fail the test, traverse the test results of each test data set corresponding to the second tested model, and for any test data set corresponding to the second tested model, determine whether there is at least one version of the second tested model that passes the test on this test data set;

[0022] If so, based on the test scenarios of each test data set and the versions of the second tested model that pass the test on each test data set, determine the scenario mapping relationship between the test scenarios and the version numbers of the second tested model, and based on the mapping relationship, generate the release model fusion recommendation information for this release, where the release model fusion recommendation information includes the second tested model identifier and the scenario mapping relationship corresponding to the second tested model;

[0023] If not, generate the second test failure information for this release, where the second test failure information includes the second tested model identifier.

[0024] Optionally, after generating the release model fusion recommendation information for this release, the method further includes:

[0025] Based on the test scenarios and their corresponding version numbers of the second tested model in the scenario mapping relationship, determine the scenarios to be fused and the version numbers of the models to be fused, and obtain the models to be fused corresponding to the version numbers to be fused;

[0026] Obtain a preset scenario classification model, and based on the scenarios to be fused, determine the model mapping relationship between the classification results of the scenario classification model and the models to be fused;

[0027] Based on the scenario classification model, the models to be fused, and the model mapping relationship, construct the second tested fusion model corresponding to the second tested model, where the second tested fusion model is used to classify the target data to be calculated through the scenario classification model, and then input the target data into the model to be fused that matches the scenario classification result for calculation.

[0028] Optionally, the obtaining of the initial release evaluation document includes:

[0029] Create the initial release evaluation document in a preset document platform; or, open the document in a preset path in the preset document platform as the initial release evaluation document;

[0030] Encapsulate the document style function for the initial release evaluation document;

[0031] Correspondingly, after writing the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release, the method further includes:

[0032] Storing the target release evaluation document in the preset document platform, generating a document link corresponding to the target release evaluation document, and sending the document link to a preset terminal;

[0033] When the preset document platform receives an access request for the document link, reading the target release evaluation document and invoking the document style function to visually display the target release evaluation document.

[0034] According to another aspect of the present application, there is provided a device for generating a release evaluation document, the device including:

[0035] A document layout module, configured to obtain an initial release evaluation document and layout the initial release evaluation document based on preset evaluation parameters for this release to obtain an intermediate release evaluation document, where the preset evaluation parameters include concerned performance indicators corresponding to at least one tested model for this release;

[0036] A data acquisition module, configured to traverse the test links of each of the tested models, and obtain the test data of the tested models in the test links, where the test data includes the model identifier and model performance test data of the tested models;

[0037] A document generation module, configured to determine the concerned performance test data of the tested models based on the test data and the concerned performance indicators of the tested models, and write the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release.

[0038] Optionally, each of the tested models corresponds to at least one version and at least one test data set; the data acquisition module is further configured to:

[0039] Through the perception evaluation platform, use each test data set corresponding to the tested models to test different versions of the tested models, and save the test data in the test links corresponding to the test content, where the test data further includes the tested model version number and the test data set identifier corresponding to the test data;

[0040] Correspondingly, the document generation module is further configured to:

[0041] Write the concerned performance test data, the tested model version number corresponding to the concerned performance test data, and the test data set identifier into the intermediate release evaluation document to form the target release evaluation document for this release.

[0042] Optionally, the document generation module is further configured to:

[0043] Obtain the preset test criteria for each of the models under test, where the preset test criteria include the test passing conditions for the performance indicators of interest of the models under test;

[0044] For each of the models under test, determine the test results for each version of the model under test based on the preset test criteria and the performance test data of interest of the model under test;

[0045] Write the performance test data of interest of each of the models under test into the intermediate release evaluation document, and mark the test results for each version of each of the models under test to form the target release evaluation document for this release.

[0046] Optionally, the apparatus further includes: an information output module, configured to:

[0047] If at least one version of each of the models under test passes the test, generate the test passing information for this release and / or generate the recommended information for the release models for this release, where the test passing information includes the version numbers that pass the test corresponding to each of the models under test, and the recommended information for the release models includes the model version numbers corresponding to the optimal test results for each of the models under test;

[0048] If none of the versions of any first model under test pass the test, generate the first test failure information for this release, where the first test failure information includes the first model under test identifier.

[0049] Optionally, the apparatus further includes: an information output module, configured to:

[0050] If none of the versions of any second model under test pass the test, traverse the test results of each test data set corresponding to the second model under test, and for any test data set corresponding to the second model under test, determine whether there is at least one version of the second model under test that passes the test on this test data set;

[0051] If so, determine the scenario mapping relationship between the test scenario and the second model under test version number based on the test scenarios of each test data set and the second model under test versions that pass the test on each test data set, and generate the recommended information for the integrated release models for this release based on the mapping relationship, where the recommended information for the integrated release models includes the second model under test identifier and the scenario mapping relationship corresponding to the second model under test;

[0052] Otherwise, generate a second test failure message for this release, where the second test failure message includes a second tested model identifier.

[0053] Optionally, the apparatus further includes: a model fusion module, configured to:

[0054] Based on the test scenarios in the scenario mapping relationship and their corresponding second tested model version numbers, determine the scenarios to be fused and the version numbers of the models to be fused, and obtain the models to be fused corresponding to the version numbers to be fused;

[0055] Obtain a preset scenario classification model, and determine the model mapping relationship between the classification result of the scenario classification model and the model to be fused according to the scenarios to be fused;

[0056] Construct a second tested fusion model corresponding to the second tested model according to the scenario classification model, the model to be fused, and the model mapping relationship, where the second tested fusion model is used to classify the target data to be calculated through the scenario classification model, and then input the target data into the model to be fused that matches the scenario classification result for calculation.

[0057] Optionally, the document layout module is further configured to:

[0058] Create the initial release evaluation document in a preset document platform; or, open the document in a preset path in the preset document platform as the initial release evaluation document;

[0059] Encapsulate the document style function for the initial release evaluation document;

[0060] Correspondingly, the apparatus further includes: a document display module, configured to:

[0061] Store the target release evaluation document in the preset document platform, generate a document link corresponding to the target release evaluation document, and send the document link to a preset terminal;

[0062] When the preset document platform receives an access request for the document link, read the target release evaluation document, and call the document style function to visually display the target release evaluation document.

[0063] According to another aspect of the present application, there is provided a storage medium, on which a computer program is stored, and when the program is executed by a processor, the above method for generating a release evaluation document is implemented.

[0064] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the program, the above method for generating a release evaluation document is implemented.

[0065] By means of the above technical solution, a method and device for generating a release evaluation document, a storage medium, and a computer device provided by the present application typeset a document according to the performance indicators of each model required for this release. After testing each model required for this release, the performance test data corresponding to each tested model is obtained by analyzing the test data, and the performance test data is written into a unified release evaluation document to form a target release evaluation document containing the performance test data of all tested models. In the embodiments of the present application, a perception evaluation platform is used to uniformly test each model required for release, and the performance test data of each model is analyzed according to the test data and written into a unified release evaluation document, which improves the automation degree of writing evaluation documents, helps to improve the model test efficiency and test result analysis efficiency, and further helps to improve the release efficiency.

[0066] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. In order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. Brief Description of the Drawings

[0067] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0068] Figure 1 A flowchart showing a method for generating a release evaluation document provided by an embodiment of the present application is shown;

[0069] Figure 2 A flowchart showing another method for generating a release evaluation document provided by an embodiment of the present application is shown;

[0070] Figure 3 A schematic diagram of an intermediate release evaluation document provided by an embodiment of the present application is shown;

[0071] Figure 4 A schematic diagram of the structure of a fusion model provided by an embodiment of the present application is shown;

[0072] Figure 5 A schematic diagram of the structure of a device for generating a release evaluation document provided by an embodiment of the present application is shown;

[0073] Figure 6 The figure shows a schematic diagram of the device structure of a computer device provided by an embodiment of the present application. Specific embodiments

[0074] In the following, the present application will be described in detail with reference to the accompanying drawings and in combination with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0075] In this embodiment, a method for generating a release evaluation document is provided. As Figure 1 shown, the method includes:

[0076] Step 101, obtain an initial release evaluation document, and typeset the initial release evaluation document based on preset evaluation parameters for the current release, to obtain an intermediate release evaluation document, where the preset evaluation parameters include at least one attention performance index corresponding to the models to be tested in the current release.

[0077] The embodiment of the present application can achieve unified testing of all models involved in a release and unified output of test results. First, obtain a blank initial release evaluation document, and use the preset evaluation parameters pre-constructed according to the attention parameters of the models for the current release to perform typesetting design on the initial release evaluation document, to obtain a typeset intermediate release evaluation document. Among them, the preset evaluation parameters include the attention performance indexes of each model to be tested in the current release. For example, the evaluation parameters of model A include accuracy and recall rate, and the evaluation parameters of model 2 include precision rate, accuracy, recall rate, and so on. When performing typesetting design on the initial evaluation document, the document can be designed in the form of a table, and the test results of different models can be displayed in one document by designing different table headers.

[0078] Step 102, test the models to be tested through a perception evaluation platform, and traverse the test links of each model to be tested, and obtain the test data of the models to be tested in the test links, where the test data includes the model identifier and model performance test data of the models to be tested.

[0079] Secondly, the unified testing of all models to be tested in the current release is realized through a unified testing platform, that is, the models to be tested in the current release are tested through the perception evaluation platform, and after the test is completed, the test links URL corresponding to each model to be tested is traversed, and the test data of each test is obtained in different test links. When the test data is stored in the test link, not only the performance test data of the model is saved, but also the model identifier of the model is saved, so as to distinguish the model to be tested corresponding to the test data.

[0080] Step 103: Based on the test data and the concerned performance metrics of the model under test, determine the concerned performance test data of the model under test, and write the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release.

[0081] Finally, write the test data of each model under test obtained from the test link into the typeset intermediate release document to form the target release evaluation document for unified display. Specifically, the model performance test data in the test data can be extracted and analyzed according to the concerned performance metrics of each model under test to obtain the concerned performance test data of the model under test, and then the concerned performance test data is written into the document. For example, if the concerned performance metrics of model A include the average accuracy of model A on different test sets, then the accuracy of model A on each test set should be obtained first for average calculation, and then the calculated average accuracy of model A is written into the document to achieve statistical analysis of multiple test results, so that each test of the model is no longer isolated, but the test results of different models under different conditions can be analyzed uniformly.

[0082] By applying the technical solution of this embodiment, the typesetting of the document is carried out according to the concerned performance metrics of each model required for this release. After testing each model required for this release, the concerned performance test data corresponding to each model under test is obtained by analyzing the test data, and the concerned performance test data is written into the unified release evaluation document to form the target release evaluation document containing the concerned performance test data of all models under test. The embodiment of the present application uses the perception evaluation platform to uniformly test each model required for release, and analyzes the concerned performance test data of each model according to the test data and writes it into the unified release evaluation document, which improves the automation degree of writing the evaluation document, helps to improve the model test efficiency and the test result analysis efficiency, and helps to further improve the release efficiency.

[0083] Further, as a refinement and extension of the specific implementation manner of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another method for generating a release evaluation document is provided, as Figure 2 shown. This method includes:

[0084] Step 201: Create the initial release evaluation document in a preset document platform; or, open the document in a preset path in the preset document platform as the initial release evaluation document.

[0085] Step 202: Encapsulate the document style function for the initial release evaluation document; typeset the initial release evaluation document based on the preset evaluation parameters for this release to obtain an intermediate release evaluation document, where the preset evaluation parameters include the performance indicators of interest corresponding to at least one model under test for this release; each model under test corresponds to at least one version and at least one test data set.

[0086] In the embodiments of the present application, any document platform can be selected as the carrier for generating the evaluation document. For example, the Feishu document platform is selected, and a specified document is created or opened in the preset document platform as the initial release evaluation document. Then, the document style function of this document is encapsulated and typeset for subsequent visual display of the document. Among them, in order to ensure the success rate of the release in one release, each model under test can have multiple versions. In addition, multiple test data sets can be set for each model under test, and different test data sets can be set according to the scenarios corresponding to the data under test. For example, in this release of the intelligent driving system, it includes the image recognition model A. This image recognition model corresponds to 3 test data sets, namely the test data set for the outdoor daytime scenario, the test data set for the outdoor nighttime scenario, and the test data set for the indoor scenario. As Figure 3 shown, it is a schematic diagram of an intermediate release evaluation document.

[0087] Step 203: Through the perception evaluation platform, use each test data set corresponding to each model under test to test different versions of the model under test, save the test data in the test link corresponding to the test content, and traverse the test links of each model under test to obtain the test data of the model under test in the test link, where the test data includes the model identifier of the model under test, the model performance test data, the version number of the model under test, and the test data set identifier corresponding to the test data.

[0088] In the above embodiment, after abstracting and standardizing the evaluation processes of each model under test in the perception evaluation platform in advance to endow the perception evaluation platform with the test ability for each model under test, after configuring different versions and different test data sets of each model under test for the perception evaluation platform, each model under test can be uniformly tested through the perception evaluation platform, and the test data is stored in a unique test link that matches the test content one by one according to the different test contents. After the test, by traversing each test link, all the test data is obtained in the test link, where each piece of test data includes the model identifier of the corresponding model under test, the version number of the model under test, the test data set identifier, and the model performance test data.

[0089] Step 204: Determine the performance test data of interest for the model under test based on the test data and the performance indicators of interest for the model under test.

[0090] Step 205: Obtain the preset test criteria for each of the models under test, where the preset test criteria include the passing conditions for the test of the performance indicators of interest of the models under test; for each of the models under test, determine the test results of each version corresponding to the model under test based on the preset test criteria of the model under test and the performance test data of interest.

[0091] Step 206: Write the performance test data of interest of each of the models under test, as well as the version number of the model under test and the test dataset identifier corresponding to the performance test data of interest, into the intermediate release evaluation document, and mark the test results of each version of each of the models under test to form the target release evaluation document for this release.

[0092] In the above embodiment, after obtaining the test data of each version of the models under test under different test datasets, calculate the performance test data of interest of each of the models under test respectively, and then use the performance test data of interest to analyze whether the models under test pass the test. Specifically, for each of the models under test, the passing conditions for the test of the performance indicators of interest of each of the models under test can be written in advance, so as to form the preset test criteria for each of the models under test. Thus, after obtaining the performance test data of interest of each of the models under test, obtain the preset test criteria for each of the models under test, and combine this criterion to determine whether each of the models under test passes the test. Taking one of the models under test, model A, as an example, model A includes two versions, A.1 and A.2, and the performance indicators of interest include indicator 1 and indicator 2. After the test, determine whether the corresponding indicator 1 and indicator 2 of model A.1 meet the corresponding passing conditions for the test, and determine whether the corresponding indicator 1 and indicator 2 of model A.2 meet the corresponding passing conditions for the test, to obtain the test results of two versions of model A. Thus, fill the performance test data of interest of each of the models under test, as well as the version number of the model under test and the test dataset identifier corresponding to the performance test data of interest, in the document. In addition, the test results of different versions of each of the models under test can also be marked in the document to form the final target release evaluation document for this release.

[0093] Step 207: Store the target release evaluation document in the preset document platform, generate a document link corresponding to the target release evaluation document, and send the document link to a preset terminal.

[0094] Step 208: When the preset document platform receives an access request to the document link, read the target release evaluation document, and call the document style function to visually display the target release evaluation document.

[0095] In the above embodiments, the finally formed target release evaluation document is saved in a preset document platform, and a document link for querying this document is generated. The document link is sent to a preset terminal for receiving release evaluation results. Testers can access the document link through the preset terminal to view the target release evaluation document, or forward the document link to other terminals for other users to view the target release evaluation document by accessing this document link. When the preset document platform receives an access request for the document link from any terminal device, it can read out the target release evaluation document corresponding to this document link, and perform visual display of this document by calling a pre-encapsulated document style function.

[0096] In an embodiment of the present application, optionally, after completing the testing of each tested model, the test result of this release can also be determined by analyzing the test results of each tested model. After determining the test results of each version corresponding to the tested model, the method further includes at least one of S1 - S3:

[0097] S1: If each of the tested models has at least one version passing the test, then generate the test passing information for this release and / or generate the release model recommendation information for this release. Among them, the test passing information includes the test passing version numbers corresponding to each of the tested models, and the release model recommendation information includes the model version numbers of the optimal test results corresponding to each of the tested models.

[0098] In this embodiment, if each tested model has one or more versions passing the test, then it can be considered that this release passes the test. Among them, a certain version of a tested model passing the test means that the concerned performance test data of this version of the tested model on each test data set meets the preset test standard. At this time, the test passing information can be generated and sent to the tester's terminal for notification. It is also possible to determine the version with the best test effect among each tested model as the model version recommended for release based on the versions of each tested model passing the test, so as to generate the release model recommendation information. This recommendation information includes the model version numbers of the optimal test results corresponding to each of the tested models, so as to recommend the model version to the tester and improve the automation of the release process. For example, if there are 3 tested models, the version numbers with the best test results corresponding to these 3 tested models are recommended respectively.

[0099] S2: If all versions of any first tested model among the tested models fail to pass the test, then generate the first test failure information for this release. Among them, the first test failure information includes the first tested model identifier.

[0100] In this embodiment, if none of the versions of a model under test pass the test, it can be considered that the model fails the test. At this time, the first test failure information for this release can be generated. For example, if the test results of any version of the first model under test in the model under test are all unqualified, the first test failure information carries the first model under test identifier.

[0101] S3: If none of the versions of any second model under test in the model under test pass the test, traverse the test results of each test data set corresponding to the second model under test, and for any test data set corresponding to the second model under test, determine whether there is at least one version of the second model under test that passes the test on this test data set;

[0102] If so, based on the test scenarios of each test data set and the version of the second model under test that passes the test on each test data set, determine the scenario mapping relationship between the test scenario and the second model version number, and based on the mapping relationship, generate the release model fusion recommendation information for this release, where the release model fusion recommendation information includes the second model under test identifier and the scenario mapping relationship corresponding to the second model under test;

[0103] If not, generate the second test failure information for this release, where the second test failure information includes the second model under test identifier.

[0104] In this embodiment, in some cases where the release time is relatively tight or it is difficult to further optimize the model parameters, even if the test results of a model under test show that none of its versions pass the test, the problem of the model test failure affecting the release can be overcome by the method of multi-version model fusion. Specifically, if none of the versions of any second model under test in the model under test pass the test, the test results of each version of the second model under test on different test data sets can be further analyzed. For each test data set of the second model under test, it is determined one by one whether there is at least one version of the second model under test that can pass the test on this test data set. For example, the second model under test B includes versions 1, 2, and 3, and the corresponding test data sets include data set 1, data set 2, and data set 3. Then it is successively determined whether there is a version of model B that can pass the test on data set 1, whether there is at least one version of model B that can pass the test on data set 2, and whether there is at least one version of model B that can pass the test on data set 3.

[0105] If at least one version of the second model under test passes the test for each test data set, that is, the judgment result is yes, then the scenario mapping relationship between the test scenario and the model version number can be established according to the test scenario of each test data set and the version number of the second model under test that passes the test on each test data set, and the release model fusion recommendation information can be generated based on this mapping relationship. For example, if model versions 1 and 2 of the second model under test B pass the test on data set 1, model versions 2 and 3 pass the test on data set 2 of the second model under test B, and model version 3 passes the test on data set 3 of the second model under test B, then the scenario mapping relationships between data set 1 and versions 1 and 2, data set 2 and versions 2 and 3, and data set 3 and version 3 can be established. So as to perform subsequent model fusion based on this mapping relationship.

[0106] If no version of the second model under test passes the test for a certain test data set, then the second test failure information for this release can be generated, and this information includes the identifier of the second model under test to prompt the developer to continue to optimize and train the second model under test.

[0107] In the embodiments of the present application, optionally, model fusion can also be performed based on the release model fusion recommendation information in S3, including: determining the scenario to be fused and the version number of the model to be fused based on the test scenario and the corresponding version number of the second model under test in the scenario mapping relationship, and obtaining the model to be fused corresponding to the version number to be fused; obtaining a preset scenario classification model, and determining the model mapping relationship between the classification result of the scenario classification model and the model to be fused according to the scenario to be fused; constructing a second fused model corresponding to the second model under test according to the scenario classification model, the model to be fused, and the model mapping relationship, where the second fused model is used to input the target data to be calculated into the model to be fused that matches the scenario classification result for calculation after classifying the target data to be calculated through the scenario classification model.

[0108] In this embodiment, when performing model fusion, first, each test scenario in the scenario mapping relationship is used as the scenario to be fused, and the second version number of the tested model corresponding to the test scenario in the scenario mapping relationship is used to select the models to be fused. Specifically, the models to be fused with the minimum number that can meet the conditions can be selected. For example, the scenario mapping relationship includes version 1 and version 2 corresponding to dataset 1; version 2 and version 3 corresponding to dataset 2; version 3 corresponding to dataset 3, and dataset 1, 2, and 3 correspond to the outdoor daytime scenario, outdoor nighttime scenario, and indoor scenario respectively. It can be determined that the scenarios to be fused are the outdoor daytime scenario, outdoor nighttime scenario, and indoor scenario, and the models to be fused are the version 2 model and the version 3 model. Next, a scenario classification model is obtained. The classification results corresponding to the scenario classification model include the above-mentioned scenarios to be fused. Then, based on each scenario to be fused, the model mapping relationship between the classification results of the scenario classification model and the models to be fused is determined. For example, the outdoor daytime scenario corresponds to the version 2 model, the outdoor nighttime scenario corresponds to the version 2 model, and the indoor scenario corresponds to the version 3 model. Finally, based on the scenario classification model, the models to be fused, and the above-mentioned model mapping relationship, a fusion model is constructed. Specifically, as Figure 4 shown in the schematic diagram of the model structure of a fusion model, after scene classification is performed by the scenario classification model in the fusion model, the target data to be calculated is then input into the model to be fused corresponding to the scene classification result for data prediction. Suppose a piece of data to be calculated. First, the scenario classification model is used to determine that the scenario of this data is the outdoor daytime scenario, and then this data is input into the version 2 model in the fusion model for calculation.

[0109] It should be noted that for the tested models that have undergone model fusion, the input data format of the original tested models can also be changed to a certain extent. For example, a scene flag bit is added, so that the scene flag bit can be recognized by the scenario classification model in the fusion model to determine the scene classification result, and then the data to be calculated after removing the scene flag bit is input into the model to be fused corresponding to this classification result.

[0110] Through the above model fusion method, the problem that the current tested models cannot meet the use requirements of all scenarios can be solved to a certain extent, the release efficiency can be improved, and it is helpful for the rapid release and online of new versions.

[0111] Furthermore, as Figure 1 a specific implementation of the method, an embodiment of the present application provides a device for generating a release evaluation document, as Figure 5 shown, the device includes:

[0112] A document layout module, configured to obtain an initial release evaluation document and perform layout on the initial release evaluation document based on preset evaluation parameters of this release to obtain an intermediate release evaluation document, where the preset evaluation parameters include at least one performance index of interest corresponding to the tested models of this release;

[0113] A data acquisition module, configured to traverse the test links of each of the models under test, and acquire the test data of the models under test from the test links, wherein the test data includes the model identifier of the model under test and the model performance test data;

[0114] A document generation module, configured to determine the concerned performance test data of the model under test based on the test data and the concerned performance indicators of the model under test, and write the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release.

[0115] Optionally, each model under test corresponds to at least one version and at least one test data set; the data acquisition module is further configured to:

[0116] Through the perception evaluation platform, use each test data set corresponding to each model under test to test different versions of the models under test, and save the test data in the test link corresponding to the test content, wherein the test data further includes the version number of the model under test and the test data set identifier corresponding to the test data;

[0117] Correspondingly, the document generation module is further configured to:

[0118] Write the concerned performance test data, the version number of the model under test corresponding to the concerned performance test data, and the test data set identifier into the intermediate release evaluation document to form the target release evaluation document for this release.

[0119] Optionally, the document generation module is further configured to:

[0120] Obtain the preset test criteria of each model under test, wherein the preset test criteria include the test passing conditions of the concerned performance indicators of the model under test;

[0121] For each model under test, determine the test results of each version corresponding to the model under test based on the preset test criteria and the concerned performance test data of the model under test;

[0122] Write the concerned performance test data of each model under test into the intermediate release evaluation document, and mark the test results of each version of each model under test to form the target release evaluation document for this release.

[0123] Optionally, the device further includes: an information output module, configured to:

[0124] If at least one version of each of the models to be tested passes the test, generate the test pass information for this release and / or generate the recommended release model information for this release. Among them, the test pass information includes the test pass version number corresponding to each of the models to be tested, and the recommended release model information includes the model version number with the best test result corresponding to each of the models to be tested;

[0125] If all versions of any first model to be tested among the models to be tested do not pass the test, generate the first test failure information for this release. Among them, the first test failure information includes the first model to be tested identifier.

[0126] Optionally, the device further includes: an information output module, configured to:

[0127] If all versions of any second model to be tested among the models to be tested do not pass the test, traverse the test results of each test data set corresponding to the second model to be tested, and for any test data set corresponding to the second model to be tested, determine whether there is at least one version of the second model to be tested that passes the test on this test data set;

[0128] If so, based on the test scenarios of each test data set and the version of the second model to be tested that passes the test on each test data set, determine the scenario mapping relationship between the test scenario and the second model version number, and based on the mapping relationship, generate the recommended release model fusion information for this release. Among them, the recommended release model fusion information includes the second model to be tested identifier and the scenario mapping relationship corresponding to the second model to be tested;

[0129] If not, generate the second test failure information for this release. Among them, the second test failure information includes the second model to be tested identifier.

[0130] Optionally, the device further includes: a model fusion module, configured to:

[0131] Based on the test scenarios in the scenario mapping relationship and their corresponding second model version numbers, determine the scenario to be fused and the version number of the model to be fused, and obtain the model to be fused corresponding to the version number to be fused;

[0132] Obtain a preset scenario classification model, and based on the scenario to be fused, determine the model mapping relationship between the classification result of the scenario classification model and the model to be fused;

[0133] Construct a second measured fusion model corresponding to the second measured model according to the scene classification model, the fused model, and the model mapping relationship, where the second measured fusion model is used to classify the target data to be calculated through the scene classification model and then input the target data into the fused model that matches the scene classification result for calculation.

[0134] Optionally, the document layout module is further configured to:

[0135] Create the initial release evaluation document in a preset document platform; or, open the document in a preset path in the preset document platform as the initial release evaluation document;

[0136] Encapsulate the document style function for the initial release evaluation document;

[0137] Correspondingly, the device further includes: a document display module, configured to:

[0138] Store the target release evaluation document in the preset document platform, generate a document link corresponding to the target release evaluation document, and send the document link to a preset terminal;

[0139] When the preset document platform receives an access request to the document link, read the target release evaluation document and call the document style function to visually display the target release evaluation document.

[0140] It should be noted that for other corresponding descriptions of each functional unit involved in the device for generating a release evaluation document provided in the embodiments of the present application, reference can be made to Figures 1 to 2 the corresponding descriptions in the method, which will not be elaborated here.

[0141] The embodiments of the present application further provide a computer device, specifically a personal computer, a server, a network device, etc. As Figure 6 shown, the computer device includes a bus, a processor, a memory, and a communication interface, and may further include an input / output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements the steps in the method embodiments.

[0142] Those skilled in the art can understand, Figure 6The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0143] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and has a computer program stored thereon. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0144] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0146] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0147] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0148] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for generating a release evaluation document, characterized in that, The method includes: Obtaining an initial release evaluation document, and typesetting the initial release evaluation document based on preset evaluation parameters for this release to obtain an intermediate release evaluation document, where the preset evaluation parameters include at least one concerned performance indicator corresponding to each tested model for this release; Testing the tested models through a perception evaluation platform, and traversing the test links of each tested model to obtain the test data of the tested models in the test links, where the test data includes the model identifier and model performance test data of the tested models; Based on the test data and the concerned performance indicators of the tested models, determining the concerned performance test data of the tested models, and writing the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release.

2. The method according to claim 1, wherein Each tested model corresponds to at least one version and at least one test data set; the testing of the tested models through the perception evaluation platform includes: Through the perception evaluation platform, using each test data set corresponding to each tested model to test different versions of the tested models, and saving the test data in the test link corresponding to the test content, where the test data further includes the version number of the tested model and the test data set identifier corresponding to the test data; Correspondingly, the writing of the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for this release includes: Writing the concerned performance test data, the version number of the tested model corresponding to the concerned performance test data, and the test data set identifier into the intermediate release evaluation document to form the target release evaluation document for this release.

3. The method according to claim 2, characterized in that, The writing of the concerned performance test data, the version number of the tested model corresponding to the concerned performance test data, and the test data set identifier into the intermediate release evaluation document to form the target release evaluation document for this release includes: Obtaining the preset test criteria for each tested model, where the preset test criteria include the test passing conditions for the concerned performance indicators of the tested models; For each tested model, determining the test results of each version corresponding to the tested model based on the preset test criteria and the concerned performance test data of the tested model; Writing the concerned performance test data of each tested model, the version number of the tested model corresponding to the concerned performance test data, and the test data set identifier into the intermediate release evaluation document, and marking the test results of each version of each tested model to form the target release evaluation document for this release.

4. The method according to claim 3, characterized in that, After determining the test results of each version corresponding to the tested model, the method further includes: If at least one version of each tested model passes the test, generating test passing information for this release and / or generating release model recommendation information for this release, where the test passing information includes the version numbers that pass the test corresponding to each tested model, and the release model recommendation information includes the model version numbers with the optimal test results corresponding to each tested model; If all versions of any first tested model in the tested model fail the test, generate first test failure information for the current release, where the first test failure information includes a first tested model identifier.

5. The method according to claim 3, wherein After determining the test results of each version of the tested model, the method further includes: If all versions of any second tested model in the tested model fail the test, traverse the test results of each test data set corresponding to the second tested model, and for any test data set corresponding to the second tested model, determine whether there is at least one version of the second tested model that passes the test on this test data set; If so, based on the test scenarios of each test data set and the versions of the second tested model that pass the test on each test data set, determine the scenario mapping relationship between the test scenarios and the second tested model version numbers, and based on the mapping relationship, generate release model fusion recommendation information for the current release, where the release model fusion recommendation information includes a second tested model identifier and the scenario mapping relationship corresponding to the second tested model; If not, generate second test failure information for the current release, where the second test failure information includes a second tested model identifier.

6. The method according to claim 5, wherein After generating the release model fusion recommendation information for the current release, the method further includes: Based on the test scenarios and their corresponding second tested model version numbers in the scenario mapping relationship, determine the scenario to be fused and the version number of the model to be fused, and obtain the model to be fused corresponding to the version number to be fused; Obtain a preset scenario classification model, and based on the scenario to be fused, determine the model mapping relationship between the classification result of the scenario classification model and the model to be fused; Based on the scenario classification model, the model to be fused, and the model mapping relationship, construct a second fused model corresponding to the second tested model, where the second fused model is used to classify the target data to be calculated through the scenario classification model and then input the target data into the model to be fused that matches the scenario classification result for calculation.

7. The method according to any one of claims 1 to 6, characterized in that, The obtaining of the initial release evaluation document includes: Create the initial release evaluation document in a preset document platform; or, open the document in a preset path in the preset document platform as the initial release evaluation document; Encapsulate the document style function for the initial release evaluation document; Correspondingly, after writing the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for the current release, the method further includes: Store the target release evaluation document in the preset document platform, generate a document link corresponding to the target release evaluation document, and send the document link to a preset terminal; When the preset document platform receives an access request for the document link, read the target release evaluation document and call the document style function to visually display the target release evaluation document.

8. An apparatus for generating a release evaluation document, characterized in that, The device includes: A document layout module, configured to obtain an initial release evaluation document and perform layout on the initial release evaluation document based on preset evaluation parameters for the current release, so as to obtain an intermediate release evaluation document, wherein the preset evaluation parameters include at least one concerned performance index corresponding to the model under test for the current release; A data acquisition module, configured to traverse the test links of each model under test and acquire the test data of the model under test from the test links, wherein the test data includes the model identifier and the model performance test data of the model under test; A document generation module, configured to determine the concerned performance test data of the model under test based on the test data and the concerned performance index of the model under test, and write the concerned performance test data into the intermediate release evaluation document to form the target release evaluation document for the current release.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images