Chip evaluation report generation method and device, storage medium and electronic equipment

Abstract

The invention relates to a chip evaluation report generation method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence, and the method comprises the steps: determining a to-be-tested chip and to-be-tested performance associated with the to-be-tested chip, obtaining original chip performance data corresponding to the to-be-tested chip and a historical test report example associated with the to-be-tested performance; based on a preset report outline generation model, according to the to-be-tested performance and the historical test report examples, generating a test report outline corresponding to the to-be-tested chip; generating a performance test result according to the original chip performance data based on the content generation large model, and mapping the performance test result to a test report outline to obtain an original chip evaluation report; and evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report. According to the invention, the generation efficiency of the evaluation report is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of artificial intelligence, and in particular, to a chip evaluation report generation method and device, a computer readable storage medium, and an electronic device. BACKGROUND

[0002] In the prior art, a chip evaluation report can be generated in the following manner: first, a tester needs to filter out test data related to an intelligent chip from a MySQL database; second, a test purpose, test environment description, test process description, and data chart drawing and analysis are sequentially written in a Word document by manual operation according to industry standards or internal requirements. However, this method has the following defects: on the one hand, the entire process relies entirely on manual operation, which reduces the efficiency of generating the evaluation report; on the other hand, since the evaluation report is manually written, the accuracy of the evaluation report is reduced due to human error.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The present disclosure aims to provide a chip evaluation report generation method and device, a computer readable storage medium, and an electronic device, thereby at least partially overcoming the problems of low efficiency and low accuracy of generating an evaluation report due to the limitations and defects of related technologies.

[0005] According to one aspect of the present disclosure, a chip evaluation report generation method is provided, comprising:

[0006] determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested;

[0007] generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example;

[0008] generating a performance test result based on a preset content generation model according to the original chip performance data, and mapping the performance test result to the test report outline to obtain an original chip evaluation report;

[0009] The original chip evaluation report is evaluated based on a preset report evaluation model to obtain a report evaluation result, and a target chip evaluation report is determined according to the report evaluation result and the original chip evaluation report.

[0010] According to an aspect of the present disclosure, a chip evaluation report generation device is provided, comprising:

[0011] A to-be-tested chip generation module is configured to determine a to-be-tested chip and a to-be-tested performance associated with the to-be-tested chip, and obtain original chip performance data corresponding to the to-be-tested chip and a historical test report example associated with the to-be-tested performance.

[0012] A test report outline generation module is configured to generate a test report outline corresponding to the to-be-tested chip based on a preset report outline generation model according to the to-be-tested performance and the historical test report example.

[0013] A performance test result generation module is configured to generate a performance test result based on a preset content generation model according to the original chip performance data, and map the performance test result to the test report outline to obtain an original chip evaluation report.

[0014] A chip evaluation report generation module is configured to evaluate the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determine a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

[0015] According to an aspect of the present disclosure, a computer readable storage medium having a computer program stored thereon is provided, wherein the computer program is executed by a processor to implement the chip evaluation report generation method of any one of the above.

[0016] According to an aspect of the present disclosure, an electronic device is provided, comprising:

[0017] A processor; and

[0018] A memory configured to store executable instructions of the processor;

[0019] The processor is configured to execute the executable instructions to implement the chip evaluation report generation method of any one of the above.

[0020] The chip evaluation report generation method provided by the embodiment of the present disclosure, on the one hand, by determining the to-be-tested chip and the to-be-tested performance associated with the to-be-tested chip, and obtaining the original chip performance data corresponding to the to-be-tested chip and the historical test report example associated with the to-be-tested performance, then generating a test report outline corresponding to the to-be-tested chip based on the preset report outline generation model according to the to-be-tested performance and the historical test report example, and then generating a performance test result based on the preset content generation model according to the original chip performance data, and mapping the performance test result to the test report outline to obtain an original chip evaluation report, and finally evaluating the original chip evaluation report based on the preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report, the automatic generation of the target chip evaluation report is realized, and the generation efficiency of the target chip evaluation report is improved; on the other hand, since the original chip evaluation report can be evaluated based on the preset report evaluation model to obtain a report evaluation result, and a target chip evaluation report is determined according to the report evaluation result and the original chip evaluation report, the accuracy of the obtained target chip evaluation report is improved.

[0021] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained according to these drawings without creative labor for those skilled in the art.

[0023] Figure 1 A flowchart of a chip evaluation report generation method according to an example embodiment of the present disclosure is schematically shown.

[0024] Figure 2 A framework example diagram of a chip evaluation report generation system according to an example embodiment of the present disclosure is schematically shown.

[0025] Figure 3 A structure example diagram of a large language model according to an example embodiment of the present disclosure is schematically shown.

[0026] Figure 4 A structure example diagram of a hybrid expert model in a large language model according to an example embodiment of the present disclosure is schematically shown.

[0027] Figure 5A scenario example diagram of a selected chip to be tested according to an example embodiment of the present disclosure is schematically shown.

[0028] Figure 6 A scenario example diagram of a selected performance to be tested corresponding to a chip to be tested according to an example embodiment of the present disclosure is schematically shown.

[0029] Figure 7 A scenario example diagram of a resulting raw text outline according to an example embodiment of the present disclosure is schematically shown.

[0030] Figure 8 A scenario example diagram of a resulting raw chip evaluation report according to an example embodiment of the present disclosure is schematically shown.

[0031] Figure 9 A structure example diagram of a chip evaluation report generation apparatus according to an example embodiment of the present disclosure is schematically shown.

[0032] Figure 10 An electronic device for implementing a chip evaluation report generation method according to an example embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0033] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0034] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and serve to explain the principles of the present disclosure. Features, elements, and / or properties that are

[0035] In the evaluation report generation scene of the intelligent chip, the existing feasible evaluation report generation methods mainly include but are not limited to manual generation, template engine and large model text generation. Among them:

[0036] The specific implementation process of the manual generation method is as follows: the tester first needs to select the test data related to the intelligent chip from the MySQL database; wherein, the test data can include the original data table of the inference throughput, inference time delay and aggregate communication bandwidth of the intelligent chip in different scenes, which also records the hardware configuration information in the test log in detail, such as the model of the intelligent chip, the memory capacity, the storage architecture and the software environment parameters; the software environment parameters can include but are not limited to the operating system version, the driver program version and the inference framework version number, etc. Further, in writing the test report, the Word document can be used as the carrier, and the test purpose, test environment description, test process description, data chart drawing and analysis and other contents can be written in turn according to the industry standard or enterprise internal requirement. For example, when analyzing the performance of the artificial intelligence (AI, Artificial Intelligence) inference chip, the throughput data under different concurrency numbers need to be compared manually, and then the change trend is shown by drawing a line chart, and finally the analysis conclusion of the application scene is written combined with the chip architecture characteristics. The defects of this method are: the whole process completely depends on manual operation, not only the tester needs to have deep professional knowledge, but also from data arrangement to report formation, the writing period of a single report is as long as 2-3 working days, which makes the generation efficiency of the test report low.

[0037] The specific implementation process of the template engine method is as follows: the enterprise designs a standardized report template based on Word in advance, and uses placeholders (such as ${chip model}, ${test result}, ${environment configuration}) as data filling identifiers; on this basis, the technical personnel can write script programs through programming languages such as Python or Java to establish the mapping relationship between the database fields and the placeholders. In specific implementation, the script program will extract the corresponding test data from the MySQL database according to the preset rules; for example, in the GPU (Graphics Processing Unit, graphics processing unit) performance test report template, the measured floating point operation capability, memory bandwidth and core frequency and other data are automatically filled into the specified position of the template according to the mapping relationship, so as to generate the preliminary draft of the report containing basic data; further, if multiple test data comparison is involved, complex loop statements need to be written to realize batch processing and filling of data. The defects of this method are: when the test scene changes, for example, a new edge computing chip test item is added, the template structure needs to be redesigned to adjust the data mapping rules, and the script code also needs to be modified to adapt to the new test requirements, which makes the efficiency of the test report low.

[0038] The specific implementation process of the large model text generation method is as follows: first, the intelligent computing chip test data is formatted to form text content suitable for inputting the model; for example, the data of chip computing power 100TFLOPS, power consumption 250W, and time delay 5ms is organized according to the prompt word format of "please analyze the performance of the GPU test data in the AI inference scene: [specific data]"; then, the data and prompt words are sent to the large model through the API interface, and the model generates corresponding report text content based on the pre-trained knowledge and input content; further, in the actual application process, some enterprises will also inject intelligent computing chip industry terms and test standards and other data to build a dedicated field knowledge fine-tuning model, so as to achieve the purpose of optimizing the output results of the model. The defects of this method are as follows: in actual application, the output process of the large model lacks effective intervention means, and it is difficult to ensure the consistency of the generated content with the real test data, thereby making the accuracy of the test report obtained lower.

[0039] Based on the above-mentioned content, it can be known that the existing feasible evaluation report generation method has the following defects: on the one hand, the manual writing method is low in efficiency and is easily affected by human factors, has the risk of data transcription errors and analysis logic contradictions, and is difficult to adapt to the test needs of rapid iteration of chip technology; on the other hand, the template engine method has poor flexibility and expandability, and in the face of complex and variable test scenarios, a large amount of manpower needs to be invested to redesign the template and adjust the code; on the other hand, the large model text generation method has the illusion problem, which is easy to fabricate untested performance indicators or conclusions, and the generated content lacks structured constraints, logical confusion, and non-standard use of professional terms. Therefore, the above problems lead to the defects of long cycle, high error rate, and insufficient analysis depth in the generation of test reports of intelligent computing chips, which cannot provide timely and reliable basis for key technical decisions such as chip selection and performance optimization of enterprises.

[0040] Based on this, the chip evaluation report generation method provided in the example embodiment can be run on a terminal device (a mobile terminal device and / or a fixed terminal device), a server, a server cluster, or a cloud server, etc. Of course, those skilled in the art can also run the method of the present disclosure on other platforms according to the needs, which is not specially limited in the example embodiment. Specifically, referring to FIG. 1, the chip evaluation report generation method can include the following steps: Figure 1

[0041] Step S110. Determine the chip to be tested and the performance to be tested associated with the chip to be tested, and obtain the original chip performance data corresponding to the chip to be tested and the historical test report examples associated with the performance to be tested;

[0042] ​Step S120. Generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and historical test report examples;

[0043] Step S130. Generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report;

[0044] Step S140. Evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

[0045] In the chip evaluation report generation method described above, on the one hand, by determining the chip to be tested and the performance to be tested associated with the chip to be tested, and obtaining the original chip performance data corresponding to the chip to be tested and the historical test report examples associated with the performance to be tested; then generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and historical test report examples; then generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; finally, evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report, the automatic generation of the target chip evaluation report is realized, and the generation efficiency of the target chip evaluation report is improved; on the other hand, since the original chip evaluation report can be evaluated based on the preset report evaluation model to obtain a report evaluation result, and the target chip evaluation report is determined according to the report evaluation result and the original chip evaluation report, the accuracy of the obtained target chip evaluation report is improved.

[0046] In the following, the chip evaluation report generation method described in the example embodiments of the present disclosure will be further explained and described in conjunction with the accompanying drawings.

[0047] First, the terms involved in the example embodiments of the present disclosure are explained and described.

[0048] AI Chip: A chip designed specifically for intelligent computing, aiming to meet the high computing power needs of artificial intelligence algorithms such as deep learning and machine learning in model training and inference processes. In practical applications, AI chips are optimized for the computing characteristics of artificial intelligence, which can greatly reduce energy consumption and improve computing density while ensuring computing accuracy, thus greatly improving the speed and efficiency of data processing. Common AI chips may include but are not limited to GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), etc.

[0049] LLM (Large Language Model): An artificial intelligence model based on deep learning technology, with a large parameter size and strong language understanding and generation capabilities, capable of processing and generating natural language text. Through training on a large amount of text data to learn the patterns and rules of language, it can realize various natural language processing tasks such as dialogue, translation, and text generation.

[0050] AI (Artificial Intelligence): A new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Through giving computers the ability to learn, reason and make decisions, they can handle complex tasks like humans, including but not limited to natural language processing tasks, image recognition tasks, speech recognition tasks and robot control tasks.

[0051] Python: An object-oriented dynamic type programming language with simple and readable syntax and rich standard library, widely used in web development, data science, automation scripting and artificial intelligence fields, supporting functional programming and structured programming paradigms, with a large third-party library ecosystem such as Django and Flask web frameworks and NumPy and Pandas data processing libraries.

[0052] MySQL: An open source relational database management system that uses SQL (Structured Query Language) for data operations, with high performance, reliability and scalability, widely used in web application backend data storage, supporting transaction processing, foreign key constraints and index optimization functions. In practical applications, MySQL can be deployed on multiple operating systems and is often used in combination with programming languages such as PHP and Python.

[0053] Markdown: A lightweight markup language that uses simple symbols to format text, making it easy to convert into formats such as HTML (HyperText Markup Language) and PDF. It is commonly used in document writing, README files, and blog content creation. In practical applications, due to its easy-to-read and easy-to-write characteristics, it can improve the readability and editing efficiency of text without the need for complex formatting tools to display formats such as headings, lists, links, and code blocks.

[0054] Knowledge base: A structured collection of systematically stored, organized and managed knowledge, which can realize efficient retrieval, reasoning and application of knowledge through computer technology. In practical applications, knowledge base is different from ordinary database, it not only can store raw data, but also pay attention to semantic representation of knowledge (such as concept, relationship and rule, etc.), aiming to provide knowledge support for artificial intelligence system, decision support system or expert system.

[0055] Secondly, the technical implementation principle of the example embodiments of the present disclosure is explained and described. Specifically, the chip evaluation report generation method provided by the example embodiments of the present disclosure aims to solve the technical problems of serious manual dependence, uncontrollable large model output, and loose report logic in existing intelligent chip test report generation. At the same time, aiming at the pain points of low efficiency of manual writing, lack of flexibility of template generation, and easy illusion of end-to-end large model, by constructing a strong association mechanism between outline and data, combining dynamic scoring optimization process, the full-process automatic generation of test report is realized. This scheme significantly improves the report generation efficiency, greatly shortens the delivery cycle, and at the same time guarantees the content professionalism and logical coherence, ensures the report data to be real and reliable, and the analysis to be rigorous and in-depth. Further, through automation and intelligent means, the labor cost is reduced, and the adaptation ability of the system to different test scenarios is enhanced, providing efficient and accurate support for enterprise chip performance evaluation and technical decision-making.

[0056] Further, the chip evaluation report generation system involved in the example embodiments of the present disclosure is explained and described. Specifically, referring to Figure 2As shown, the chip evaluation report generation system can include a report outline generation agent 210, a knowledge base 220, a database 230, a data processing and report generation agent 240, and a report scoring optimization agent 250; wherein the report outline generation agent, the data processing and report generation agent, and the report scoring optimization agent are respectively in communication connection with the knowledge base and the database; the data processing and report generation agent is respectively in communication connection with the report outline generation agent and the report scoring optimization agent. In the actual application process, first, after the system receives the selected test items, the report outline generation agent retrieves the artificial test report examples in the knowledge base, drives the large model to automatically generate the structured outline in Markdown format according to the report specification and prompt words; wherein the outline takes the test type as the chapter title, and generates a unique identifier for each chapter as a content placeholder, which is accurately associated with the test table data and log files in the database, thereby building a logically clear report framework; secondly, the data processing and report generation agent converts the data in the database into Markdown format through a Python program, and at the same time, combines the prompt words in the knowledge base to guide the large model to analyze the data in depth; at the same time, the large model not only needs to read the log to generate detailed test environment description, but also needs to output drawing code to call the visualization library to generate statistical charts; then, according to the placeholders in the outline, the environment description, data table, statistical chart and analysis conclusion are integrated to generate a complete Markdown report; finally, the report scoring optimization agent generates scoring verification prompt words according to the preset report scoring standard, and uses the large model to quantitatively score the data consistency and analysis rationality of the report text content. In the actual application process, if the score does not reach the set threshold, the system will trigger the iterative optimization process to make the large model modify the report accordingly and re-score, and this process will be repeated until the score meets the standard, so as to ensure the quality of the report output.

[0057] In the following, the large language model involved in the example embodiments of the present disclosure will be explained and described. Specifically, referring to Figure 3 As shown, the large language model includes an embedding mapping layer 310, an encoding layer 320, and a hybrid expert model 330; wherein the embedding mapping layer described herein can include an Embedding embedding mapping layer and / or a Bert embedding mapping layer; at the same time, the encoding layer described herein can be implemented based on multiple bidirectional Transformers; further, the hybrid expert model described herein can include a gating network model and a plurality of hybrid expert network models, and the specific structure example diagram can be referred to Figure 4 As shown; at the same time, the functions required to be performed by each model layer in the specific application process will be described one by one in the following, and further description will not be made here.

[0058] Hereinafter, the generation method of the chip evaluation report shown in Figures 2-4 The generation method of the chip evaluation report shown in Figure 1 will be further explained and described. Specifically:

[0059] In step S110, the chip to be tested and the performance to be tested associated with the chip to be tested are determined, and the original chip performance data corresponding to the chip to be tested and the historical test report example associated with the performance to be tested are obtained.

[0060] In the example embodiment, first, the chip to be tested and the performance to be tested associated with the chip to be tested are determined; specifically, this can be achieved by the following manner: in response to the selection operation of the tester acting on the chip and the corresponding chip performance displayed on the display interface (i.e. the Web interface or the client tool), the chip to be tested and the performance to be tested associated with the chip to be tested (i.e. the test item of the intelligent algorithm chip) are determined; wherein the chip to be tested recorded herein can include one or more, which can be distinguished according to the manufacturer + model of the chip, which can be referred to as shown in Figure 5 ; the performance to be tested recorded herein can also include one or more, which can include but not limited to GPU inference performance, AI large model training performance, large model inference performance, collective communication performance, memory bandwidth performance, small model inference performance, etc., which can be determined according to the actual situation of the chip in the actual application, and the example does not make special restrictions on this; at the same time, the selected performance to be tested can be referred to as shown in Figure 6 .

[0061] Secondly, when the to-be-tested chip and the to-be-tested performance associated with the to-be-tested chip are selected, a submit control on the display interface can be clicked to generate a test request; then, based on the original chip performance data corresponding to the to-be-tested chip and the historical test report examples associated with the to-be-tested performance included in the test request, the original chip performance data (for example, which can include test result table data and test log information) can be obtained from the database; wherein the database recorded herein can include MySQL relational database storage data (simple data), or MongoDB and the like document type database (adopted when facing a large amount of unstructured data such as test log text) or graph database (adopted when needing to process complex data association such as multi-chip test comparison), in the actual application process, can be selected according to actual needs, and this example does not make special restrictions; further, the historical test report examples associated with the to-be-tested performance can be obtained from the knowledge base associated with the to-be-tested performance; wherein the knowledge base recorded herein is generated based on the performance of the historical tested chip and the test report examples associated with the performance; that is, the test report examples of the historical tested chip can be manually generated and associatedly stored, so as to obtain the required knowledge base.

[0062] In step S120, a test report outline corresponding to the to-be-tested chip is generated based on a preset report outline generation model according to the to-be-tested performance and the historical test report examples.

[0063] Specifically, the specific generation process of the test report outline can be realized by the following manner: determining the performance evaluation index and the evaluation index meaning of the to-be-tested performance based on the historical test report examples, and generating first model prompt information associated with the to-be-tested performance according to the performance evaluation index and the evaluation index meaning; inputting the to-be-tested performance and the first model prompt information associated with the to-be-tested performance into the preset report outline generation model to obtain an original text outline; determining the uniqueness placeholder of the original text outline according to the primary key field of the to-be-tested performance, and establishing the mapping relationship between the uniqueness placeholder and the original text outline, so as to generate the test report outline corresponding to the to-be-tested chip.

[0064] In an exemplary embodiment, the determination of the uniqueness placeholder of the original text outline according to the primary key field of the to-be-tested performance can be realized by the following manner: determining the primary key field of the to-be-tested performance according to the historical test report examples, and determining the uniqueness placeholder possessed by each chapter of the original text outline according to the primary key field; wherein each to-be-tested performance corresponds to a chapter, and the uniqueness placeholder has global uniqueness and is used for accurately positioning the position of the performance test result of the to-be-tested performance in the original text outline.

[0065] Further explanation and description will be given below to the specific generation process of the test report outline. Specifically, in the actual application process, first, the system knowledge base is called to obtain the artificial test report examples stored therein, which can cover the report specification format, writing style and structural characteristics of different types of test items; second, based on the obtained test item information and report examples, and combined with the evaluation indicators of each test item and their definitions, the first model prompt information is assembled to form the prompt words of the large model to generate the outline; wherein the first model prompt information can include format requirements, chapter logic and key content guidance, etc. information, so as to accurately guide the large model (i.e. the preset report outline generation model) to output the outline (i.e. the original text outline) conforming to the specification; then, the assembled prompt words and the performance to be tested are input into the large model, so that the large model outputs the outline text in Markdown format or HTML format (i.e. the original text outline) through natural language processing technology; wherein the original text outline takes the performance to be tested of the chip to be tested as the chapter title, which can be referred to as shown in Figure 7 Further, the test result table data and test log database information associated with the test item information (i.e. the chip to be tested) are obtained in the MySQL database to determine the data storage location and the primary key field; further, a unique placeholder is generated for each chapter of the outline according to the primary key field of the test data table, which has global uniqueness and can be used to accurately locate and replace the corresponding chapter content (i.e. the performance test result); at the same time, the generated unique placeholder is associated with each chapter of the report outline to establish a stable data mapping relationship, ensuring the accuracy of subsequent data retrieval and content integration; finally, the structured outline in Markdown format or HTML format containing the unique placeholder and clear structure (i.e. the test report outline) is output to provide framework support for subsequent report content generation.

[0066] In an example embodiment, the performance evaluation indicators described above can include but are not limited to basic performance evaluation indicators, computing performance evaluation indicators, power consumption and efficiency evaluation indicators, physical property evaluation indicators, specific scenario evaluation indicators, cost and reliability evaluation indicators, etc.; wherein the basic performance evaluation indicators can include but are not limited to frequency, throughput and latency; the computing performance evaluation indicators can include but are not limited to computing power; the power consumption and efficiency evaluation indicators can include but are not limited to power consumption and energy efficiency ratio; the physical property evaluation indicators can include but are not limited to storage capacity and area; the specific scenario evaluation indicators can include but are not limited to gain, bandwidth and noise coefficient; the cost and reliability evaluation indicators can include but are not limited to price and reliability.

[0067] In an example embodiment, the evaluation index of the performance evaluation index described above means: Frequency, which can be in units of megahertz (MHz) or gigahertz (GHz), reflects the clock speed; Throughput, which is used to represent the amount of data processed per unit time, and is particularly important for multi-core chips; Latency, which is the time required to perform a task, which directly affects the response speed; Computing power can include MIPS / MOPS and MFLOPS; MIPS / MOPS represents the number of instructions or operations per second, which can be used to measure processing efficiency; MFLOPS represents the number of floating-point operations per second, which can be applied to complex computing scenarios; Power consumption, which is the power consumption during operation, in units of watts (W); Power efficiency, which is the amount of tasks completed under a unit of power consumption, which can be used to measure energy-saving performance; Storage capacity, the number of binary bits that the chip can store, which directly affects the data storage capacity; Die area, the physical size of the chip, which affects production cost and integration density; Gain, which is the ratio of input and output signals in a radio frequency chip, which can determine the signal amplification capability; Bandwidth, which is the frequency range supported by the radio frequency chip, which affects communication efficiency; Noise figure, which can be used to measure the signal-to-noise ratio of the received signal, which affects the communication quality; Price, which includes the cost per unit of storage and the cost of peripheral circuits; Reliability, which is used to measure the ability of the chip to work continuously under specified conditions.

[0068] In step S130, the preset content generation large model generates a performance test result according to the original chip performance data, and maps the performance test result to the test report outline to obtain an original chip evaluation report.

[0069] In the example embodiment, first, the preset content generation large model is used to generate performance test results based on original chip performance data; specifically, the following methods can be used: structured analysis and format reconstruction of test result table data to obtain structured test result data, and generation of second model prompt information associated with the to-be-tested performance based on historical test report examples; inputting the structured test result data, test log data and second model prompt information into the preset content generation large model to generate performance test results based on original chip performance data. That is, in actual application, a special data interface written in Python can be used to establish a secure connection with the database to accurately read the stored test result table data and test log data, ensuring data integrity and accuracy; second, the designed data conversion rule code is used to perform structured analysis and format reconstruction on the read test result table data to efficiently convert SQL format or MongoDB format or graph data format data into Markdown format or HTML format structured test result data, which adapts to the data input requirements of the preset content generation large model (i.e., large language model); then, the system knowledge base is called and combined with the characteristics of different test items (i.e., to-be-tested chips) to extract relevant report examples and evaluation indicators from the knowledge base through semantic retrieval and keyword matching algorithms, outputting large model prompt words (i.e., second model prompt information) for data summary and analysis, environment explanation generation, and chart code generation sub-tasks, providing clear guidance for large model data processing; then, the converted Markdown format data, test log data and second model prompt information are input into the preset content generation large model to make the preset content generation large model process the data through natural language processing, analyze the test log to generate detailed test environment explanation; perform deep analysis on the Markdown format table data (i.e., structured test result data) to output professional summary conclusions; generate data visualization code according to the data table characteristics in the structured test result data, call the library to draw various statistical charts, and output the corresponding performance test results.

[0070] In a possible example embodiment, the test result table data described above can include, but is not limited to, basic performance, computing performance, power consumption and efficiency, physical characteristics, specific scenarios, cost and reliability, etc. The basic performance indicators can include, but are not limited to, frequency, throughput and latency. The computing performance indicators can include, but are not limited to, computing power. The power consumption and efficiency indicators can include, but are not limited to, power consumption and energy efficiency ratio. The physical characteristics indicators can include, but are not limited to, storage capacity and area. The specific scenario indicators can include, but are not limited to, gain, bandwidth and noise coefficient. The cost and reliability indicators can include, but are not limited to, price and reliability, etc. Further, the test log data described above can include, but is not limited to, environment configuration information, which can include software configuration information and hardware configuration information, etc. The hardware configuration information can include, but is not limited to, the model of the intelligent algorithm chip, the capacity of the display memory and the storage architecture, etc. The software configuration information can include, but is not limited to, software environment parameters such as operating system version, driver version, inference framework version number and development environment, etc.

[0071] In an example embodiment, inputting the structured test result data, test log data and second model prompt information into the preset content generation large model to generate the performance test result according to the original chip performance data can be achieved by the following manner: generating the to-be-predicted basic performance information according to the structured test result data and test log data, and generating the to-be-predicted context information according to the second model prompt information; performing embedding mapping processing on the to-be-predicted basic performance information based on the embedding mapping layer to obtain first performance features, and performing embedding mapping processing on the to-be-predicted context information based on the embedding mapping layer to obtain context sequences; performing encoding processing on the first performance features and context sequences based on the encoding layer to obtain overall context representations, and performing performance test on the context sequences and overall context representations based on the hybrid expert model to obtain the performance test result. Specifically, in actual application, the embedding mapping processing can be performed on the to-be-predicted basic performance information based on the Embedding embedding mapping layer to obtain the first performance features, and the embedding mapping processing can be performed on the to-be-predicted context information based on the Bert embedding mapping layer to obtain the context sequences. On this basis, the first performance features and context sequences are sent into the multi-layer bidirectional Transformer encoding layer to obtain the overall context representations.

[0072] In an example embodiment, the performance test result based on the hybrid expert model for the context sequence and the context overall representation can be achieved by the following way: determining the first model weight of each expert neural network model in performing the performance test task in the test environment description dimension, the second model weight in performing the performance test task in the data summary analysis dimension and the third model weight in performing the performance test task in the statistical chart drawing dimension based on the gating network model according to the context sequence; determining the first target network model required for performing the performance test task in the test environment description dimension, the second target network model required for performing the performance test task in the data summary analysis dimension and the third target network model required for performing the performance test task in the statistical chart drawing dimension from each expert neural network model according to the first model weight, the second model weight and the third model weight; inputting the context sequence and the context overall representation into the first target network model, the second target network model and the third target network model respectively to obtain the first test result in the test environment description dimension, the second test result in the data summary analysis dimension and the third test result in the statistical chart drawing dimension, and generating the performance test result according to the first test result, the second test result and the third test result. That is, in the process of generating the performance test result, it is required to achieve from three different dimensions, and then the prediction results obtained from the three different dimensions are spliced to obtain the final performance test result.

[0073] In an example embodiment, mapping the performance test result to the test report outline to obtain the original chip evaluation report can be achieved by determining the chapter to which the performance to be tested belongs in the test report outline, and determining a first unique placeholder corresponding to the test environment description, a second unique placeholder corresponding to the data summary analysis result, and a third unique placeholder corresponding to the statistical chart drawing result in the chapter; establishing a first mapping relationship between the first unique placeholder and a first test result in the performance test result in the test environment description dimension, a second mapping relationship between the second unique placeholder and a second test result in the data summary analysis dimension, and a third mapping relationship between the third unique placeholder and a third test result in the statistical chart drawing dimension; based on the first mapping relationship, the second mapping relationship, and the third mapping relationship, the first test result, the second test result, and the third test result in the performance test result are set in the chapter to which the performance to be tested belongs, to obtain the original chip evaluation report. That is, in actual application, the generated test environment description (first test result), data summary analysis (second test result), and visual statistical chart (third test result) can be replaced into the corresponding chapter position in the outline by using the unique placeholders of each chapter in the report outline, to ensure that the filled content and the outline structure correspond one by one; further, after completing the content replacement of all chapters, a complete Markdown format or HTML format evaluation report (i.e., the original chip evaluation report, which can be referred to in detail in FIG. 8) is generated, realizing the automatic conversion from raw data to a professional report. Figure 8

[0074] In step S140, the original chip evaluation report is evaluated based on a preset report evaluation model to obtain a report evaluation result, and a target chip evaluation report is determined according to the report evaluation result and the original chip evaluation report.

[0075] In the example embodiment, first, the original chip evaluation report is evaluated to obtain a report evaluation result; specifically, the following method can be used: generating third model prompt information corresponding to the original chip evaluation report; wherein the third model prompt information includes model prompt information in the data consistency dimension and model prompt information in the data professionalism dimension; inputting the original chip evaluation report, the structured test result data, and the third model prompt information into a preset report evaluation model, so that the report evaluation model evaluates the original chip evaluation report from the data consistency dimension and the data professionalism dimension to obtain a report evaluation result; wherein the report evaluation result is obtained by weighted summation of a first score result in the data consistency dimension and a second score result in the data professionalism dimension.

[0076] ​In the following, the specific determination process of the report evaluation result will be further explained and described. Specifically, in actual application, the generated original chip evaluation report in Markdown format or HTML format can be input into the scoring report evaluation model to score the data consistency and professionalism; in the actual scoring process, the system will automatically extract the corresponding test table data in Markdown format (i.e. structured test result data) in the report evaluation model; at the same time, for the analysis professionalism, the system will call the knowledge base report examples and test evaluation index definition; at the same time, it is also necessary to construct the corresponding large model prompt word (i.e. the third model prompt information); among them, the third model prompt information can include data consistency check and data professionalism check, the data consistency check is mainly used to guide the large model to judge whether the numerical value involved in the analysis content is consistent with the actual table data, and the data professionalism check is used to guide the large model to judge whether the generated report analysis conclusion is logically rigorous and description standard according to the knowledge base content; on this basis, the report content and the constructed prompt word are input into the large model, the large model compares and checks the test numerical value in the report with the test table data based on the preset data consistency scoring standard, and outputs the data consistency score; at the same time, according to the analysis professionalism evaluation standard, the depth, accuracy and standardization of the analysis content are evaluated, and the analysis professionalism score is output; finally, the system calculates the two scores to obtain the corresponding report evaluation result.

[0077] It needs to be supplemented here that in actual application, a multi-source scoring fusion mode can also be used to output the report evaluation result; for example, artificial expert scoring and historical report quality comparison scoring dimensions can be included, and a weighted algorithm is used to obtain a comprehensive score, so that the result is more objective; of course, a reinforcement learning algorithm can also be introduced, taking the scoring result as feedback, dynamically optimizing the generation strategy of the large language model, reducing human intervention, and continuously improving the report quality.

[0078] In an example embodiment, determining the target chip evaluation report according to the report evaluation result and the original chip evaluation report can be implemented in the following manner: determining whether the report evaluation score of the original chip evaluation report is greater than or equal to a preset score threshold according to the report evaluation result, and taking the original chip evaluation report as the target chip evaluation report when it is determined that the report evaluation score is greater than or equal to the preset score threshold; when it is determined that the report evaluation score is less than the preset score threshold, optimizing and adjusting the original chip evaluation report until the report evaluation score is greater than or equal to the preset score threshold, and taking the original chip evaluation report after the optimization and adjustment as the target chip evaluation report. Specifically, in the actual application process, the report evaluation score in the report evaluation result and the preset score threshold (i.e., the preset score threshold) can be compared and judged to determine whether the report meets the standard; further, if the score meets the standard, the system automatically outputs the final qualified report (i.e., the target chip evaluation report); if it does not meet the standard, the large model locates the problems in the report (such as consistency problems or professional problems) according to the score feedback, adjusts the analysis content, supplements the data details or corrects the conclusion, regenerates the optimized report, and enters the scoring process again to form a closed-loop iterative optimization mechanism.

[0079] At this point, the chip evaluation report generation method described in the example embodiments of the present disclosure has been fully implemented. In the following, the specific implementation process of the chip evaluation report generation method described in the example embodiments of the present disclosure will be further explained and described in combination with specific examples.

[0080] (1) In the system startup phase, the tester selects the completed test items in the test task list through the visual interface of the user interaction layer, such as "testing the large model inference performance of a certain brand model A100 chip, testing the collective communication performance, testing the memory bandwidth, testing the visual small model inference, etc."; after the system receives the demand, the information is synchronously transmitted to the report outline generation module and the knowledge base of the data storage layer, and the retrieval function related to each performance test in the knowledge base is activated.

[0081] (2) In the core processing phase, the report outline generation agent relies on the large language model and the knowledge base content to generate a Markdown structured outline conforming to the industry specifications through a preset prompt word; wherein the outline takes the test type of the test item as the chapter title, such as large model inference performance, collective communication performance, memory bandwidth performance, small model inference performance, etc.; at the same time, each chapter covers the test environment description, test table data, visual statistical chart, data summary and analysis, etc. core part, and gives each chapter content a globally unique placeholder, establishing a precise mapping relationship with the corresponding test data and log files in the MySQL database.

[0082] (3) The data processing and report generation agent calls the Python data interface to retrieve the original test data tables of the A100 chip from the MySQL database and converts them to Markdown text. Meanwhile, it outputs customized prompt words for each sub-content generation task based on knowledge base retrieval, drives the large language model to analyze the data, generates performance analysis conclusions, analyzes test logs to form environment descriptions, and outputs statistical chart visualization code to draw bar or line charts. Subsequently, in the report synthesis link, the data processing and report generation module accurately embeds the environment description, performance analysis text, and visualized charts into the corresponding chapters based on the outline placeholder replacement rules, forming a complete test report draft.

[0083] (4) Finally, the initial report is input into the report scoring optimization module, which calls database data and professional knowledge base to build scoring verification prompt words, and uses the large language model to quantitatively evaluate the report from the aspects of data consistency and analysis professionalism according to the preset scoring rules. If the initial score does not meet the standard (e.g., full score 10, average score less than 5), the system triggers the iterative optimization mechanism, and the large language model accurately locates the data expression omissions and conclusion support deficiencies in the report based on the scoring feedback, optimizes the content accordingly, and re-submits it to the scoring module. After multiple rounds of optimization, the report quality meets the standard, and a professional and standardized chip performance test report is finally output. The whole process realizes the full-link automation from test requirements to report delivery, significantly improving the efficiency and reliability of test report generation.

[0084] As can be seen from the above, in actual application, users only need to select test items through the interactive interface, and the system can automatically complete the whole process operation from outline generation to report output, significantly reducing the requirement for users' professional knowledge. At the same time, by constructing a unique placeholder and data mapping mechanism through the report outline generation module, the randomness of the large model generated content is avoided, ensuring that the test report content accurately matches the actual data, effectively reducing structural disorder, data fabrication, and other problems compared to traditional large model generation methods, significantly improving the accuracy and reliability of the report content. Further, the data processing and report generation agent converts SQL data to Markdown format and combines knowledge base and prompt words to guide the large model analysis, automatically completing data interpretation, environment description, and chart generation, replacing the tedious process of manual sorting and analysis, improving report generation efficiency, significantly shortening the delivery cycle, and reducing enterprise labor costs. Further, the closed-loop iterative mechanism of the report scoring optimization agent performs multiple rounds of evaluation and correction on the report based on the preset scoring standard, ensuring that the output report meets high standards in terms of data consistency and analysis professionalism, providing a reliable basis for enterprise chip selection, performance optimization, and other technical decisions.

[0085] Furthermore, the chip evaluation report generation method disclosed in the example embodiments of the present disclosure has the following advantages: on the one hand, the evaluation report step-by-step construction process in Markdown format realizes the full-process automation from test item selection, outline generation, data processing, content integration to report optimization; based on this, the low efficiency problem caused by multi-link manual intervention in the manual writing mode can be overcome, and the code adjustment cost in the complex scene of the template engine mode can be avoided; and the report generation period can be greatly shortened, the human input can be reduced, the standardization of the report structure and the integrity of the content can be ensured, timely and reliable report support can be provided for chip technology decision-making to adapt to the batch generation demand in the high-frequency test scene; on the other hand, the report outline is generated in advance by using a large model and a knowledge base, and the association mapping mechanism of the Markdown outline chapter and the database test data is established by using a unique identifier, so that the structured framework generated by the large model is dynamically bound with the underlying test data; based on this, the limitation that the template engine in the prior art needs to manually design a fixed template can be solved, the report structure does not need to be fixed in advance, the chapter content can be automatically matched according to the test data characteristics, and the problem of insufficient template expandability in the complex test scene can be solved; on the other hand, the SQL data is automatically converted into Markdown format by using a Python program, and the large model is guided to generate the environment description, analysis conclusion and visualization code based on the measured data by using the knowledge base prompt word, so that the system link from data input to content generation is constructed; based on this, the database data can be mapped to the input form that can be easily understood by the language large model under the premise of combining the data format conversion rule, so that the accuracy of data analysis is enhanced; at the same time, compared with the illusion problem that may occur when the large model generates text, the technology constrains the output boundary of the large model by using the measured data and the knowledge base example, avoids the disconnection between the generated content and the real test result, and ensures the scientificity of the generated content; finally, a report scoring optimization closed-loop system based on the large model is constructed, the report content is automatically scored by using the preset data consistency checking rule and the analysis professional evaluation index, and iteration optimization is triggered when the report content does not meet the standard; based on this, the scoring and automatic correction mechanism can eliminate the subjective bias of manual review, improve the data consistency and analysis logic rigor of the report content, realize the standardized control of the report quality, and reduce the manual review input.

[0086] The following is a device embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the device embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0087] The example embodiments of the present disclosure also provide a chip evaluation report generation device. Specifically, refer to Figure 9As shown, the chip evaluation report generation apparatus can include a to-be-tested chip generation module 910, a test report outline generation module 920, a performance test result generation module 930, and a chip evaluation report generation module 940. Among them:

[0088] The to-be-tested chip generation module 910 can be used to determine a to-be-tested chip and a to-be-tested performance associated with the to-be-tested chip, and obtain original chip performance data corresponding to the to-be-tested chip and a historical test report example associated with the to-be-tested performance.

[0089] The test report outline generation module 920 can be used to generate a test report outline corresponding to the to-be-tested chip based on a preset report outline generation model according to the to-be-tested performance and the historical test report example.

[0090] The performance test result generation module 930 can be used to generate a performance test result based on a preset content generation model according to the original chip performance data, and map the performance test result to the test report outline to obtain an original chip evaluation report.

[0091] The chip evaluation report generation module 940 can be used to evaluate the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determine a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

[0092] In an exemplary embodiment of the present disclosure, generating a test report outline corresponding to a to-be-tested chip based on a preset report outline generation model according to a to-be-tested performance and a historical test report example includes: determining a performance evaluation index and an evaluation index meaning of the to-be-tested performance based on the historical test report example, and generating first model prompt information associated with the to-be-tested performance according to the performance evaluation index and the evaluation index meaning; inputting the to-be-tested performance and the first model prompt information associated with the to-be-tested performance into the preset report outline generation model to obtain an original text outline; determining a uniqueness placeholder of the original text outline according to a primary key field of the to-be-tested performance, and establishing a mapping relationship between the uniqueness placeholder and the original text outline to generate a test report outline corresponding to the to-be-tested chip.

[0093] In an example embodiment of the present disclosure, the unique placeholders of the original text outline are determined according to the primary key field of the performance to be tested, including: determining the primary key field of the performance to be tested according to the historical test report example, and determining the unique placeholder possessed by each chapter of the original text outline according to the primary key field; wherein each performance to be tested corresponds to a chapter, and the unique placeholder has global uniqueness and is used to accurately locate the position of the performance test result of the performance to be tested in the original text outline.

[0094] In an example embodiment of the present disclosure, the original chip performance data includes test result table data and test log data; wherein the performance test result is generated according to the original chip performance data based on the preset content generation large model, including: performing structural analysis and format reconstruction on the test result table data to obtain structured test result data, and generating second model prompt information associated with the performance to be tested according to the historical test report example; inputting the structured test result data, test log data and second model prompt information into the preset content generation large model to generate the performance test result according to the original chip performance data.

[0095] In an example embodiment of the present disclosure, the preset content generation large model includes an embedding mapping layer, an encoding layer and a hybrid expert model; wherein the performance test result is generated according to the original chip performance data based on the preset content generation large model, including: generating the to-be-predicted basic performance information according to the structured test result data and test log data, and generating the to-be-predicted context information according to the second model prompt information; performing embedding mapping processing on the to-be-predicted basic performance information based on the embedding mapping layer to obtain first performance features, and performing embedding mapping processing on the to-be-predicted context information based on the embedding mapping layer to obtain context sequences; performing encoding processing on the first performance features and context sequences based on the encoding layer to obtain overall context representations, and performing performance test on the context sequences and overall context representations based on the hybrid expert model to obtain the performance test result.

[0096] In an example embodiment of the present disclosure, the hybrid expert model comprises a gating network model and a plurality of expert neural network models; wherein the performance test result is obtained based on performance testing of the context sequence and the context overall representation by the hybrid expert model, comprising: determining, based on the gating network model, the first model weight of each of the expert neural network models in performing a performance test task in the test environment description dimension, the second model weight in performing a performance test task in the data summary analysis dimension, and the third model weight in performing a performance test task in the statistical chart drawing dimension according to the context sequence; determining, according to the first model weight, the second model weight, and the third model weight, the first target network model required for performing a performance test task in the test environment description dimension, the second target network model required for performing a performance test task in the data summary analysis dimension, and the third target network model required for performing a performance test task in the statistical chart drawing dimension from each of the expert neural network models; inputting the context sequence and the context overall representation into the first target network model, the second target network model, and the third target network model respectively to obtain the first test result in the test environment description dimension, the second test result in the data summary analysis dimension, and the third test result in the statistical chart drawing dimension, and generating the performance test result according to the first test result, the second test result, and the third test result.

[0097] In an example embodiment of the present disclosure, mapping the performance test result to the test report outline to obtain the original chip evaluation report comprises: determining the chapter to which the performance to be tested belongs in the test report outline, and determining the first unique placeholder corresponding to the test environment description, the second unique placeholder corresponding to the data summary analysis result, and the third unique placeholder corresponding to the statistical chart drawing result in the chapter; establishing the first mapping relationship between the first unique placeholder and the first test result in the test environment description dimension in the performance test result, the second mapping relationship between the second unique placeholder and the second test result in the data summary analysis dimension, and the third mapping relationship between the third unique placeholder and the third test result in the statistical chart drawing dimension; based on the first mapping relationship, the second mapping relationship, and the third mapping relationship, setting the first test result, the second test result, and the third test result in the performance test result in the chapter to which the performance to be tested belongs to obtain the original chip evaluation report.

[0098] In an example embodiment of the present disclosure, the original chip evaluation report is evaluated based on a preset report evaluation model to obtain a report evaluation result, including: generating third model prompt information corresponding to the original chip evaluation report; wherein the third model prompt information includes model prompt information of a data consistency dimension and model prompt information of a data professionalism dimension; inputting the original chip evaluation report, the structured test result data and the third model prompt information into the preset report evaluation model, so that the report evaluation model evaluates the original chip evaluation report from the data consistency dimension and the data professionalism dimension to obtain a report evaluation result; wherein the report evaluation result is obtained by weighted summation of a first score result on the data consistency dimension and a second score result on the data professionalism dimension.

[0099] In an example embodiment of the present disclosure, the target chip evaluation report is determined according to the report evaluation result and the original chip evaluation report, including: determining whether the report evaluation score of the original chip evaluation report is greater than or equal to a preset score threshold according to the report evaluation result, and taking the original chip evaluation report as the target chip evaluation report when it is determined that the report evaluation score is greater than or equal to the preset score threshold; when it is determined that the report evaluation score is less than the preset score threshold, optimizing and adjusting the original chip evaluation report until the report evaluation score is greater than or equal to the preset score threshold, and taking the original chip evaluation report after optimization and adjustment as the target chip evaluation report.

[0100] The specific details of each module in the chip evaluation report generation device described above have been described in detail in the corresponding chip evaluation report generation method, so this will not be repeated here.

[0101] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units.

[0102] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this is not required or implied that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0103] In the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0104] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" herein.

[0105] The electronic device 1000 according to this embodiment of the present disclosure will be described below with reference to Figure 10 Figure 10 The display electronic device 1000 is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0106] As shown in Figure 10 , the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 can include, but are not limited to, the at least one processing unit 1010 described above, the at least one storage unit 1020 described above, a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010), and a display unit 1040.

[0107] The storage unit stores program code that can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification. For example, the processing unit 1010 can perform the steps shown in Figure 1 , such as step S110 of determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; step S120 of generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; step S130 of generating a performance test result according to the original chip performance data based on a preset content generation model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; and step S130 of evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

[0108] ​The storage unit 1020 can include a readable medium in the form of volatile storage such as random access memory (RAM) 10201 and / or cache memory 10202, and also can include a non-volatile storage such as read only memory (ROM) 10203.

[0109] The storage unit 1020 also can include a program / utility 10204 having a set (at least one) of program modules 10205, including an operating system, one or more application programs, other program modules, and program data, each of which can give the electronic device 1000 the ability, singly or in combination, to operate in a networked environment using any of the various applications.

[0110] The bus 1030 can represent one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics acceleration port, a processor or local bus using any of a variety of bus architectures.

[0111] The electronic device 1000 also can communicate with one or more external devices 1100, such as a keyboard or pointing device, a Bluetooth device, etc.; and a display or other type of output device (e.g., a printer) can be present, as are other possible devices (e.g., a television). In general, an output device can use almost any type of technology to convey information, such as storage or display. In one embodiment, the electronic device 1000 can include or communicate with one or more devices that enable a user to interact with the electronic device 1000; one possible interaction can be handling telephone conversations, such as voice communications. Other possible interactions can include

[0112] Those skilled in the art will readily recognize that the example embodiments described herein can be implemented using software and / or hardware in combination with software. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0113] In exemplary embodiments of the present disclosure, a computer readable storage medium having stored thereon a program product capable of implementing the above-described methods of the present specification is also provided. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product including a program code that, when run on a terminal device, causes the terminal device to perform the steps described in the above "Exemplary Methods" section according to various exemplary embodiments of the present disclosure.

[0114] A program product for implementing the above-described methods according to embodiments of the present disclosure can take the form of a portable compact disc read-only memory (CD-ROM) and include a program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in the present document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0115] The program product can take any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0116] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the readable program code is embodied. Such propagated data signal can take multiple forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable signal medium can also be any readable medium that is not a readable storage medium and that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0117] The program code contained on the readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, and the like, or any suitable combination thereof.

[0118] The program code may be executed by one or more programmable processing devices, which can include processors, microprocessor, microcomputer or microcontrollers, as well as other forms of computing or processing devices. The processing devices can be central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other similar components or devices. The processing devices can be single-core or multi-core, and the programs may

[0119] Further, the above-described diagrams are merely schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended for limiting purposes. It is readily understood that the processes illustrated in the above-described diagrams do not indicate or limit the time sequence of the processes. In addition, it is readily understood that the processes can be executed synchronously or asynchronously, for example, in a plurality of modules.

[0120] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the description herein, the drawings, and the annexed claims. The present application is intended to cover any variations, uses, or adaptations of the present disclosure following, in general, the principles of the present disclosure and including such further modifications as come within the true scope of the present disclosure. The application is likely to have other embodiments and the present disclosure is intended to include all adaptations and variations of the specific embodiments discussed herein that are within the scope of this present disclosure. The application is not limited to the specific embodiments described herein, but only by the claims, which follow.

Claims

1. A method for generating a chip evaluation report, characterized in that The method comprises the following steps: determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

2. The method of claim 1, wherein, The method comprises the following steps: determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; 3. The method of claim 2, wherein, evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report. The method comprises the following steps: determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; 4. The method of claim 1, wherein, generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

5. The method of claim 4, wherein, The method comprises the following steps: determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report. The method comprises the following steps: determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report. The method comprises the following steps: determining a chip to be tested and a performance to be tested associated with the chip to be tested, and obtaining original chip performance data corresponding to the chip to be tested and a historical test report example associated with the performance to be tested; generating a test report outline corresponding to the chip to be tested based on a preset report outline generation model according to the performance to be tested and the historical test report example; generating a performance test result according to the original chip performance data based on a preset content generation large model, and mapping the performance test result to the test report outline to obtain an original chip evaluation report; evaluating the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and determining a target chip evaluation report according to the report evaluation result and the original chip evaluation report. generate the to-be-predicted basic performance information according to the structured test result data and test log data, and generate the to-be-predicted context information according to the second model prompt information; perform embedding mapping processing on the to-be-predicted basic performance information based on the embedding mapping layer to obtain first performance features, and perform embedding mapping processing on the to-be-predicted context information based on the embedding mapping layer to obtain context sequences; perform encoding processing on the first performance features and the context sequences based on the encoding layer to obtain a context overall representation, and perform performance testing on the context sequences and the context overall representation based on the hybrid expert model to obtain the performance test result.

6. The method of claim 5, wherein, The hybrid expert model comprises a gating network model and a plurality of expert neural network models. The performance testing on the context sequences and the context overall representation based on the hybrid expert model to obtain the performance test result comprises: determining, based on the gating network model, first model weights of each of the expert neural network models in performing a performance testing task in a test environment description dimension, second model weights of each of the expert neural network models in performing a performance testing task in a data summary analysis dimension, and third model weights of each of the expert neural network models in performing a performance testing task in a statistical chart drawing dimension according to the context sequences; determining, according to the first model weights, the second model weights, and the third model weights, a first target network model required for performing the performance testing task in the test environment description dimension, a second target network model required for performing the performance testing task in the data summary analysis dimension, and a third target network model required for performing the performance testing task in the statistical chart drawing dimension from each of the expert neural network models; inputting the context sequences and the context overall representation into the first target network model, the second target network model, and the third target network model respectively to obtain a first test result in the test environment description dimension, a second test result in the data summary analysis dimension, and a third test result in the statistical chart drawing dimension, and generating the performance test result according to the first test result, the second test result, and the third test result.

7. The method of claim 1, wherein, mapping the performance test result to a test report outline to obtain an original chip evaluation report, comprising: determining a chapter to which the to-be-tested performance belongs in the test report outline, and determining a first unique placeholder corresponding to test environment description, a second unique placeholder corresponding to data summary analysis result, and a third unique placeholder corresponding to statistical chart drawing result in the chapter; establishing a first mapping relationship between the first unique placeholder and the first test result in the test environment description dimension in the performance test result, a second mapping relationship between the second unique placeholder and the second test result in the data summary analysis dimension, and a third mapping relationship between the third unique placeholder and the third test result in the statistical chart drawing dimension; and establishing a first mapping relationship between the first unique placeholder and the first test result in the test environment description dimension in the performance test result, a second mapping relationship between the second unique placeholder and the second test result in the data summary analysis dimension, and a third mapping relationship between the third unique placeholder and the third test result in the statistical chart drawing dimension. Based on the first mapping relationship, the second mapping relationship and the third mapping relationship, the first test result, the second test result and the third test result in the performance test result are set in the chapter to which the to-be-tested performance belongs, and the original chip evaluation report is obtained.

8. The method of claim 1, wherein, Based on the preset report evaluation model, the original chip evaluation report is evaluated to obtain a report evaluation result, including: A third model prompt information corresponding to the original chip evaluation report is generated; wherein the third model prompt information includes model prompt information of a data consistency dimension and model prompt information of a data professionalism dimension; The original chip evaluation report, the structured test result data and the third model prompt information are input into the preset report evaluation model, so that the report evaluation model evaluates the original chip evaluation report from the data consistency dimension and the data professionalism dimension to obtain a report evaluation result; Wherein, the report evaluation result is obtained by weighting and summing the first score result on the data consistency dimension and the second score result on the data professionalism dimension.

9. The method of claim 1, wherein, According to the report evaluation result and the original chip evaluation report, a target chip evaluation report is determined, including: According to the report evaluation result, it is determined whether the report evaluation score of the original chip evaluation report is greater than or equal to a preset score threshold, and when it is determined that the report evaluation score is greater than or equal to the preset score threshold, the original chip evaluation report is taken as the target chip evaluation report; When it is determined that the report evaluation score is less than the preset score threshold, the original chip evaluation report is optimized and adjusted until the report evaluation score is greater than or equal to the preset score threshold, and the original chip evaluation report after optimization and adjustment is taken as the target chip evaluation report.

10. A chip evaluation report generation apparatus characterized by comprising: Including: A to-be-tested chip generation module is configured to determine a to-be-tested chip and a to-be-tested performance associated with the to-be-tested chip, and to obtain original chip performance data corresponding to the to-be-tested chip and a historical test report example associated with the to-be-tested performance; A test report outline generation module is configured to generate a test report outline corresponding to the to-be-tested chip based on a preset report outline generation model according to the to-be-tested performance and the historical test report example; A performance test result generation module is configured to generate performance test results based on a preset content generation model according to the original chip performance data, and to map the performance test results to the test report outline to obtain an original chip evaluation report; A chip evaluation report generation module is configured to evaluate the original chip evaluation report based on a preset report evaluation model to obtain a report evaluation result, and to determine a target chip evaluation report according to the report evaluation result and the original chip evaluation report.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the chip evaluation report generation method of any one of claims 1-9.

12. An electronic device, comprising: Including: A processor; And A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to execute the chip evaluation report generation method of any one of claims 1-9.

Images