Online automatic evaluation method and system for multi-granularity Verilog hardware description language
Through the multi-grained online automatic evaluation method, Verilog hardware description language code is subject to multi-grained evaluation, which solves the problems of low efficiency of traditional evaluation methods and inability to intuitively understand the connection between code and waveforms, and achieves efficient evaluation result grading and student learning efficiency improvement.
Patent Information
- Application Number
- CN202411971183.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-06-03
AI Technical Summary
The traditional Verilog hardware description language code evaluation method is inefficient and cannot give detailed scores for some correct answers, which cannot allow students to intuitively understand the connection between waveforms and code in Verilog hardware description language.
The multi-grained online automatic evaluation method is used to simulate the code to be tested and the sample evaluation through the Icarus Verilog simulator. It compares the multi-grained evaluation results based on multiple evaluation points, including scores, waveform diagrams of wrong answers and waveform diagrams of correct answers.
It realizes multi-graining and hierarchical evaluation results, improves teachers' teaching quality and students' learning efficiency, helps students understand code errors more intuitively and improve their writing skills.
Smart Images
Figure CN120087292A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer program evaluation, and particularly to an online automatic evaluation method and system for multi-granularity Verilog hardware description language. Background Art
[0002] Usually in teaching, it is necessary to evaluate the Verilog hardware description language code submitted by students to improve the teaching quality, enhance the effect of homework feedback and teaching efficiency.
[0003] In the current educational environment and background, computer science and engineering have been increasingly emphasized. As an important hardware description language in computer languages, Verilog hardware description language plays an important role in digital system design and integrated circuit design. However, the traditional code evaluation method has great limitations in the judgment of specific code writing. For example, it is necessary to manually run the code again, and it can only show whether the question is right or wrong, and cannot show the score for partially correct answers. Students cannot get refined and multi-granular homework feedback, etc. At the same time, it is very difficult for students to intuitively understand the code errors from the simulation waveforms only by constantly modifying the code, which also brings some troubles to students' further improvement of the writing ability of Verilog hardware description language code and systematic engineering thinking.
[0004] Therefore, there is an urgent need for an online automatic evaluation method for multi-granularity Verilog hardware description language to solve the technical problems that the traditional code judgment method has low efficiency, cannot give refined scores for partially correct answers, and cannot enable students to intuitively understand the connection between waveforms and codes in Verilog hardware description language. Summary of the Invention
[0005] The purpose of the present invention is to overcome the deficiencies of the prior art, and provide an online automatic evaluation method and system for multi-granularity Verilog hardware description language, which realizes the multi-granularity and grading of evaluation results, and further improves the teaching quality of teachers and the learning efficiency of students.
[0006] The present invention adopts the following technical solutions:
[0007] On the one hand, the present invention provides an online automatic evaluation method for multi-granularity Verilog hardware description language, including:
[0008] S1. Accept a program evaluation request;
[0009] S2. Read the evaluation sample and perform simulation;
[0010] S3. Read the code to be tested and perform simulation;
[0011] S4. Compare the simulation results of the evaluation samples obtained in step S2 with the simulation results of the code under test obtained in step S3, perform multi-granularity evaluation based on multiple evaluation points, and output the multi-granularity evaluation results;
[0012] There is no order between steps S2 and S3.
[0013] In any of the possible implementation manners described above, a further implementation manner is provided. In step S1, when a program evaluation request is made, input the code file under test F1, the sample code file F2, and the test stimulus file F3; F1 is a.v file, F2 is a.v file, and F3 is one or more_tb.v files; after receiving the evaluation request, output the file name lists of F1, F2, and F3, denoted as L1, L2, and L3 respectively.
[0014] In any of the possible implementation manners described above, a further implementation manner is provided. In step S2, with L2 and L3 as the input, use the Icarus Verilog simulator to complete the simulation of the evaluation samples, and output the simulation result sequence R1;
[0015] In step S3, with L1 and L3 as the input, use the Icarus Verilog simulator to complete the simulation of the code under test, and output the simulation result sequence R2.
[0016] In any of the possible implementation manners described above, a further implementation manner is provided. The simulation parameters include the program simulation time and the number of evaluation points set; each evaluation point corresponds to a test stimulus file F3, and it also includes the main test stimulus file for the main evaluation.
[0017] In any of the possible implementation manners described above, a further implementation manner is provided. In step S4, when comparing, with the simulation result sequences R1 and R2 as the input, output the test differential comparison file X, and each evaluation point corresponds to a test differential comparison file; determine whether the simulation results are consistent by judging whether the test differential comparison file is empty;
[0018] The multi-granularity evaluation includes the correctness test of the code under test and the correctness test of the simulation results; the multi-granularity evaluation results include the score based on the number of evaluation points, the sample reference answers and the wrong answers of the code under test for the evaluation points where errors occur, and the comparison of the simulation waveforms of the code under test and the sample code.
[0019] In any of the possible implementation manners described above, a further implementation manner is provided. The specific multi-granularity evaluation is as follows:
[0020] S41. Main evaluation: The test content is the set of independent sub-functions tested by each evaluation point, and there is an interaction between the sub-functions; if the main evaluation passes, the score is 100 and the test ends; if the main evaluation fails, go to step S42;
[0021] S42. Test the evaluation points in sequence until all unpassed evaluation points are detected. Count the number of passed evaluation points. After multiplying the ratio of the number of passed evaluation points to the total number of evaluation points by the weight value of the corresponding evaluation point, multiply by 100 and divide by the total weight value to obtain the score. At the same time, output the comparison of the simulation waveforms of the evaluation point information, the code to be tested, and the sample code.
[0022] In any of the possible implementation manners described above, a further implementation manner is provided. The simulation waveform is stored in svg format and serialized into a JSON-formatted string.
[0023] In any of the possible implementation manners described above, a further implementation manner is provided. In step S1, the requester of the program evaluation request includes the student user terminal participating in submitting the assignment.
[0024] On the other hand, the present invention further provides an online automatic evaluation system for multi-granularity Verilog hardware description language. The system is used to implement the method as described above. The system includes:
[0025] A user terminal for receiving a program evaluation request and inputting a code file F1 to be tested, a sample code file F2, and a test stimulus file F3.
[0026] A main evaluation module for calling the code file F1 to be tested, the sample code file F2, and the test stimulus file F3, using the Icarus Verilog simulator to complete the simulation of the evaluation sample and the code to be tested, obtaining the output evaluation information, and converting the output into a JSON format for display.
[0027] An evaluation function implementation module for comparing the simulation results of the evaluation sample and the code to be tested, performing multi-granularity evaluation based on multiple evaluation points, and outputting multi-granularity evaluation results.
[0028] A data display module for displaying the score obtained by the code to be tested, the waveform diagram of the wrong answer, and the waveform diagram of the correct answer.
[0029] In any of the possible implementation manners described above, a further implementation manner is provided. The evaluation function implementation module has the function of automatically detecting the number of evaluation points. During the evaluation process, this module will first perform a main evaluation to determine whether the code file to be tested is completely correct. If the main evaluation passes, other evaluation points will not be run. If the main evaluation fails, multiple evaluation points will be tested in sequence, and the number of passed evaluation points will be calculated to give the score and the information of the wrong evaluation points.
[0030] The beneficial effects of the present invention are:
[0031] 1. It solves the problem that multi-granularity evaluation cannot be completed in the current Verilog hardware description language evaluation method. Through setting the number of evaluation points in the evaluation stage, the present invention achieves true multi-granularity evaluation; according to the evaluation results, scores from 0 to 100 are given, and the corresponding simulation waveforms and machine comments of the tested files with errors are given; it is beneficial to improve the evaluation efficiency and evaluation results of Verilog hardware description language code.
[0032] 2. It solves the technical problem that students cannot intuitively understand the connection between waveforms and code in Verilog hardware description language. According to the evaluation results, the comparison between the waveform diagram corresponding to the error code and the waveform diagram corresponding to the sample code answer given can help students more easily know the specific location of the error, so as to modify the code, and further help students more profoundly understand the connection between Verilog hardware description language code and the corresponding waveform diagram. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 The figure shows a schematic diagram of the overall process in an online automatic evaluation method for a multi-granularity Verilog hardware description language according to an embodiment of the present invention.
[0034] Figure 2 The figure shows the file structure of an online automatic evaluation method for a multi-granularity Verilog hardware description language in an embodiment.
[0035] Figure 3 The figure shows the file structure of a test stimulus file in an online automatic evaluation method for a multi-granularity Verilog hardware description language in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0036] The following will describe in detail specific embodiments of the present invention with reference to specific drawings. It should be noted that the technical features described in the following embodiments or the combination of technical features should not be considered isolated, and they can be combined with each other to achieve better technical effects.
[0037] An online automatic evaluation method for a multi-granularity Verilog hardware description language according to an embodiment of the present invention includes:
[0038] S1. Accept a program evaluation request;
[0039] S2. Read the evaluation sample code and perform simulation;
[0040] S3. Read the code to be tested and perform simulation;
[0041] S4. Compare the simulation results of the evaluation sample code obtained in step S2 with the simulation results of the code to be tested obtained in step S3, perform multi-granularity evaluation based on single or multiple evaluation points, and output multi-granularity evaluation results;
[0042] Steps S2 and S3 have no sequence.
[0043] Such as Figure 1 , Figure 2 As shown, the present invention provides a file structure of an online automatic evaluation method for a multi-granularity Verilog hardware description language, including: a main evaluation component and a test data component. Among them, the main evaluation component contains the main core code, which is jointly written in the python language and the shell script language, and is the core component of the present invention, realizing the main function of the present invention.
[0044] In a specific embodiment, as Figure 1 shown, an online automatic evaluation method for a multi-granularity Verilog hardware description language includes the following steps:
[0045] S101. Receive a program evaluation request sent by a requester.
[0046] The requester includes a party that sends the Verilog hardware description language code to be evaluated and the corresponding test stimuli to the evaluation system to request the evaluation of the code and the test stimuli; for example, the requester can be a student user terminal participating in submitting homework. The requester can log in to the online evaluation website, submit the code to be evaluated and the corresponding test stimuli, and send an evaluation request for evaluation.
[0047] S102, S103. Read the sample code file F2 and the test stimulus file F3 and perform simulation, and read the code file F1 to be tested and the test stimulus file F3 and perform simulation.
[0048] The test stimulus is a combination of test inputs for testing the correctness of the code and can be one or more files; for example, a test stimulus can include 5 evaluation points, named point1 to point5 respectively. Among the evaluation points in the form of point + number, it includes a test stimulus file based on the number of test points and a main test stimulus file. The main test component will automatically judge the number of evaluation points in the test stimulus; the file structure of the test stimulus file is as Figure 3 shown;
[0049] For the main test stimulus, through the code simulation file in the main evaluation module, the sample and the code to be tested are respectively simulated and the simulation results are output for further comparison and judgment in step S104; when multi-evaluation point evaluation is required, that is, when the simulation results of the code to be tested and the sample code in the main test differential comparison file are inconsistent, for each test stimulus file of the evaluation point, the main evaluation module will respectively simulate the sample and the code to be tested and output the simulation results.
[0050] S104. Perform multi-granularity evaluation based on the number of evaluation points according to the sample simulation results and the simulation results of the code under test.
[0051] The simulation results of the code are data results generated by judging whether the circuit functions generated after the code passes compilation are the same.
[0052] For example, by comparing the simulation result 0 generated by simulating the sample and the main test stimulus file in step S102, and the simulation result 1 generated by simulating the code under test and the main test stimulus file in step S103, a main test differential comparison file can be generated, and by judging whether the main test differential comparison file is empty, it can be further determined whether the simulation results are the same.
[0053] If the main test differential comparison file is empty, it indicates that the simulation results are the same, and the program will not perform subsequent evaluation point evaluations. When the simulation results are different, return to steps S102 and S103 to perform multiple evaluations based on the set number of evaluation points. By comparing the simulation result 2 generated by simulating the sample and one test stimulus file in one evaluation point in step S102, and the simulation result 3 generated by simulating the code under test and one test stimulus file in one evaluation point in step S103, an evaluation point differential comparison file can be generated, and by judging whether the evaluation point differential comparison file is empty, it can be determined whether the simulation results are the same, and multiple evaluations are performed according to the number of evaluation points.
[0054] Based on this, in an optional example of the present application, when performing multi-granularity evaluation in the present application, first, the evaluation of the main test stimulus file will be performed, and the corresponding main evaluation differential comparison file will be generated to determine whether the simulation results are the same. If it is determined that the simulation results are the same, it will be regarded as passing the main test stimulus and full marks will be given. If the simulation results are different, the evaluation program will evaluate the test stimulus files in each evaluation point separately with the simulation results generated by the sample and the code under test to obtain a simulation result differential comparison file based on all evaluation points, so as to determine the correctness of the code under test for each evaluation point. Further, by calculating the ratio of the number of evaluation points that the code under test passes correctly to the total number of evaluation points, a multi-granularity evaluation score based on a percentage system can be given.
[0055] S105. Output the multi-granularity evaluation results.
[0056] The multi-granularity evaluation results here refer to a series of JSON-formatted strings generated by the multi-granularity automatic evaluation method, including machine comments showing the passing situation of each evaluation point, scores, as well as total comments, waveforms at error locations (stored in svg format and serialized into JSON), etc. These information can be sent to an online evaluation website for display.
[0057] In an optional example of the present application, the evaluation results include, but are not limited to, syntax error "SYNTAX ERROR", answer correct "ACCEPTED", answer score "SCORE", specific passed test points "POINT0 WRONG,POINT1 CORRECT", etc.
[0058] An online automatic evaluation system for a multi-granularity Verilog hardware description language according to an embodiment of the present invention includes:
[0059] A client for submitting a program evaluation request and inputting a code file F1 to be tested, a sample code file F2, and a test stimulus file F3;
[0060] A main evaluation module for calling the code file F1 to be tested, the sample code file F2, and the test stimulus file F3, using the Icarus Verilog simulator to complete the simulation of the evaluation sample and the code to be tested, obtaining the output evaluation information, and converting the output into a JSON format for display;
[0061] An evaluation function implementation module for comparing the simulation results of the evaluation sample and the code to be tested, performing multi-granularity evaluation based on multiple evaluation points, and outputting multi-granularity evaluation results;
[0062] A data display module for displaying the score obtained by the code to be tested, the waveform diagrams of wrong answers and correct answers.
[0063] In a specific embodiment, the evaluation function implementation module has the function of automatically detecting the number of evaluation points. During the evaluation process, this module will first perform a main evaluation to determine whether the code file to be tested is completely correct. If the main evaluation passes, other evaluation points will not be run; if the main evaluation fails, multiple evaluation points will be tested in sequence, and the number of passed evaluation points will be calculated to give the score and information on wrong evaluation points.
[0064] Although several embodiments of the present invention have been given in this article, those skilled in the art should understand that the embodiments in this article can be changed without departing from the spirit of the present invention. The above embodiments are only exemplary and should not be used as a limitation of the scope of the rights of the present invention.
Claims
1. An online automatic evaluation method for multi-granularity Verilog hardware description language, characterized in that: The method comprises: S1. Accept the request for program evaluation; S2, read the evaluation sample and perform simulation; S3, read the code to be tested and perform simulation; S4, comparing the simulation result of the test sample obtained in step S2 with the simulation result of the code to be tested obtained in step S3, performing multi-granularity evaluation based on multiple evaluation points, and outputting the multi-granularity evaluation result; There is no order of precedence for steps S2 and S3.
2. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 1, characterized in that: In step S1, when a program evaluation request is received, the code file to be tested F1, the sample code file F2, and the test stimulus file F3 are input; F1 is a .v file, F2 is a .v file, and F3 is one or more _tb.v files; after accepting the evaluation request, a list of file names of F1, F2, and F3 is output, which are recorded as L1, L2, and L3, respectively.
3. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 2, characterized in that: In step S2, the input is L2 and L3, and the Icarus Verilog simulator is used to complete the simulation of the evaluation sample, and the simulation result sequence R1 is output; In step S3, the input is L1 and L3, and the Icarus Verilog simulator is used to complete the simulation of the code to be tested, and the simulation result sequence R2 is output.
4. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 3, characterized in that: The simulation parameters include the program simulation time and the number of evaluation points to be set; each evaluation point corresponds to a test stimulus file F3.
5. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 3, characterized in that: In step S4, when the comparison is performed, the input is the simulation result sequences R1 and R2, and the output is the test difference comparison file X, where each evaluation point corresponds to a test difference comparison file; By judging whether the test difference comparison file is empty, we can judge whether the simulation results are consistent; The multi-granularity evaluation includes a test of the correctness of the code to be tested and a test of the correctness of the simulation results; The multi-granularity evaluation results include scores based on the number of evaluation points, sample reference answers for evaluation points where errors occurred and incorrect answers for the code to be tested, and simulation waveform comparisons between the code to be tested and the sample code.
6. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 5, characterized in that: The multi-granularity evaluation is specifically as follows: S41, main evaluation: the test content is a collection of independent sub-functions tested at each evaluation point, and there is interaction between each sub-function; if the main evaluation passes, the score is 100 and the test ends; if the main evaluation fails, go to step S42; S42. Test the evaluation points in sequence until all failed evaluation points are detected, count the number of passed evaluation points, multiply the ratio of the number of passed evaluation points to the total number of evaluation points by the corresponding evaluation point weight, multiply by 100 and divide by the total weight to get the score; at the same time, output the evaluation point information, the simulation waveform comparison of the code to be tested and the sample code.
7. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 5, characterized in that: The simulation waveform is stored in SVG format and serialized into a string in JSON format.
8. The online automatic evaluation method of the multi-granularity Verilog hardware description language as claimed in claim 1, characterized in that: In step S1, the requester of the program evaluation request includes a student user terminal participating in submitting an assignment.
9. An online automatic evaluation system for multi-granularity Verilog hardware description language, characterized in that: The system is used to implement the method according to any one of claims 1 to 8, and the system includes: The user end is used to accept the program evaluation request and input the code file to be tested F1, the sample code file F2, and the test stimulus file F3; The main evaluation module is used to call the code file F1 to be tested, the sample code file F2, and the test stimulus file F3, use the Icarus Verilog simulator to complete the simulation of the evaluation sample and the code to be tested, obtain the output evaluation information, and convert the output into a JSON format that can be displayed; The evaluation function realization module is used to compare the simulation results of the evaluation sample with the simulation results of the code to be tested, perform multi-granularity evaluation based on multiple evaluation points, and output multi-granularity evaluation results; The data display module is used to display the score obtained by the code to be tested, the waveform graph of the wrong answer and the waveform graph of the correct answer.
10. The online automatic evaluation system of the multi-granularity Verilog hardware description language according to claim 9, characterized in that: The evaluation function realization module has the function of automatically detecting the number of evaluation points. During the evaluation process, the module first performs the main evaluation to determine whether the code file to be tested is completely correct. If the main evaluation passes, other evaluation points will not be run; If the main evaluation fails, multiple evaluation points will be tested in sequence, and the number of passed evaluation points will be calculated to give a score and error evaluation point information.