Verification system construction method and verification method

Abstract

The application discloses a verification system construction method and a verification method, which comprises the following steps: receiving configuration information specified by a user; the configuration information comprises path information of a plurality of components and a verification program identifier; reading source files of the plurality of components according to the path information of the plurality of components, and compiling the source files of each component according to a preset dependency relationship to obtain a first image file of each component; the preset dependency relationship is used for indicating a processing sequence of the source files of the plurality of components; obtaining a verification program according to the verification program identifier, and integrating the verification program into a root file system to obtain a second image file; and constructing a verification system based on the plurality of first image files and the second image file, thereby improving the efficiency of constructing the verification system and the efficiency of verification.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and particularly relates to a verification system construction method and a verification method. BACKGROUND

[0002] With the rapid development of integrated circuit technology, processor design is becoming increasingly complex. Before a chip is taped out, using a field programmable gate array to perform software and hardware co-verification and performance evaluation has become a key link to ensure the success of chip design.

[0003] Currently, when a verification environment is constructed for a field programmable gate array prototype, a manual process is usually used, and a series of compilation operations need to be manually executed by an engineer one by one. Manual operation requires the engineer to have a deep understanding of the construction process of the entire software stack, and to remember a large number of complex compilation commands and parameters. The operation process is lengthy and prone to errors, resulting in low efficiency of software construction and, in turn, low efficiency of the verification process. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a verification system construction method and a verification method, which can improve the efficiency of constructing a verification system and the efficiency of verifying a device.

[0005] In a first aspect, the embodiments of the present application provide a verification system construction method, which comprises:

[0006] receiving configuration information specified by a user; the configuration information comprising path information of a plurality of components and a verification program identifier;

[0007] reading source files of the plurality of components according to the path information of the plurality of components, and compiling the source files of each component according to a preset dependency relationship to obtain a first image file of each component; the preset dependency relationship is used to indicate the processing order of the source files of the plurality of components;

[0008] obtaining a verification program according to the verification program identifier, and integrating the verification program into a root file system to obtain a second image file;

[0009] constructing a verification system based on the plurality of first image files and the second image file.

[0010] In a second aspect, the embodiments of the present application provide a verification method, which comprises:

[0011] downloading a verification system from a first electronic device and starting the verification system;

[0012] creating a test environment for each processor in a second electronic device;

[0013] According to a target test case identifier pre-stored in the verification system, an execution parameter of the target test case is determined;

[0014] In a test environment of each processor, the target test case is run using the execution parameter to verify the processor, and a verification result is obtained.

[0015] In a third aspect, an embodiment of the present application provides a verification system construction device, which comprises:

[0016] A receiving module is configured to receive configuration information specified by a user, wherein the configuration information comprises path information of a plurality of components and a verification program identifier;

[0017] A compiling module is configured to read source files of the plurality of components according to the path information of the plurality of components, and compile the source files of each component according to a preset dependency relationship to obtain a first image file of each component, wherein the preset dependency relationship is used to indicate a processing order of the source files of the plurality of components;

[0018] An integrating module is configured to obtain a verification program according to the verification program identifier, and integrate the verification program into a root file system to obtain a second image file;

[0019] A construction module is configured to construct a verification system based on the plurality of first image files and the second image file.

[0020] In a fourth aspect, an embodiment of the present application provides a verification device, which comprises:

[0021] A starting module is configured to download a verification system from a first electronic device and start the verification system;

[0022] A creating module is configured to create a test environment for each processor in a second electronic device;

[0023] A parameter determining module is configured to determine an execution parameter of a target test case according to a target test case identifier pre-stored in the verification system;

[0024] A verification module is configured to verify the processor using the execution parameter to run the target test case in the test environment of each processor, and obtain a verification result.

[0025] In a fifth aspect, an embodiment of the present application provides an electronic device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement steps of the method.

[0026] In a sixth aspect, an embodiment of the present application provides a readable storage medium, the readable storage medium storing a program or instructions, the program or instructions being executed by a processor to implement the steps of the method described above.

[0027] In a seventh aspect, an embodiment of the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to run a program or instructions to implement the method described above.

[0028] In the embodiment of the present application, by using the predefined dependency relationship, a plurality of original manual construction steps are integrated into an automatic process. A user only needs to perform simple configuration, and the system can automatically complete all work from source code compilation, root file system customization to final image generation, thereby shortening the construction time from several hours or even several days to several minutes, and greatly improving the efficiency of constructing software carrying a verification program. During the verification process, the system can automatically detect the number of processor cores after starting, and create an isolated test environment for each processor core, thereby supporting all processor cores to execute test tasks in parallel, and improving the verification efficiency. By using the dual-process model of the test process and the time monitoring process, the execution time of the test can be accurately counted, and the health status of each core can be reflected in real time, thereby facilitating the rapid positioning of hardware failures such as downtime. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative labor.

[0030] Figure 1 is a step flow chart of an embodiment of a verification system construction method of the present application;

[0031] Figure 2 is a step flow chart of another embodiment of a verification system construction method of the present application;

[0032] Figure 3 is an architecture schematic diagram of a verification system of the present application;

[0033] Figure 4 is a step flow chart of an embodiment of a verification method of the present application;

[0034] Figure 5 is a step flow chart of another embodiment of a verification method of the present application;

[0035] Figure 6 is a step flow chart of another embodiment of a verification method of the present application;

[0036] Figure 7 is a structural block diagram of a verification system construction device of the present application;

[0037] Figure 8 is a structural block diagram of a verification device of the present application;

[0038] Figure 9 is a structural block diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the protection scope of the present application.

[0040] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a category and do not limit the number of objects, for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects. The term "multiple" in the embodiments of the present application means two or more, and other quantifiers are similar.

[0041] Method embodiments

[0042] Reference Figure 1 , a step flowchart of a verification system construction method embodiment of the present application is shown, the method comprises:

[0043] Step 101, receiving user-specified configuration information; the configuration information includes path information of a plurality of components and verification program identification.

[0044] Step 102, reading source files of a plurality of components according to the path information of the plurality of components, and compiling the source files of each component according to a preset dependency relationship to obtain a first image file of each component; the preset dependency relationship is used to indicate the processing order of the source files of the plurality of components.

[0045] Step 103, according to the verification program identification acquisition verification program, and integrate the verification program into the root file system, get the second mirror file.

[0046] Step 104, based on a plurality of the first mirror file and the second mirror file, construct a verification system.

[0047] For steps 101 to 104, it needs to be explained that the verification system construction method provided by the embodiment can be applied to the field of integrated circuit design, specifically, a complete software verification system is automatically constructed for a second electronic device on a first electronic device before a dedicated integrated circuit is taped out, exemplarily, the first electronic device can be a software development machine of x86 architecture, and the second electronic device can be a field programmable gate array (FPGA) prototype verification board carrying a fifth generation reduced instruction set computing (RISC-V) processor design to be verified. The verification system is used to comprehensively test the processor design on the FPGA prototype machine for functional correctness, performance and multi-core stability.

[0048] Specifically, the method of constructing a verification system in the present application mainly provides a graphical user interface through the Kconfig configuration system, receives the configuration information input by the user, exemplarily, in a schematic diagram of a configuration information selection interface, the user can input the path information of each component and the verification program identifier to be selected in the configuration information selection interface; an automatic compilation framework is built by using a generation file tool (Makefile) to build a system. The automatic compilation framework automatically schedules and executes the entire construction process based on the dependency relationship between the user configuration and the predefined components, and finally outputs a verification system mirror file that can be directly downloaded to the FPGA prototype machine, realizing the automatic construction of the verification system, and significantly improving the compilation efficiency compared with manual compilation by the user. Wherein, kconfig is a configuration option management system, and the developer can determine the functions, modules or parameters to be compiled through kconfig, makefile is a text file defining the compilation rules, which is parsed and executed by make tool, and is used for automatic management of the compilation and linking process of the project.

[0049] Among them, the configuration information is various information needed in the process of constructing the verification system input by the user, which is the input and basis of the entire automation process. Specifically, the form of configuration information can usually be a data set composed of a series of key-value pairs, which is usually saved in a configuration file named.config, and the configuration information mainly includes the path information of a plurality of components and the verification program identifier.

[0050] The path information of the plurality of components is used to indicate the location of each component source file required for the build verification system, and specifically, the path information is the directory location in the file system where each component source file is stored. The path information input by the user through the graphical interface is usually in the form of a string, such as / home / user / linux-riscv. Each component is a source code or template that has been prepared in advance, but the user can also flexibly customize the version of the component used by specifying the path indicated by the path information, for example, selecting to use the Linux 5.10 or Linux 6.1 kernel for verification.

[0051] The verification program identifier is used to uniquely indicate a performance test program to be integrated, and is the basis for the system to find and obtain the corresponding executable file in the preset program library. The verification program is usually in the form of a predefined string constant, such as SPEC2017 BLENDER TEST. For example, when the user selects this identifier, it means that the user wants to integrate the Blender test program in the SPEC Central Processing Unit (CPU) 2017 test set into the final verification system.

[0052] The source file of the component is a human-readable programming code file for most components, and for the root file system component, the source file of the root file system component is more accurately understood as a root file system template directory, which contains the basic file system structure and necessary tool software.

[0053] The preset dependency defines the order of each component in the build process, and is the core logic for driving the correct execution of the automated process, which can be a directed acyclic graph describing target-dependent task graphs. These directed acyclic graphs indicate the conditions required to achieve the target, and the preset dependency is written in the Makefile file.

[0054] The verification system construction method provided in the embodiments of the present application can compress the manual compiling process which needs several hours, needs to be executed in multiple steps and is highly dependent on the experience of experts into an automatic process which can be completed by one command, simplifies manual operation and thus improves the compiling efficiency. Moreover, in the manual compiling process, the user needs to manually repeat the whole compiling process for each test case, while in the embodiments of the present application, the user only needs to select a certain number of verification program identifiers, and the automatic compiling framework will compile based on the configuration information and the verification program identifiers provided by the user, and the user finally obtains is directly the mirror file of the complete verification system. Moreover, in the embodiments of the present application, the correctness and reliability of the construction process are ensured through the dependency management, the preset dependency relationship ensures that each component is compiled in the correct order, completely eliminates the construction failure caused by the sequence error of manual operation, guarantees the success rate of each construction attempt, and improves the reliability of the construction of the verification system.

[0055] With reference to Figure 2 , a step flowchart of another verification system construction method embodiment of the present application is shown, and the method comprises:

[0056] Step 201, receiving the configuration information specified by the user; the configuration information comprises path information of multiple components and verification program identifiers.

[0057] In the embodiments of the present application, the multiple components comprise a device tree, an operating system kernel and a bootloader, and the receiving of the configuration information specified by the user comprises receiving path information of the device tree, path information of the operating system kernel and path information of the bootloader. Specifically, the path information refers to the storage path of the source file of each component. For example, in a configuration information selection interface, the user inputs the path information of the device tree, the path information of the operating system kernel and the path information of the bootloader in the input box, as shown below:

[0058] (riscv-linux)kernel path

[0059] (riscv-linux / arch / riscv / boot / dts / bosc / nanhu-v5-1core.dtb)dtb path

[0060] (opensbi)opensbi path

[0061] Step 202, compiling the source file of the device tree to obtain the mirror file of the device tree.

[0062] Step 203, compiling the source file of the operating system kernel to obtain the mirror file of the operating system kernel.

[0063] Step 204, compiling the source file of the bootloader to obtain the image file of the bootloader.

[0064] It should be noted that, for steps 202 to 204, in the embodiments of the present application, the preset dependency relationship refers to the logical sequence between the component image file generation tasks defined in the automated build framework. The preset dependency relationship can ensure that the build process of the verification system is performed in the correct sequence, and the execution of the subsequent task depends on the successful completion of the preceding task. Specifically, the preset dependency relationship in the embodiments of the present application is that the generation of the final verification system image file depends on the fact that the operating system kernel image, the bootloader image, the device tree binary file and the root file system image are all ready. And the compilation of the operating system kernel and the bootloader needs to depend on the preparation of the device tree and other basic environments.

[0065] Among them, the device tree is a data structure describing the composition and resource configuration of a hardware system, and the role of the device tree is to pass the detailed hardware information of the FPGA prototype board, such as CPU type, memory size and address, peripheral type and address, etc., to the operating system kernel before the operating system kernel starts, so that the same kernel image can adapt to different hardware platforms. Generally, the device tree exists in the form of a human-readable text file, and the source file extension is.dts. For example, a file named fpga_board.dts describes that the number of processor cores on a specific FPGA development board is 4, the starting address of the memory is 0x80000000, and the like.

[0066] The compilation of the device tree source file is completed by the device tree compiler. The compilation of the device tree source file is to convert the.dts file in text format into a binary format file that can be directly parsed by the kernel, which is the image file corresponding to the device tree. The command used is usually: dtc -I dts -O dtb -o fpga_board.dtb fpga_board.dts.

[0067] The operating system kernel (such as Linux) is the core of the system software, responsible for managing all hardware resources and providing basic services for applications running on it. In this verification system, it provides core services such as process management, memory management, and file system necessary for running test programs such as Standard Performance Evaluation Corporation (SPEC). The operating system is composed of several source code packages, each of which is a large collection of millions of lines of C and assembly language code stored in user-specified directories, such as the source code tree at / home / user / linux-riscv.

[0068] The compilation process for the operating system kernel is as follows: using the automation framework, enter the kernel source path, call its own build system (usually using the make command), and specify the target architecture and cross-compilation toolchain, then compile and link all source code files to generate a kernel image. Depending on the configuration, the operating system kernel image file name is vmlinux, Image, etc.

[0069] The bootloader is the first piece of software code that runs after the system is powered on, such as the Open Supervisor Binary Interface (OpenSBI) commonly used on RISC-V platforms. The bootloader is responsible for initializing the lowest level of hardware, preparing a safe execution environment for the operating system, and ultimately transferring control to the operating system kernel. The operations performed by the bootloader can include setting up exception vectors, configuring memory controllers, and initializing timers.

[0070] The bootloader itself is also a software project composed of C and RISC-V assembly language source code, stored in a separate source code path, such as / home / user / opensbi. The compilation of the bootloader is also done through its own build system. In the source code directory, execute a command similar to make PLATFORM=generic through the automation framework, which compiles for a specific platform. The compiled bootloader firmware image file is, for example, fw_jump.bin for OpenSBI.

[0071] For example, assume that the user has configured the following paths through the graphical interface and executed the make command:

[0072] Device tree source file path: / home / user / linux-riscv / arch / riscv / boot / dts / fpga_board.dts

[0073] The operating system kernel source path is / home / user / linux-riscv

[0074] The bootloader source path is / home / user / opensbi

[0075] The automated build framework will perform the following processes based on the preset dependencies:

[0076] First, the device tree compiler is called to compile / home / user / ... / fpga_board.dts into fpga_board.dtb, that is, to obtain the device tree image file;

[0077] Subsequently, the framework enters the / home / user / linux-riscv directory and executes the make ARCH=riscv... command to compile and generate vmlinux, that is, to obtain the operating system kernel image file.

[0078] At the same time or thereafter, the framework enters the / home / user / opensbi directory and executes the make PLATFORM=generic... command to compile and generate fw_jump.bin, that is, to obtain the bootloader image file. Finally, the three component image files will be packaged together with the root file system image file to form a complete verification system image. Each component is a component of the verification system, and reference Figure 3 , shows a schematic diagram of the architecture of a verification system, which includes a root file system (tootfs), a bootloader (opensbi), and an operating system kernel (kernel).

[0079] Step 205, determining a test case set according to the verification program identifier.

[0080] Step 206, determining a target test case according to a selection operation of a user in the test case set.

[0081] Step 207, copying an executable file of the target test case to a specified directory of the root file system and saving a target test case identifier for use in a verification process.

[0082] For steps 205-207, it needs to be noted that the test case set is a list of multiple verification programs that can be selected by the user, and the test case set provides a clear and manageable interface for the user to specify the specific performance test items that need to be integrated into the verification system. For example, in the test case set selection interface, the test case set is usually presented in the form of a checkbox list or a radio button group. The user first selects a test case set from spec2006_testcase, spec2017_testcase and none in the interface. After selecting a test case set, the user needs to further select more specific test cases from this test case set as the final target test cases for device verification. After selecting the test case set, enter the schematic diagram of the test case selection interface. For example, a set for SPEC CPU 2017 test set may contain:

[0083] SPEC2017 BLENDER TEST

[0084] SPEC2017 GCC TEST

[0085] SPEC2017 CACTUBSSN TEST ...

[0086] The user further selects the final test case for verification from the interface corresponding to the test case set. The number of target test cases is determined according to the test requirements, which can be one, two, or multiple.

[0087] In the embodiments of the present application, the verification program identifier is used to indicate a whole test case set. The user can select the final target test case from the test case set selection interface by checking one or more checkboxes in the graphical interface. The target test case is a subset of the test case set, which is the specific verification program that needs to be actually integrated into the root file system in this build process. The target test case is the test case that finally verifies the device, that is, the test item that the user finally checks. The executable file of the target test case will be integrated into the final verification system and automatically executed after the FPGA prototype machine is started, which is used to perform performance stress testing and functional verification on the processor.

[0088] Executable refers to a binary program file that has been compiled for the RISC-V target architecture and can be run directly on the operating system. For example, the executable corresponding to SPEC2017 BLENDER TEST can be a binary file named blender_r. Copy is a file system operation that refers to copying the target executable file located in the pre-compiled verification program set path to the specified location in the root file system directory structure being built, for example, copying the executable file of the target test case to the / bin or / usr / bin directory.

[0089] The root file system is the first file system mounted by the Linux operating system when starting, which contains all the directory structures, application programs, library files, configuration files and device nodes necessary for the operation of the operating system. During the build process, the root file system is a directory that already contains standard directories such as / bin, / etc, / lib, etc. For example, / home / user / busybox_rootfs / ; finally, this directory will be packaged into an image file, such as rootfs.cpio.gz. It should be noted that the user will also input the path information of the root file system in the configuration information selection interface, for example, (system / rootfs) system path.

[0090] Exemplarily, assume that the user inputs the configuration information and the selection operation as follows:

[0091] Pre-compiled verification program set path: / home / user / spec_cpu2017 / bin /

[0092] Root file system template path: / home / user / busybox_rootfs /

[0093] User-selected target test case identifier: SPEC2017 BLENDER TEST

[0094] The automated compilation framework determines that the executable file to be copied is blender_r according to the identifier SPEC2017 BLENDER TEST.

[0095] The automated compilation framework executes the copy command: cp / home / user / spec_cpu2017 / bin / blender_r / home / user / busybox_rootfs / bin / .

[0096] Write the string "SPEC2017_BLENDER_TEST" into a configuration file in the root file system template, such as / home / user / busybox_rootfs / etc / test_case.

[0097] It should be noted that in addition to the executable file of the integration target test case in the root file system, an automatic execution script is also configured, which is automatically executed in the last stage of Linux system startup, responsible for pulling up the verification management program, thereby triggering the entire automated test process without any manual intervention. This script can be placed in the directory under the root file system template that will be automatically executed when the operating system is initialized, such as / etc / rc.local. The script is mainly used to start a master management program. After the Linux system on the FPGA completes the startup, the / etc / rc.local script is automatically executed, reads the previously saved test_case configuration file, obtains the "SPEC2017_BLENDER_TEST" identifier; then queries the internal parameter table to obtain the complete execution command blender_r sh3_no_char.blend...; then detects the CPU core, creates an independent directory, and finally executes the command on each core.

[0098] Step 208, based on the plurality of first image files and the second image file, constructing a verification system.

[0099] Step 208 can refer to step 104 described above, which will not be repeated here.

[0100] For example, the user opens the graphical configuration interface makemenuconfig generated by Kconfig on the first electronic device and configures as follows:

[0101] Linux kernel source path: / home / user / workspace / linux-riscv

[0102] Device tree source file path: / home / user / workspace / linux-riscv / arch / riscv / boot / dts / fpga_board.dts

[0103] OpenSBI source path: / home / user / workspace / opensbi

[0104] Root file system template path: / home / user / workspace / busybox_rootfs

[0105] Pre-compiled SPEC suite path: / home / user / workspace / spec_cpu2017 / bin

[0106] Then the verification program selection test case set is tested on the first graphical interface, the first graphical interface provides a verification program identifier for selecting a test case set, after the user selects a test case set, a second graphical interface corresponding to the test case set will be further popped up, and the second graphical interface is used to provide an identifier of each test case in the test case set, so that the user can select a target test case.

[0107] In the test case set provided in the second graphical interface, check:

[0108] [X] SPEC2017_BLENDER_TEST

[0109] [X] SPEC2017_GCC_TEST

[0110] The user saves the configuration and exits the interface, and the automatic compilation framework stores all configurations in the.config file.

[0111] The user executes the make command on the first electronic device, the automatic build framework is started, and the following processes are executed based on the preset dependency relationship:

[0112] The storage path of the device tree source file is accessed by the command dtc -I dts -O dtb -o / build / output / fpga_board.dtb / home / user / workspace / linux-riscv / arch / riscv / boot / dts / fpga_board.dts, the device tree is compiled, and the device tree binary file / build / output / fpga_board.dtb is obtained

[0113] The storage path of the Linux operating system kernel source file is accessed by the command cd / home / user / workspace / linux-riscv&&make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- -j8 O= / build / kernel_output, the Linux kernel is compiled, and the image file of the kernel / build / kernel_output / arch / riscv / boot / Image is obtained

[0114] The storage path of the bootloader opensbi source file is accessed by the command cd / home / user / workspace / opensbi&&make PLATFORM=generic CROSS_COMPILE=riscv64-unknown-linux-gnu- FW_PAYLOAD_PATH= / build / kernel_output / arch / riscv / boot / Image, the bootloader is compiled, and the image file / build / output / fw_jump.bin of the bootloader is obtained

[0115] The root file system of the verification program is constructed and integrated, the copy command is executed to copy the executable file of the target test case to the specified directory in the root file system, and the identification of the target test case is saved. For example, the file / build / rootfs_work / etc / test_cases is created, and the content is as follows:

[0116] text

[0117] SPEC2017_BLENDER_TEST

[0118] SPEC2017_GCC_TEST

[0119] The automatic execution script is generated, and the automatic execution script is also integrated into the specified directory in the root file system.

[0120] Then, the root file system is packaged, and the image file of the root file system is obtained.

[0121] The image file of the finally generated verification system can include: an operating system kernel, a device tree, a bootloader, a root file system, an automatic execution script, a test case identification, and an executable script of a test case.

[0122] To sum up, the verification system construction method provided in the embodiments of the present application automatically constructs a complete verification system based on the compilation order or construction order among the pre-set operating system kernel, opensbi, root file system, and test case, without the need for the user to participate in the compilation process of each component, simplifying the operation of the user and improving the efficiency and reliability of obtaining the verification system. The user can configure different versions of the kernel, OpenSBI, root file system template, and test case, and the user can quickly customize the most suitable verification system according to different verification requirements, improving the flexibility of constructing the verification system and the adaptability of the verification system. Moreover, in the process of generating the verification system, the user can select one, two, or multiple test cases, which facilitates one-time compilation and improves the efficiency of compilation.

[0123] Referring to Figure 4 , a step flow chart of an embodiment of a verification method of the application is shown, the method comprising:

[0124] Step 301, downloading a verification system from a first electronic device and starting the verification system;

[0125] Step 302, creating a test environment for each processor in a second electronic device;

[0126] Step 303, determining the execution parameters of the target test case according to the target test case identifier pre-stored in the verification system;

[0127] Step 304, running the target test case in the test environment of each processor using the execution parameters to verify the processor and obtaining a verification result.

[0128] For steps 301 to 304, the verification method provided in the embodiment of the application can be applied in the integrated circuit design process to verify the functional correctness, performance and stability of the processor design before the application specific integrated circuit is taped out. In the application, the operations of developing, compiling and building the verification system are performed on the first electronic device, and the verification process is performed on the second electronic device, which is an FPGA prototype verification board carrying the RISC-V processor design to be verified. The first electronic device is responsible for building a complete verification system and transmitting the verification system to the second electronic device through a network or a storage device. The two constitute a development environment and a target execution environment.

[0129] It should be noted that the core idea of the verification method is to load the complete verification system automatically built and generated on the first electronic device to the FPGA prototype machine, and after the system is started, the verification process can be automatically executed. Specifically, through a pre-written automatic execution script, the preparation of the multi-core environment, the analysis of the test parameters, the parallel test execution and the result monitoring are automatically completed without any manual intervention, realizing the full-process automation from system startup to verification completion. Illustratively, the overall process of verifying the second electronic device is as follows: after the verification system image is loaded into the FPGA prototype machine memory and powered on, the boot loader, the operating system kernel are executed in turn, and finally the root file system is mounted; then the execution script pre-stored in the root file system is called to perform verification by executing the script. Through such a design, seamless connection from system startup to verification start is ensured, and no manual terminal input of any command is required on the second electronic device.

[0130] The verification system is a final image file built on the first electronic device, is software including a complete verification environment, and contains a complete set of software components required from hardware initialization to test execution. The verification system is pre-built on the first electronic device, and includes a bootloader, an operating system kernel, a device tree, and a root file system integrated with a verification program.

[0131] Each processor in the second electronic device is a verified computing unit and is also a physical carrier of verification test execution. A single-core or multi-core RISC-V processor is implemented on the FPGA prototype. The verification method provided in the embodiments of the present application creates an independent test environment for each processor core to achieve truly parallel testing and accurate performance statistics. The test environment is created for each processor core to provide an isolated running space for each processor core, prevent files and states among multiple test instances from interfering with each other, and ensure the independence and accuracy of the test results.

[0132] The target test case identifier is used to uniquely refer to a test program that needs to be executed inside the verification system. The form of the target test case identifier is usually a predefined string, which is saved in a configuration file inside the verification system. For example, "SPEC2017_BLENDER_TEST" identifier represents that a three-dimensional (3D) rendering performance test needs to be performed.

[0133] The execution parameter refers to a specific command line instruction required to start and run the test case, which ensures that the test is executed in a standard and normative manner. Specifically, the execution parameter is a complete command line string, which includes a program path, an input file, a running option, and the like. For example, the SPEC2017_BLENDER_TEST case execution command is: blender sh3_no_char.blend --render-output sh3_no_char_ --threads 1 -b -F RAWTGA -s 849 -e 849 -a; and the SPEC2017_ cactuBSSN _TEST" case execution command is. / cactuBSSNspec_ref.par.

[0134] Specifically, a configuration file pre-existing in the root file system is read to obtain the target test case identifier. A parameter mapping data structure is maintained inside the system, for example, a parameter table. The complete execution command corresponding to the target test case identifier is queried through the target test case identifier, that is, the execution parameter is obtained. According to the number of processor cores, the test task is distributed to the independent test environment of each core. In the test environment of each processor core, the complete execution command obtained from the parameter table is used to create a new process through an operating system call to run the test program.

[0135] After the test environment is created for each processor core, the tests on all processor cores are executed in parallel, and after the tests are completed, the system collects the exit status, execution time and output file of each core to form the final verification result. The verification result is the output of the verification process, and is used to evaluate the correctness and performance of the processor design. The verification result includes the exit status code of the test program, accurate execution time data, output file and system monitoring log, etc.

[0136] The embodiment of the application provides a verification method. From the power-on starting of the FPGA to the collection of the final test result, the whole process does not need the engineer to input any command in front of the serial terminal or manually record any data, greatly simplifies the operation of the user, and improves the verification efficiency. In the application, each processor core creates an environment and executes a test, fully develops the hardware potential, realizes real multi-core parallel stress testing, and improves the verification efficiency. Unlike the traditional single-core test, the multi-core parallel test can truly simulate the working state of the processor under full load, and expose complex problems such as cache consistency, memory bandwidth competition, and inter-core interrupt that may exist when multiple cores work cooperatively.

[0137] Reference Figure 5 , a step flow chart of another verification method embodiment of the application is shown, and the method comprises:

[0138] Step 401, downloading the verification system from the first electronic device and starting the verification system;

[0139] Step 401 can refer to step 301 described above, and will not be described here.

[0140] Step 402, creating a corresponding test working directory for each processor; the test working directory is used to save the intermediate files and log files generated in the running process of the target test case;

[0141] Step 403, initializing the corresponding test environment in each test working directory.

[0142] For steps 402 and 403, it should be noted that in the verification process, the number of processor cores in the second electronic device is first detected, and then a separate test directory is created for each detected core in the writable area of the root file system, or a dedicated folder is created for each processor core, for example: / test_env / cpu0, / test_env / cpu1. The test working directory is used to store temporary files, output results and logs generated during the test runtime of each processor core, achieving physical isolation of the test environment. The test working directory provides a truly multi-core parallel verification sandbox environment, preventing temporary files, output files and logs generated when multiple processor cores run test programs simultaneously from being overwritten or conflicting with each other, ensuring that the test process of each core is independent, the results are accurate and traceable. Exemplarily, on a 4-core RISC-V processor, the following test working directory structure is created:

[0143] text

[0144] / test_env /

[0145] ├──cpu0 /

[0146] ├──cpu1 /

[0147] ├──cpu2 /

[0148] └──cpu3 /

[0149] Among them, the intermediate file refers to the process data file dynamically generated by the test program during the running process, which is used to record the intermediate state, temporary result or on-site information for debugging, and is the direct product and snapshot of the test case execution process, rather than the final report; the form of the intermediate file is a binary or text file in the test program self-defined format. Exemplarily, the target test case being run is SPEC2017_BLENDER_TEST for 3D rendering test, and the intermediate file may include: partial rendering completed image block file, scene calculation temporary cache file, memory exchange data file, etc.

[0150] The intermediate file is usually cleaned up after the test is successful, but it provides key clues for analyzing the root cause of the problem when the test fails.

[0151] The log files systematically record the execution flow, standard output, standard error, system status and custom diagnostic information of the test program. The logs are used to monitor the test behavior, diagnose error causes and conduct post-audit. The log files are in the form of plain text files. For example, under the test working directory of each processor core, the following log files can be generated: stdout.log, which records the information printed to the standard output by the test program; stderr.log, which records the error and warning information of the test program; system_monitor.log, which records the resource usage of the core during the test. The log files are usually retained regardless of the success or failure of the test for analysis and reporting.

[0152] It should be noted that during the process of creating the test working directory, subdirectories that the test program can need can also be created under the test working directory cpuX, such as. / output for storing the final output and. / temp for storing temporary files. The static input files relied on by the test cases are copied from the public resource area to each working directory. For example, for SPEC2017_BLENDER_TEST, the initialization process copies the input model file sh3_no_char.blend and the parameter file spec_ref.ini to each cpuX directory. Among them, the initialization of the test environment is to ensure that each test working directory has all the dependent resources necessary for the correct operation of the test program.

[0153] Step 404, comparing the target test case identifier with each table entry in the parameter table; each table entry in the parameter table is used to store the mapping relationship between the target test case identifier and the execution parameter;

[0154] Step 405, according to the comparison result, determining the execution parameter in the table entry matched with the target test case identifier as the execution parameter of the target test case.

[0155] For steps 404 and 405, the target test case identifier is used as the unique key for searching in the parameter table to accurately locate and obtain the execution parameter corresponding to the target test case. The target test case identifier is usually in the form of a predefined string constant, which is usually selected by the user during the graphical configuration stage and saved in the system configuration file. For example, "SPEC2017_BLENDER_TEST", "SPEC2017_GCC_TEST", etc.

[0156] The parameter table is a key-value mapping data structure, which stores the standardized execution specification of each target test case. The specific form of the parameter table can be a hash table in memory, a configuration file, or a static array compiled into the program. For example, a static structure array initialized inside the program. The table entry in the parameter table is the basic unit of composition in the parameter table, and one table entry can define the execution parameters of a test case.

[0157] The table entry is a data structure containing two fields: key and value. The key is the target test case identifier, and the value is the complete execution parameter of the test case. For example, a record in the parameter table: {key: "SPEC2017_BLENDER_TEST", value: "blender_r sh3_no_char.blend..."}.

[0158] Comparison is the process of accurately matching the target test case identifier input by the user with the key field of each table entry in the parameter table. In computing, it is usually represented as a lookup operation on a hash table or array. For example, the program calls a function similar to find_parameters("SPEC2017_BLENDER_TEST"). The comparison result is the output generated after the comparison operation, that is, whether the same identifier table entry is found in the parameter table. When the input target test case identifier is exactly equal to the key field of a table entry in the parameter table, that is, a match is successful.

[0159] Execution parameters are command line instructions required to start the test process of the operating system, ensuring that the test runs in a predetermined manner. The form of the execution parameter is a complete, directly executable string in the Shell, usually containing the following parts: program path, input file, command line option, and output direction, etc. The program path refers to the location of the executable file in the root file system, such as blender_r; the input file is the input data file required for testing, such as sh3_no_char.blend; the command line option refers to parameters that control program behavior, output format, thread number, etc., such as --threads 1-b; the output direction refers to the specified way and path of the result output. For example, a complete execution parameter may be: "1.blender sh3_no_char.blend --render-output sh3_no_char_ --threads 1 -b-F RAWTGA -s 849 -e 849 -a".

[0160] Exemplarily, the pre-stored target test case identification is read from the configuration file / etc / test_cases into a line: "SPEC2017 BLENDER TEST".

[0161] The structure of the parameter table is as follows:

[0162] {

[0163] .callback = testcase_run,

[0164] .folder = "blender",

[0165] .ident = "blender",

[0166] .s_cmd = "blender",

[0167] .v_cmd = "",

[0168] .param = "sh3_no_char.blend --render-output sh3_no_char_--threads 1-b -F RAWTGA -s 849-e 849-a",

[0169] },

[0170] {

[0171] .callback = testcase_run,

[0172] .folder = "bwaves",

[0173] .ident = "bwaves_1",

[0174] .s_cmd = "bwaves",

[0175] .v_cmd = "",

[0176] .param = "bwaves_1<bwaves_1.in",

[0177] },

[0178] "SPEC2017_BLENDER_TEST" is looked up in the key set of the parameter table by traversal or hash lookup. It is found that the string is identical to the key "SPEC2017_BLENDER_TEST" of the first table item, and the match is successful. The value of the matching table item, that is, the string "blender_r sh3_no_char.blend --render-output sh3_no_char_ --threads 1 -b -F PNG -s 849 -e 849 -a", is determined as the final execution parameter of this test.

[0179] Step 406, in the test environment of each processor, simultaneously start the target test case and the time monitoring program in the process of verifying each processor.

[0180] Step 407, periodically record the timestamp information in the verification process through the time monitoring program.

[0181] Step 408, when the target test case is executed, send a termination signal to the time monitoring process.

[0182] Step 409, according to the start and end time of the timestamp information, calculate the test execution time of the target test case.

[0183] It should be noted that for steps 406 to 409, the test case is started for verification, and the time monitoring process is started at the same time. The time monitoring process periodically records the timestamp information, and the time monitoring program is an independent running daemon process, mainly responsible for periodically recording time information, and has the dual functions of accurate timing and system heartbeat monitoring. The time monitoring program is a lightweight background process, which is usually implemented by a simple loop and sleep mechanism. Illustratively, the time monitoring program is a program written in C language or Python, which executes the operation of obtaining system time and writing to file every second.

[0184] The timestamp information is an absolute time value recorded at a specific time, which is used to construct a time series and is the basic data for calculating time intervals and judging system activity. The form of timestamp information is a high-precision date and time string or a nanosecond / microsecond number calculated from a certain fixed starting point. Illustratively, the timestamp information is "2024-05-20 10:30:45.123456" or the nanosecond number 18456789012345 since the system started.

[0185] The termination signal is used for inter-process communication, notifying the time monitoring program to stop working, marking the end of the monitoring task. The termination signal is a standard inter-process communication mechanism provided by the operating system, such as the signal in Unix / Linux. Illustratively, the verification process sends a user-defined signal to the time monitoring process through kill(pid, SIGUSR1).

[0186] The test execution time can quantify the actual time experienced by the test case from start to end, and is a key indicator for evaluating processor performance. The test execution time is a time interval value, usually in seconds or milliseconds. Illustratively, the test execution time is equal to the end timestamp minus the start timestamp, equal to 3605.75 seconds.

[0187] It should be noted that the time monitoring program can also be used to analyze the stability of the recording interval of the timestamp, which can indirectly determine the system load. If the timestamp interval fluctuates abnormally, it may indicate that the system is experiencing high load or resource competition. For long-running tests, the number of recorded timestamps can be used to estimate the test progress. When running the same test on different versions of hardware design, comparing the changes in test execution time can also quantify performance improvement or identify performance regression.

[0188] Illustratively, the target test case is SPEC2017_BLENDER_TEST, the number of processor cores is 4, and the execution parameter is "blender_r sh3_no_char.blend --threads 1 -b -s 1 -e 1 -a".

[0189] First, start two processes for each processor core at the same time, and the test process executes blender_r sh3_no_char.blend... in the respective working directory; the time monitoring process executes time_monitor --log-file / cpuX / time.log --interval 1.

[0190] The time monitoring process of each core writes a timestamp to the corresponding log file every second:

[0191] text

[0192] # / cpu0 / time.log content example

[0193] 2024-05-20 10:00:00.000001

[0194] 2024-05-20 10:00:01.000125

[0195] 2024-05-20 10:00:02.000243 ...

[0196] When the test process of core 0 finishes execution at 2024-05-20 10:05:30.456789, the test process exits and returns an exit code. After the test process exits, the time monitoring process of core 0 is sent a SIGUSR1 signal, that is, a termination signal. After receiving the termination signal, the time monitoring process records the last timestamp 2024-05-20 10:05:30.456789 and then exits.

[0197] After the test ends, / cpu0 / time.log is read, the start time is 2024-05-20 10:00:00.000001, and the end time is 2024-05-20 10:05:30.456789. Therefore, the test execution time of core 0 is 10:05:30.456789-10:00:00.000001=330.456788 seconds.

[0198] If core 3 stops recording timestamps after 2024-05-20 10:02:15.123456, while other cores continue to record normally, the hypervisor will immediately detect that the heartbeat of core 3 has stopped, determine that the core may have crashed, and trigger the corresponding alarm and debugging information collection mechanism.

[0199] Optionally, step 408 comprises:

[0200] Sub-step 4081: When the target test case finishes execution, monitor a program execution flag of the target test case; the program execution flag is used to indicate whether the target test case executes successfully or fails.

[0201] Sub-step 4082: In the case that the program execution flag is failure, save intermediate files and log files in the verification process.

[0202] Sub-step 4083: In the case that the program execution flag is success, clean up the intermediate files.

[0203] It should be noted that for sub-steps 4081 to 4083, the form of the program execution flag can be a process exit status code standardized in the operating system, which is usually an integer value. For example, an exit code of 0 indicates that the program successfully completes all tasks, and an exit code other than 0 indicates that the program terminates due to an error, for example, 1 indicates a general error, 2 indicates a parameter error, and so on.

[0204] Saving intermediate files and log files when test fails is to keep the debugging environment completely, to provide sufficient evidence for problem analysis, specifically, the entire test working directory including all intermediate files and log files is compressed and packaged, a timestamp and test identification are added to generate a unique file name, and then transmitted to a dedicated persistent storage area. When the test succeeds, the storage space is released, reducing the system burden, and keeping the system clean. Specifically, identify and delete specific types of intermediate files, keep necessary log files for auditing and statistics, and reset the test environment to the initial state.

[0205] Illustratively, the test case is SPEC2017 BLENDER TEST, and the test working directory is / test_env / cpu0 / . The intermediate files generated during the verification process are / test_env / cpu0 / render_tile_1.dat, / test_env / cpu0 / temp.cache, and the generated log files are / test_env / cpu0 / stdout.log, / test_env / cpu0 / stderr.log. When the program execution flag is not equal to 0, that is, in the case of test failure, an archive is created by tar -czf / debug / blender_fail_cpu0_20240520_110023.tar.gz / test_env / cpu0 / , and all files are saved by render_tile_1.dat, temp.cache, stdout.log, stderr.log. When the program execution flag is equal to 0, that is, in the case of test success, the intermediate files are deleted by rm -f / test_env / cpu0 / render_tile_1.dat / test_env / cpu0 / temp.cache; the log files stdout.log, stderr.log are kept for result analysis. The final directory content includes stdout.log, stderr.log and the final rendering output file.

[0206] In summary, the verification method provided in the embodiments of the present application can accurately capture the real system time from process startup to end for each processor core through independent and high-frequency timestamp recording. The continuous output of the timestamp constitutes the heartbeat of each CPU core, and once the heartbeat of a certain core stops, the system can locate the down core within seconds, achieving rapid discovery and accurate positioning of hardware failure. Moreover, the execution parameters are obtained by querying the internal parameter table, which ensures that each test is run in exactly the same way. The risk of performance fluctuations caused by inconsistent manual input of command parameters is reduced, and the test results between different periods and different hardware versions have strict comparability, providing a reliable basis for performance optimization.

[0207] Referring to Figure 6 , a step flowchart of another verification method embodiment of the present application is shown, and the method comprises:

[0208] Step A1, initializing system parameters of a test environment.

[0209] Step A2, obtaining a test case name.

[0210] Step A3, initializing a current time.

[0211] Step A4, obtaining a number of processor cores in a current system.

[0212] Step A5, displaying system construction time or number.

[0213] Step A6, obtaining running parameters.

[0214] Step A7, executing a test case and a heartbeat program for each processor core.

[0215] Step A8, when the program execution is completed, detecting whether intermediate generated files and log files need to be printed, and notifying the heartbeat program to stop working.

[0216] It should be noted that, for steps A1 to A8, this is a complete and automated process of a verification process. First, all necessary information is collected, such as system parameters, test cases, core numbers, running parameters, etc., and the test environment is initialized, such as time and parameters. The hardware resources are fully utilized to synchronously initiate the test on all cores. Through the double-process mode, the test case is executed while the accurate measurement and health monitoring are simultaneously completed, realizing the unity of test efficiency and data reliability. Finally, the test result can be used to automatically decide how to process the massive output data. If the test is successful, the redundancy is cleaned up, and if the test fails, the scene is preserved, which greatly improves the debugging efficiency and storage space utilization.

[0217] In particular, the initialization system parameters include, but are not limited to, the test framework version number, the SPEC benchmark test version, the test mode is scalar or vector, the log level, etc. These system parameters are written into the log header, ensuring that the results of any test can be accurately traced and understood. The total number of logical cores on the second electronic device is obtained by reading the Linux system file or calling the system API. The number of processor cores can determine the number of test environments to be created and the total number of processes to be initiated, which is the key to dynamic parallelization. The system construction time or number of times is displayed or recorded to display or record the construction timestamp or construction ID of the verification system image. If the test fails, the problem can be narrowed down. Obtaining the running parameter means that an accurate match is made in a predefined built-in parameter table, and the corresponding command line execution string is taken out.

[0218] For each core, the system will make two fork()+exec() system calls, respectively creating a test process and a heartbeat process. The test process is used for testing or verification, and the heartbeat process is a simple monitoring program used to obtain and record timestamps. When the final test process is executed, the execution result of the test process is used to determine whether to save the file.

[0219] Exemplarily, assume that the parameters initialized by the system are as follows:

[0220] Test framework version: Ver2.1

[0221] SPEC benchmark test version: CPU2017

[0222] Test mode: vector

[0223] Log level: DEBUG

[0224] When the system obtains the test case name, the test mode (scalar / vector) is used as a key factor when querying the parameter table to obtain the running parameter. For example, the cactuBSSN test case may have two different executable files, scalar and vector. The system will select the corresponding executable file and parameters according to the test mode being vector. It may not query. / cactuBSSN_s spec_ref.par (scalar version), but. / cactuBSSN_r spec_ref.par (vector version).

[0225] When creating a test environment for each processor core, because the log level is DEBUG, the system will configure the test program and management script to output the most detailed running information to the log file. If the level is WARNING, only warning and error information will be recorded, thereby saving disk space and I / O overhead.

[0226] All initialized system parameters, along with system build time, processor core number, etc., are recorded in the header of the final verification report and log file. The final test result is obtained under the Ver2.1 framework for the vector test mode of the CPU2017 benchmark.

[0227] In the embodiments of the present application, the verification is automatically executed, the manual operation of the user is simplified, the manual cost is reduced, the verification efficiency is improved, the risk of errors caused by manual operation is reduced, and the reliability in the verification process is improved. Moreover, the verification efficiency is improved in a parallel manner by testing multiple processor cores independently, and the load environment closer to reality is achieved.

[0228] Device embodiments

[0229] Reference Figure 7 Fig. 1 shows a logic block diagram of a verification system construction device provided by an embodiment of the present application. The device can include:

[0230] The receiving module 510 is configured to receive configuration information specified by a user. The configuration information includes path information of multiple components and a verification program identifier.

[0231] The compiling module 520 is configured to read source files of the multiple components according to the path information of the multiple components, and compile the source files of each component according to a preset dependency relationship to obtain a first image file of each component. The preset dependency relationship is used to indicate a processing order of the source files of the multiple components.

[0232] The integrating module 530 is configured to obtain a verification program according to the verification program identifier, and integrate the verification program into a root file system to obtain a second image file.

[0233] The construction module 540 is configured to construct a verification system based on the multiple first image files and the second image file.

[0234] Optionally, the compiling module includes:

[0235] The first compiling module is configured to compile the source file of the device tree to obtain an image file of the device tree.

[0236] The second compiling module is configured to compile the source file of the operating system kernel to obtain an image file of the operating system kernel.

[0237] The third compiling module is configured to compile the source file of the bootloader to obtain an image file of the bootloader.

[0238] Optionally, the integrating module includes:

[0239] A use case set determination module is configured to determine a test use case set according to the identified verification program;

[0240] A use case determination module is configured to determine a target test use case according to a selection operation of the user in the test use case set;

[0241] A copying module is configured to copy an executable file of the target test use case to a specified directory of the root file system, and save a target test use case identifier for use by a verification process.

[0242] In summary, the verification system construction apparatus provided in the embodiments of the present application can compress a manual compilation process that needs to be performed in multiple steps and highly depends on expert experience into an automatic process that can be completed by one command, thereby simplifying manual operation and improving the compilation efficiency. In the embodiments of the present application, the user only needs to select a certain number of verification program identifiers, and the automatic compilation framework will perform compilation based on the configuration information and the verification program identifiers provided by the user, and the user finally obtains is directly the mirror image file of the complete verification system. In the embodiments of the present application, the correctness and reliability of the construction process are ensured through dependency management, the preset dependency relationship ensures that each component is compiled in a correct order, the construction failure caused by the order error of manual operation is completely eliminated, the success rate of each construction attempt is ensured, and the reliability of the construction of the verification system is improved.

[0243] Reference Figure 8 , a logic block diagram of a verification apparatus provided in an embodiment of the present application is shown, which can include:

[0244] A starting module 610 is configured to download a verification system from a first electronic device and start the verification system;

[0245] A creating module 620 is configured to create a test environment for each processor in a second electronic device;

[0246] A parameter determination module 630 is configured to determine an execution parameter of a target test use case according to a target test use case identifier pre-stored in the verification system;

[0247] A verification module 640 is configured to run the target test use case in the test environment of each processor using the execution parameter to verify the processor, and obtain a verification result.

[0248] Optionally, the parameter determination module includes:

[0249] a comparison module, configured to compare the target test case identifier with each table entry in the parameter table; each table entry in the parameter table is used to store a mapping relationship between a target test case identifier and an execution parameter;

[0250] a parameter determination sub-module, configured to determine, according to the comparison result, the execution parameter in the table entry matched with the target test case identifier as the execution parameter of the target test case.

[0251] Optionally, the verification module comprises:

[0252] a starting module, configured to start the target test case and the time monitoring program simultaneously in the process of verifying each processor;

[0253] a recording module, configured to record, by the time monitoring program, time stamp information in the verification process periodically;

[0254] a sending module, configured to send a termination signal to the time monitoring process when the target test case is executed completely;

[0255] a calculation module, configured to calculate a test execution time of the target test case according to start and end times of the time stamp information.

[0256] Optionally, the sending module comprises:

[0257] a monitoring module, configured to monitor a program execution flag of the target test case when the target test case is executed completely; the program execution flag is used to indicate whether the target test case is executed successfully or unsuccessfully;

[0258] a saving module, configured to save intermediate files and log files in the verification process when the program execution flag is failed;

[0259] a cleaning module, configured to clean the intermediate files when the program execution flag is successful.

[0260] Optionally, the creating module comprises:

[0261] a directory creating module, configured to create a corresponding test working directory for each processor; the test working directory is used to save intermediate files and log files generated in the running process of the target test case;

[0262] an initialization module, configured to initialize a corresponding test environment in each test working directory.

[0263] In summary, the embodiment of the present application provides a verification device, from the power-on start of the FPGA to the final test result collection, the whole process does not require an engineer to input any command in front of a serial terminal or manually record any data, greatly simplifies the operation of the user, and improves the verification efficiency. Moreover, in the present application, each processor core creates an environment and performs a test, fully develops the hardware potential, realizes real multi-core parallel stress testing, and improves the verification efficiency. Unlike traditional single-core testing, multi-core parallel testing can truly simulate the working state of the processor under full load, and expose complex problems such as cache consistency, memory bandwidth competition, and inter-core interruption that may exist when multiple cores work together.

[0264] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0265] Reference Figure 9 is a structural block diagram of an electronic device for verifying system construction or verification provided by the embodiment of the present application. As shown in Figure 9 , the electronic device comprises a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface complete mutual communication through the communication bus; the memory is used for storing executable instructions, and the executable instructions make the processor execute the method of the foregoing embodiment.

[0266] The processor can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, digital signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array), or other programmable devices, transistor logic devices, hardware components, or any combination thereof. The processor can also be a combination that realizes computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0267] The communication bus can include a channel for transmitting information between the memory and the communication interface. The communication bus can be a PCI (Peripheral Component Interconnect, peripheral component interconnect) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9Only one line is used to represent a bus, but it could also be a combination of buses or bus segments, a simple wire, or various other means.

[0268] The memory can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0269] The embodiments of the present application also provide a non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device (a server or a terminal), the processor can execute the method of the foregoing embodiments.

[0270] The embodiments of the present application further provide a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions, realizes various processes of the foregoing embodiments, and can achieve the same technical effects. To avoid repetition, details are not described herein.

[0271] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0272] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0273] Those skilled in the art should understand that the embodiments of the embodiments of the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0274] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It is to be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of the flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal devices to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing terminal devices, generate a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0275] These computer program instructions can also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal devices to work in a predictive manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0276] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0277] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0278] Finally, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it should be taken in its broadest possible sense. For example, the terms "first" and "second" are used herein only to differentiate one element from another, and do not imply any required order or sequence unless otherwise indicated by the context. Further, the description should not be construed as using limiting terminology such as "only" or "exclusively." In addition, as used herein, the singular forms "a," "an" and "the" include plural references unless the context clearly dictates otherwise. Also, terms such as "including," "containing," or "comprising" are intended to be open and non-limiting and are understood to mean that anything other than what is recited is possible.

[0279] The above provides a kind of verification system construction method, device, electronic equipment and readable storage medium and a kind of verification method, device, electronic equipment and readable storage medium provided by the present application, have carried out detailed introduction, the principle and implementation mode of the present application are described in this paper with specific examples, the above example is only for helping understanding the method of the present application and its core idea;For the person skilled in the art, according to the idea of the present application, there will be changes in specific implementation mode and application range, as described above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A method for constructing a verification system, characterized in that, Applied to a first electronic device, the method includes: Receive configuration information specified by the user; the configuration information includes path information of multiple components and verification program identifier; The source files of the multiple components are read according to the path information of the multiple components, and the source files of each component are compiled according to the preset dependency relationship to obtain the first image file of each component; the preset dependency relationship is used to indicate the processing order of the source files of the multiple components. The verification program is obtained based on the verification program identifier, and the verification program is integrated into the root file system to obtain the second image file; A verification system is constructed based on multiple first image files and second image files.

2. The method according to claim 1, characterized in that, The components include an operating system kernel, a bootloader, and a device tree; the process of compiling the source files of each component according to preset dependencies to obtain a first image file for each component includes: The source files of the device tree are compiled to obtain the image file of the device tree; The source files of the operating system kernel are compiled to obtain an image file of the operating system kernel; The source files of the bootloader are compiled to obtain the image file of the bootloader.

3. The method according to claim 1, characterized in that, The step of obtaining the verification program based on the verification program identifier and integrating the verification program into the root file system to obtain the second image file includes: The test case set is determined based on the verification program identifier; The target test case is determined based on the user's selection action in the test case set; Copy the executable file of the target test case to the specified directory of the root file system and save the target test case identifier for use in the verification process.

4. A verification method, characterized in that, Applied to a second electronic device, the method includes: The verification system is downloaded from the first electronic device and then launched; the verification system is obtained by the method described in any one of claims 1 to 3. Create a test environment for each processor in the second electronic device; Based on the target test case identifier pre-stored in the verification system, determine the execution parameters of the target test case; In the test environment of each processor, the target test case is run using the execution parameters to verify the processor and obtain the verification results.

5. The method according to claim 4, characterized in that, The step of determining the execution parameters of the target test case based on the target test case identifier pre-stored in the verification system includes: The target test case identifier is compared with each entry in the parameter table; each entry in the parameter table is used to store the mapping relationship between the target test case identifier and the execution parameters. Based on the comparison results, the execution parameters in the table entries that match the target test case identifier are determined as the execution parameters of the target test case.

6. The method according to claim 4, characterized in that, The step of verifying the processor by running the target test case using the execution parameters includes: During the verification process for each processor, the target test cases and time monitoring program are launched simultaneously. The time monitoring program periodically records timestamp information during the verification process; When the target test case is completed, a termination signal is sent to the time monitoring program; The test execution time of the target test case is calculated based on the start and end times of the timestamp information.

7. The method according to claim 6, characterized in that, The step of sending a termination signal to the time monitoring program when the target test case has been executed includes: When the target test case is completed, the program execution flag of the target test case is monitored; the program execution flag is used to indicate whether the target test case was executed successfully or failed. If the program execution flag indicates failure, save the intermediate files and log files from the verification process. If the program execution flag indicates success, clean up the intermediate files.

8. The method according to claim 4, characterized in that, Creating a test environment for each processor in the second electronic device includes: Create a corresponding test working directory for each processor; the test working directory is used to store intermediate files and log files generated during the execution of the target test cases; Initialize the corresponding test environment in each test working directory.

9. A verification system construction apparatus, characterized in that, The device includes: A receiving module is used to receive configuration information specified by the user; the configuration information includes path information of multiple components and a verification program identifier; The compilation module is used to read the source files of the multiple components according to the path information of the multiple components, and compile the source files of each component according to the preset dependency relationship to obtain the first image file of each component; the preset dependency relationship is used to indicate the processing order of the source files of the multiple components. An integration module is used to obtain a verification program based on the verification program identifier and integrate the verification program into the root file system to obtain a second image file; A building module is used to build a verification system based on multiple first image files and second image files.

10. A verification device, characterized in that, The device includes: A startup module is configured to download and start the verification system from a first electronic device; the verification system is obtained by the method described in any one of claims 1 to 3. Create modules to create test environments for each processor in the second electronic device; The parameter determination module is used to determine the execution parameters of the target test case based on the target test case identifier pre-stored in the verification system. The verification module is used to verify the processor by running the target test case in the test environment of each processor using the execution parameters, and to obtain the verification results.

Images