Chip visual data inference capability testing method and electronic device

Abstract

The application discloses a chip visual data reasoning capability test method and electronic equipment, and relates to the technical field of chip testing. The method comprises the following steps: processing visual original data sets based on a data arrangement mechanism to obtain visual test data sets, solving the one-sidedness and limitations of single data types and single data formats on the calibration chip capability, and helping to scientifically and comprehensively evaluate the to-be-tested chip; testing the to-be-tested chip according to a preset test model set to obtain an output result set, solving the one-sidedness and limitations of a single test model on the calibration chip capability, and helping to scientifically and comprehensively evaluate the to-be-tested chip; and judging the effectiveness of the output result set according to a monitoring result set, thereby improving the accuracy of the visual test result, and enabling the visual test result to accurately represent the visual data reasoning capability of the to-be-tested chip.

Description

Technical Field

[0001] This application relates to the field of chip testing technology, and in particular to a method for testing the visual data reasoning ability of a chip and an electronic device. Background Technology

[0002] Artificial intelligence (AI) chips are mainly classified into the following types: (i) Graphics Processing Unit (GPU): initially used for graphics processing, but due to its parallel processing capabilities, it has become an indispensable part of AI model training; (ii) Field-Programmable Gate Array (FPGA): highly programmable and suitable for AI tasks requiring advanced customization; (iii) Application-Specific Integrated Circuit (ASIC): customized for specific AI applications, and its performance is usually superior to more general-purpose chips; (iv) Neural Processing Unit (NNU): specifically designed for deep learning and neural networks, capable of rapidly processing large amounts of data. These AI chips can significantly improve data processing efficiency while reducing the processor's workload, allowing the processor to concentrate resources on running software and applications, maximizing the system's value. Existing AI chips generally use nominal computing power to represent their visual data reasoning capabilities, typically measured in tops (TOPS), which indicates how many integer or floating-point calculations can be processed per computation cycle. However, actual tests show that the nominal computing power of an AI chip does not represent its actual visual data reasoning capability during application. Judging and selecting AI chips based on nominal computing power is a misconception in the industry. For example, an AI chip with a nominal computing power of 100 TOPS may be able to process more images than an AI chip with a nominal computing power of 275 TOPS.

[0003] Therefore, there is an urgent need to propose a testing method for the visual data reasoning ability of chips and electronic devices that can obtain accurate test results. Summary of the Invention

[0004] This application provides a method and electronic device for testing the visual data reasoning capability of a chip, in order to at least solve the problem in the related art that the nominal computing power of artificial intelligence chips does not represent their true visual data reasoning capability in application.

[0005] This application provides a method for testing the visual data inference capability of a chip. The method includes: receiving a set of raw visual data, wherein the raw visual data set includes at least a first type of raw data, a second type of raw data, and a third type of raw data; processing the first type of raw data, the second type of raw data, and the third type of raw data based on a data orchestration mechanism to obtain a set of visual test data; testing the chip under test according to a preset test model set, a preset test order, and the set of visual test data to obtain a set of output results, while monitoring the testing process of the chip under test to obtain a set of monitoring results; determining the validity of the set of output results based on the set of monitoring results; retesting the chip under test if the set of output results is invalid; and determining the set of model weight values ​​corresponding to the preset test model set if the set of output results is valid, and obtaining the visual test result of the chip under test based on the set of model weight values ​​and the set of output results.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement a method for testing the visual data reasoning capability of a chip, comprising at least the following steps: receiving a set of raw visual data, wherein the raw visual data set includes at least a first type of raw data, a second type of raw data, and a third type of raw data; processing the first type of raw data, the second type of raw data, and the third type of raw data based on a data arrangement mechanism to obtain a set of visual test data; testing the chip under test according to a preset test model set, a preset test order, and the set of visual test data to obtain a set of output results, while simultaneously monitoring the testing process of the chip under test to obtain a set of monitoring results; determining the validity of the set of output results based on the set of monitoring results; retesting the chip under test in response to an invalid set of output results; and determining a set of model weight values ​​corresponding to the preset test model set in response to a valid set of output results, and obtaining the visual test result of the chip under test based on the set of model weight values ​​and the set of output results.

[0007] This application utilizes a data orchestration mechanism to process raw visual data sets, obtaining a visual test data set. This addresses the limitations and biases of using a single data type or format to calibrate chip capabilities, facilitating a scientific and comprehensive evaluation of the chip under test. Furthermore, by testing the chip under test using a pre-defined test model set, an output result set is obtained. This also overcomes the limitations and biases of using a single test model to calibrate chip capabilities, further contributing to a scientific and comprehensive evaluation. Finally, the validity of the output result set is determined based on the monitoring result set, thereby improving the accuracy of the visual test results and enabling them to accurately represent the visual data reasoning capabilities of the chip under test. Attached Figure Description

[0008] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating the method for testing the visual data reasoning capability of the chip in the first embodiment;

[0010] Figure 2 This is a flowchart illustrating the inference execution steps in the first embodiment;

[0011] Figure 3 This is a flowchart illustrating the reasoning and judgment steps in the first embodiment;

[0012] Figure 4 This is a structural block diagram of the chip's visual data reasoning capability testing system in the second embodiment;

[0013] Figure 5 This is a diagram of the internal structure of the electronic device in the third embodiment. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0015] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0016] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Step 1: Prepare hardware resources, deploy the server and AI chip, and check if the AI ​​chip has visual data inference capabilities. If it does not, testing is unnecessary; if it does, install the operating system, check the machine status, and ensure the AI ​​chip is functioning correctly and the server has no errors. Step 2: Install the necessary driver packages and software environment for the AI ​​chip (including system software such as C++, Python, Java, AI chip drivers, encoding / decoding tools, AI inference tools, and various software development kits). Execute encoding / decoding commands to ensure the AI ​​chip runs correctly within the driver package and software environment. Deploy the AI ​​model set (including four main categories: image classification, object recognition, image segmentation, and action recognition models) and ensure each model runs correctly. Step 3: Prepare test data, including images, videos, and video streams. For video streams, it is recommended to obtain the data using a camera directly connected to the server's network port. Image and video data resolutions should cover 720p, 1080p, 4K, and 8K. Step 4: Check the availability of test data and whether the camera can be opened and streamed successfully through the server. After data preparation, evaluate the visual data inference capability of the AI ​​chip according to the testing method. Step 5: During testing, pay attention to various check results and the load on the AI ​​chip to ensure full resource utilization. Incomplete resource utilization means the test results do not reflect its full capability and should not be considered the final result. Monitor the usage of each component. During execution, the AI ​​chip generally only uses the resources of the AI ​​chip card and its internal video memory. If it uses system processor memory and bandwidth, it indicates that the AI ​​chip cannot process image and video data independently and requires the processor to complete the work, which is inefficient. This situation requires in-depth analysis and should be reflected in the report. Step 6: Based on the AI ​​inference results and the analysis of previous steps, check whether the report content matches the analysis conclusions of Step 5. If there are problems, modify or retest; if there are no problems, accept it as the final result.

[0018] In the first embodiment, such as Figure 1As shown, a method for testing the visual data inference capability of a chip is provided. The method includes: receiving a set of raw visual data, wherein the raw visual data set includes at least a first type of raw data, a second type of raw data, and a third type of raw data; processing the first type of raw data, the second type of raw data, and the third type of raw data based on a data orchestration mechanism to obtain a set of visual test data; testing the chip under test according to a preset test model set, a preset test order, and the set of visual test data to obtain a set of output results, while monitoring the testing process of the chip under test to obtain a set of monitoring results; determining the validity of the set of output results based on the set of monitoring results; retesting the chip under test if the set of output results is invalid; and determining the set of model weight values ​​corresponding to the preset test model set if the set of output results is valid, and obtaining the visual test result of the chip under test based on the set of model weight values ​​and the set of output results.

[0019] Specifically, the system processes the raw visual data set based on a data orchestration mechanism to obtain a visual test data set. This addresses the limitations and biases of using a single data type or format to calibrate chip capabilities, and facilitates a scientific and comprehensive evaluation of the chip under test. Furthermore, the system tests the chip under test using a pre-defined test model set to obtain an output result set. This also addresses the limitations and biases of using a single test model to calibrate chip capabilities, and facilitates a scientific and comprehensive evaluation of the chip under test. Finally, the system judges the validity of the output result set based on the monitoring result set, thereby improving the accuracy of the visual test results and enabling the visual test results to be accurately used to represent the visual data reasoning capabilities of the chip under test.

[0020] In a specific embodiment, the first type of raw data is image data, the second type of raw data is video data, and the third type of raw data is video stream data. For image and video data, the data must be stored on a local hard drive; for video stream data, it is generally obtained through an external network camera.

[0021] In a specific embodiment, the preset test model set includes representative models for image classification tasks: ResNet-50 residual network and Vision Transformer (ViT-B / 16); representative models for object detection tasks: YOLO series single-stage detection models and Faster R-CNN two-stage detection models; representative model for image segmentation tasks: U-Net; and representative model for video action recognition tasks: SlowFast.

[0022] Furthermore, based on a data orchestration mechanism, the first type of raw data, the second type of raw data, and the third type of raw data are processed to obtain a visual test data set. This includes: determining the data types of the first type of raw data, the second type of raw data, and the third type of raw data, wherein the first type of raw data belongs to the first data type, the second type of raw data belongs to the second data type, and the third type of raw data belongs to the third data type; determining the first data processing operation corresponding to the first data type based on the mapping relationship between data types and data processing operations, and performing the first data processing operation on the first type of raw data to obtain first type of test data; determining the second data processing operation corresponding to the second data type based on the mapping relationship between data types and data processing operations, and performing the second data processing operation on the second type of raw data to obtain second type of test data; determining the second data processing operation corresponding to the third data type based on the mapping relationship between data types and data processing operations, and performing the second data processing operation on the third type of raw data to obtain third type of test data; and obtaining a visual test data set based on the first type of test data, the second type of test data, and the third type of test data.

[0023] In a specific embodiment, the first data type is an image, the second data type is a video, and the third data type is a video stream. The data volume of the first, second, and third types of test data can also be set. After format conversion and resolution adjustment, methods such as copying can be used to ensure that the first, second, and third types of test data meet the corresponding data volume requirements.

[0024] Further, a first data processing operation is performed on the first type of raw data to obtain first type of test data, including: performing a first format conversion operation on the first type of raw data to obtain first type of intermediate data, wherein the first format conversion operation is used to convert the format of the first type of raw data according to a first data format set, the first data format set including at least a first data format, a second data format, a third data format, a fourth data format, and a fifth data format; performing a first resolution adjustment operation on the first type of intermediate data to obtain first type of test data, wherein the first resolution adjustment operation is used to adjust the resolution of the first type of intermediate data according to a preset data resolution set, the preset data resolution set including at least a first data resolution, a second data resolution, a third data resolution, and a fourth data resolution, and the first type of test data includes any combination of each data format in the first data format set and each data resolution in the preset data resolution set.

[0025] Specifically, by performing a first data processing operation on the first type of raw data, the limitations of a single data format and a single resolution in calibrating the chip's capabilities based on the first type of raw data are resolved, which helps to scientifically and comprehensively evaluate the chip under test.

[0026] In a specific embodiment, the first data format, the second data format, the third data format, the fourth data format, and the fifth data format are jpeg, png, bmp, avif, and apng, respectively, and the first resolution, the second resolution, the third resolution, and the fourth resolution are 720p, 1080p, 4K, and 8K, respectively.

[0027] Further, a second data processing operation is performed on the second type of raw data to obtain second type of test data, including: performing a second format conversion operation on the second type of raw data to obtain second type of intermediate data, wherein the second format conversion operation is used to convert the format of the second type of raw data according to a second data format set, the second data format set including at least a sixth data format, a seventh data format, an eighth data format, a ninth data format, a tenth data format, an eleventh data format, and a twelfth data format; performing a second resolution adjustment operation on the second type of intermediate data to obtain second type of test data, wherein the second resolution adjustment operation is used to adjust the resolution of the second type of intermediate data according to a preset data resolution set, the second type of test data containing any combination of each data format in the second data format set and each data resolution in the preset data resolution set.

[0028] Specifically, by performing a second data processing operation on the second type of raw data, the limitations of a single data format and a single resolution in calibrating the chip's capabilities based on the second type of raw data are resolved, which helps to scientifically and comprehensively evaluate the chip under test.

[0029] In a specific embodiment, the sixth, seventh, eighth, ninth, tenth, eleventh, and twelfth data formats are mp4, mov, avi, wmv, mkv, h264, and h265, respectively.

[0030] Further, a second data processing operation is performed on the third type of raw data to obtain third type of test data, including: performing a second format conversion operation on the third type of raw data to obtain third type of intermediate data, wherein the second format conversion operation is used to convert the format of the third type of raw data according to a second data format set; and performing a second resolution adjustment operation on the third type of intermediate data to obtain third type of test data, wherein the second resolution adjustment operation is used to adjust the resolution of the third type of intermediate data according to a preset data resolution set, and the third type of test data includes any combination of each data format in the second data format set and each data resolution in the preset data resolution set.

[0031] Further, based on the first type of test data, the second type of test data, and the third type of test data, a visual test data set is obtained, including: classifying the first type of test data, the second type of test data, and the third type of test data according to a preset data resolution set to obtain a first-level test data set; under the first-level test data set, reclassifying the test data in the first-level test data set according to a preset data type set to obtain a second-level test data set, wherein the preset data type set includes at least the first data type, the second data type, and the third data type; under the second-level test data set, reclassifying the test data in the second-level test data set according to a preset data format set to obtain a third-level test data set, wherein the preset data format set includes at least the first data format set and the second data format set; and constructing a hierarchical visual test data set based on the first-level test data set, the second-level test data set, and the third-level test data set.

[0032] Specifically, the final form of the visual test dataset is a rigorously structured and comprehensively covered tree-like dataset. This ensures that during testing, the chip under test can "traverse and test all data formats under a fixed resolution and fixed data type," or "traverse and test any combination of data types and formats under a fixed resolution," thus achieving automated, systematic, and comprehensive testing.

[0033] In a specific embodiment, the first-level test data set includes first-type test data, second-type test data, and third-type test data categorized according to a preset data resolution set, for example, first-level test data with a resolution of 720p. The second-level test data set includes first-level test data recategorized according to a preset data type set, for example, second-level test data with a resolution of 720p and a data type of the first data type. The third-level test data set includes second-level test data recategorized according to a preset data format set, for example, third-level test data with a resolution of 720p, a data type of the first data type, and a data format of the first data format.

[0034] Furthermore, such as Figure 2 As shown, the chip under test is tested according to a preset test model set, a preset test order, and a visual test data set to obtain an output result set, including: Model traversal step: Based on the current test data in the visual test data set, the preset test model set loaded on the chip under test is traversed and executed; Format switching step: In response to the completion of the test of the current test data, test data of another data format under the current data resolution and data type in the visual test data set is used as the current test data, and the model traversal step and format switching step are executed in sequence; Type switching step: In response to the completion of the test of each data format under the current data resolution and data type, test data of another data type under the current data resolution in the visual test data set is used as the current test data, and the model traversal step, format switching step, and type switching step are executed in sequence; Resolution switching step: In response to the completion of the test of each data type under the current data resolution, test data of another data resolution in the visual test data set is used as the current test data, and the model traversal step, format switching step, type switching step, and resolution switching step are executed in sequence until the test of each data resolution in the visual test data set is completed.

[0035] Specifically, the chip under test is tested according to a preset set of test models, a preset test order, and a set of visual test data. This is the key to ensuring the comprehensiveness of the test and the comparability of the results.

[0036] Furthermore, based on the current test data in the visual test data set, the preset test model set loaded on the chip under test is traversed and executed, including: Model loading step: Loading the test model pointed to by the current model pointer in the preset test model set into the memory of the chip under test; Model execution step: Executing the test model pointed to by the current model pointer loaded on the chip under test according to the current test data; Model removal step: Removing the test model pointed to by the current model pointer from the memory of the chip under test in response to the completion of the execution of the test model pointed to by the current model pointer; Model update step: Updating the current model pointer to point to another test model in the preset test model set, and repeating the model loading step, model execution step, model removal step and model update step in sequence until each test model in the preset test model set has been executed.

[0037] Further, determining the set of model weight values ​​corresponding to the preset test model set includes: determining the number of neural network layers corresponding to each test model in the preset test model set, and calculating the total number of neural network layers in the preset test model set based on the number of neural network layers corresponding to each test model; calculating the ratio of the number of neural network layers corresponding to each test model to the total number of neural network layers to obtain the model weight value corresponding to each test model; obtaining the set of model weight values ​​based on the model weight values ​​corresponding to each test model; and obtaining the visual test results of the chip under test based on the set of model weight values ​​and the output result set, including: a format-level evaluation step: for each test data in the visual test data set, determining the corresponding model weight values ​​in the output result set. The test results are calculated based on multiple output results and the model weight set. For each test data point, a format-level test result is obtained, where each output result corresponds one-to-one with a test model in the preset test model library. The resolution-level evaluation step involves aggregating multiple format-level test results for each data resolution in the visual test data set, and obtaining the corresponding resolution-level test result based on these results. The visual test results for the chip under test include at least the format-level test results for each data format in the preset data format set and the resolution-level test results for each data resolution in the preset data resolution set.

[0038] Specifically, a two-level computational model combining "weight allocation based on model complexity" and "weighted summation with arithmetic average" is adopted to improve the accuracy of visual test results. The visual test results of the chip under test include at least the format-level test results corresponding to each data format in the preset data format set and the resolution-level test results corresponding to each data resolution in the preset data resolution set, enhancing the comprehensiveness of the evaluation of the visual data reasoning ability of the chip under test.

[0039] In a specific embodiment, W X =L X / (L R +L V +L Y +L F +L U +L S Image data inference capabilities: 720P FPS; 1080p FPS; 4K FPS; 8K FPS. Video data inference capabilities: 720P FPS; 1080p FPS; 4K FPS; 8K FPS. Video stream data inference capabilities: 720P FPS; 1080p FPS; 4K FPS; 8K FPS, totaling 12 data points. FPS I =FPS R *W R +FPS V *W V +FPS Y *W Y +FPS F *W F +FPS U *W U +FPS S *W S Where W represents the weight values ​​of the test model, L represents the number of neural network layers in the corresponding test model, and FPS I This indicates the chip under test's overall visual data processing capability for test data of the current resolution and target format. Except for X, the subscript letters are the first letter of the test model name. The FPS (frames per second) of the output results for the test data of that target format across six models is used to represent this capability. X With the corresponding model weight value W X We perform weighted summation to obtain the comprehensive visual data processing capability of the test data in the target format. FPS 总 =[sum(FPS 图片 )+sum(FPS) 视频 ) + sum(FPS 视频流 )] / 12, FPS 总 This is used to represent the chip under test's comprehensive visual data processing capabilities for image test data, video test data, and video stream test data at the current resolution.

[0040] Furthermore, such as Figure 3As shown, the monitoring result set includes at least the load value of the chip under test and the processor load value. Based on the monitoring result set, the validity of the output result set is determined, including: in response to the load value of the chip under test being less than a first preset value, determining that the monitoring result set indicates an abnormality in the testing process of the chip under test, and determining that the output result set is invalid based on the monitoring result set; in response to the load value of the chip under test being equal to the first preset value, comparing the processor load value with a preset processor load threshold; in response to the processor load value being greater than the preset processor load threshold, determining that the monitoring result set indicates an abnormality in the testing process of the chip under test, and determining that the output result set is invalid based on the monitoring result set.

[0041] In a specific embodiment, the first preset value can be set to 40%. The monitoring result set includes at least: the execution volume N (number) for each type of data, the pure inference execution time T (time), and the inference capability FPS (frames per second). The inspection result information mainly includes: chip load, chip memory utilization, processor load, memory utilization, chip bandwidth, hard disk size, system working status, and log error information. During chip inference, if the chip load is not full, it indicates that its capabilities are not fully utilized, and the model output result is marked as invalid and needs to be retested; if the chip working status is normal, but the processor or memory load is very high (e.g., exceeding 40%) or the system load changes rapidly when executing the same model, it indicates that there is a problem with the test run, the data is marked as suspicious, and manual analysis is required to determine its validity; if the program is interrupted, data cannot be output, or results cannot be displayed, it indicates that there is a problem with the test environment, the model output result needs to be marked as invalid, and the test needs to be restarted.

[0042] Furthermore, after constructing a hierarchical visual test dataset, the chip's visual data reasoning capability testing method also includes: traversing the hierarchical structure of the visual test dataset and determining whether the first-level test dataset includes each data resolution in the preset data resolution set; in response to the first-level test dataset lacking at least one data resolution in the preset data resolution set, determining that the integrity verification of the visual test dataset has failed, and completing the hierarchical structure of the visual test dataset based on a data orchestration mechanism; in response to the first-level test dataset including each data resolution in the preset data resolution set, determining whether the second-level test dataset includes each data type in the preset data type set; in response to the second-level test dataset lacking at least one data type in the preset data type set... The system determines that the integrity verification of the visual test data set has failed and completes the hierarchical structure of the visual test data set based on the data orchestration mechanism. In response to the second-level test data set including each data type from the preset data type set, it determines whether the third-level test data set includes each data format from the preset data format set. In response to the third-level test data set lacking at least one data format from the preset data format set, the system determines that the integrity verification of the visual test data set has failed and completes the hierarchical structure of the visual test data set based on the data orchestration mechanism. In response to the third-level test data set including each data format from the preset data format set, the system determines that the integrity verification of the visual test data set has passed, and the chip under test is tested according to the preset test model set, preset test order, and visual test data set.

[0043] Specifically, by utilizing the tree structure of the visual test dataset, it is possible to quickly determine whether test data is missing and what the specific resolution, data type, and data format of the missing test data are, thereby accurately evaluating the visual data reasoning capability of the chip under test.

[0044] It should be understood that, although Figure 1 , Figure 2 , Figure 3 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 , Figure 2 , Figure 3At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0045] In the second embodiment, as Figure 4 As shown, a chip visual data inference capability testing system is provided. The chip visual data inference capability testing system includes: a data orchestration module, which receives a set of raw visual data, wherein the raw visual data set includes at least a first type of raw data, a second type of raw data, and a third type of raw data; the data orchestration module is also used to process the first type of raw data, the second type of raw data, and the third type of raw data based on a data orchestration mechanism to obtain a visual test data set; an inference execution module, which tests the chip under test according to a preset test model set, a preset test order, and the visual test data set to obtain an output result set, and monitors the testing process of the chip under test to obtain a monitoring result set; an inference judgment module, which determines the validity of the output result set based on the monitoring result set; the inference execution module is also used to retest the chip under test in response to an invalid output result set; and a result generation module, which determines the model weight value set corresponding to the preset test model set in response to a valid output result set, and obtains the visual test result of the chip under test based on the model weight value set and the output result set.

[0046] Specific limitations regarding the chip's visual data inference capability testing system can be found in the limitations of the chip's visual data inference capability testing method described above, and will not be repeated here. Each module in the aforementioned chip's visual data inference capability testing system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the electronic device, or stored in software in the memory of the electronic device, so that the processor can call and execute the corresponding operations of each module.

[0047] In a third embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: receiving a set of raw visual data, wherein the raw visual data set includes at least a first type of raw data, a second type of raw data, and a third type of raw data; processing the first type of raw data, the second type of raw data, and the third type of raw data based on a data orchestration mechanism to obtain a set of visual test data; testing the chip under test according to a preset test model set, a preset test order, and the set of visual test data to obtain a set of output results, while simultaneously monitoring the testing process of the chip under test to obtain a set of monitoring results; determining the validity of the set of output results based on the set of monitoring results; retesting the chip under test in response to an invalid set of output results; and determining a set of model weight values ​​corresponding to the preset test model set in response to a valid set of output results, and obtaining the visual test results of the chip under test based on the set of model weight values ​​and the set of output results.

[0048] When the program instructions are read and executed by one or more processors, they can also perform operations corresponding to the steps in the above method embodiments, as described above, and will not be repeated here. Reference Figure 5 This exemplifies the architecture of an electronic device, which may include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, and memory 520 can communicate with each other via a communication bus 530.

[0049] The processor 510 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0050] The memory 520 can be implemented as a read-only memory (ROM), random access memory (RAM), static storage device, dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the electronic device 500, and a basic input / output system (BIOS) 522 for controlling the low-level operations of the electronic device 500. Additionally, it can store a web browser 523, data storage management 524, and an icon / font processing system 525. The icon / font processing system 525 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 520 and executed by the processor 510.

[0051] Input / output interface 513 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0052] Network interface 514 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0053] Bus 530 includes a pathway for transmitting information between various components of the device, such as processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, and memory 520.

[0054] In addition, the electronic device 500 can also obtain information on specific acquisition conditions from the virtual resource object acquisition condition information database (not shown in the figure) for condition judgment.

[0055] It should be noted that although the above-described electronic device 500 only shows a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, a memory 520, and a bus 530, in specific implementations, the electronic device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0056] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause an electronic device (which may be a personal computer, cloud server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0057] In a fourth embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: receiving a set of raw visual data, wherein the raw visual data set includes at least a first type of raw data, a second type of raw data, and a third type of raw data; processing the first type of raw data, the second type of raw data, and the third type of raw data based on a data orchestration mechanism to obtain a set of visual test data; testing the chip under test according to a preset test model set, a preset test order, and the set of visual test data to obtain a set of output results, while simultaneously monitoring the testing process of the chip under test to obtain a set of monitoring results; determining the validity of the set of output results based on the set of monitoring results; retesting the chip under test in response to an invalid set of output results; and determining the set of model weight values ​​corresponding to the preset test model set in response to a valid set of output results, and obtaining the visual test results of the chip under test based on the set of model weight values ​​and the set of output results.

[0058] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0059] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A method for testing the visual data reasoning capability of a chip, characterized in that, The method includes: Receive a set of raw visual data, wherein the set of raw visual data includes at least a first type of raw data, a second type of raw data, and a third type of raw data; Based on the data orchestration mechanism, the first type of raw data, the second type of raw data, and the third type of raw data are processed to obtain a visual test data set; The chip under test is tested according to the preset test model set, the preset test order and the visual test data set to obtain the output result set. At the same time, the test process of the chip under test is monitored to obtain the monitoring result set. Based on the monitoring result set, determine the validity of the output result set; If the output result set is invalid, the chip under test is retested; In response to the validity of the output result set, a set of model weight values ​​corresponding to the preset test model set is determined, and the visual test result of the chip under test is obtained based on the set of model weight values ​​and the output result set. The monitoring result set includes at least the load value of the chip under test and the processor load value. Determining the validity of the output result set based on the monitoring result set includes: In response to the load value of the chip under test being less than a first preset value, it is determined that the monitoring result set indicates an abnormality in the testing process of the chip under test, and the output result set is determined to be invalid based on the monitoring result set; In response to the test chip load value being equal to the first preset value, the processor load value is compared with a preset processor load threshold; In response to the processor load value being greater than the preset processor load threshold, it is determined that the monitoring result set indicates an abnormality in the testing process of the chip under test, and the output result set is determined to be invalid based on the monitoring result set.

2. The method according to claim 1, characterized in that, The data orchestration mechanism is used to process the first type of raw data, the second type of raw data, and the third type of raw data to obtain a visual test data set, including: Determine the data types of the first type of raw data, the second type of raw data, and the third type of raw data, wherein the first type of raw data belongs to the first data type, the second type of raw data belongs to the second data type, and the third type of raw data belongs to the third data type. Based on the mapping relationship between the data type and the data processing operation, determine the first data processing operation corresponding to the first data type, and perform the first data processing operation on the first type of raw data to obtain the first type of test data; Based on the mapping relationship between the data type and the data processing operation, determine the second data processing operation corresponding to the second data type, and perform the second data processing operation on the original data of the second type to obtain the second type of test data; Based on the mapping relationship between the data type and the data processing operation, determine the second data processing operation corresponding to the third data type, and perform the second data processing operation on the third type of raw data to obtain the third type of test data; The visual test data set is obtained based on the first type of test data, the second type of test data, and the third type of test data.

3. The method according to claim 2, characterized in that, The step of performing the first data processing operation on the first type of raw data to obtain the first type of test data includes: Perform a first format conversion operation on the first type of raw data to obtain first type of intermediate data, wherein the first format conversion operation is used to convert the format of the first type of raw data according to a first data format set, and the first data format set includes at least a first data format, a second data format, a third data format, a fourth data format, and a fifth data format; A first resolution adjustment operation is performed on the first type of intermediate data to obtain the first type of test data. The first resolution adjustment operation is used to adjust the resolution of the first type of intermediate data according to a preset data resolution set. The preset data resolution set includes at least a first data resolution, a second data resolution, a third data resolution, and a fourth data resolution. The first type of test data includes any combination of each data format in the first data format set and each data resolution in the preset data resolution set.

4. The method according to claim 3, characterized in that, The step of performing the second data processing operation on the second type of raw data to obtain the second type of test data includes: Perform a second format conversion operation on the second type of raw data to obtain second type of intermediate data. The second format conversion operation is used to convert the format of the second type of raw data according to a second data format set, which includes at least a sixth data format, a seventh data format, an eighth data format, a ninth data format, a tenth data format, an eleventh data format, and a twelfth data format. A second resolution adjustment operation is performed on the second type of intermediate data to obtain the second type of test data. The second resolution adjustment operation is used to adjust the resolution of the second type of intermediate data according to the preset data resolution set. The second type of test data includes any combination of each data format in the second data format set and each data resolution in the preset data resolution set.

5. The method according to claim 4, characterized in that, The step of obtaining the visual test data set based on the first type of test data, the second type of test data, and the third type of test data includes: The first type of test data, the second type of test data, and the third type of test data are classified according to the preset data resolution set to obtain a first-level test data set. Under the first-level test data set, the test data in the first-level test data set is reclassified according to a preset data type set to obtain a second-level test data set, wherein the preset data type set includes at least the first data type, the second data type, and the third data type; Under the second-level test data set, the test data in the second-level test data set is reclassified according to the preset data format set to obtain the third-level test data set, wherein the preset data format set includes at least the first data format set and the second data format set; Based on the first-level test data set, the second-level test data set, and the third-level test data set, a hierarchical visual test data set is constructed.

6. The method according to claim 5, characterized in that, The step of testing the chip under test according to a preset test model set, a preset test order, and the visual test data set to obtain an output result set includes: Model traversal steps: Based on the current test data in the visual test data set, traverse and execute the preset test model set that has been loaded onto the chip under test; Format switching step: In response to the completion of the test of the current test data, test data of another data format under the current data resolution and current data type in the visual test data set is taken as the current test data, and the model traversal step and the format switching step are executed repeatedly in sequence; Type switching step: In response to completing the test of test data for each data format under the current data resolution and current data type, test data of another data type under the current data resolution in the visual test data set is taken as the current test data, and the model traversal step, the format switching step and the type switching step are executed in sequence. Resolution switching step: In response to completing the test of test data of each data type at the current data resolution, test data of another data resolution in the visual test data set is taken as the current test data, and the model traversal step, the format switching step, the type switching step and the resolution switching step are executed in sequence until the test of test data of each data resolution in the visual test data set is completed.

7. The method according to claim 6, characterized in that, The step of traversing and executing the preset test model set loaded on the chip under test based on the current test data in the visual test data set includes: Model loading step: Load the test model pointed to by the current model pointer in the preset test model set into the memory of the chip under test; Model execution steps: Based on the current test data, execute the test model pointed to by the current model pointer that has been loaded onto the chip under test; Model removal step: In response to the completion of the test model pointed to by the current model pointer, the test model pointed to by the current model pointer is removed from the memory of the chip under test; Model update steps: Update the current model pointer to point to another test model in the preset test model set, and repeat the model loading step, the model execution step, the model removal step, and the model update step in sequence until each test model in the preset test model set has been executed.

8. The method according to claim 5, characterized in that, Determining the set of model weight values ​​corresponding to the preset test model set includes: Determine the number of neural network layers corresponding to each test model in the preset test model set, and calculate the total number of neural network layers in the preset test model set based on the number of neural network layers corresponding to each test model. Calculate the ratio of the number of neural network layers corresponding to each test model to the total number of neural network layers to obtain the model weight value corresponding to each test model; Based on the model weight value corresponding to each test model, the set of model weight values ​​is obtained; The step of obtaining the visual test result of the chip under test based on the model weight value set and the output result set includes: Format hierarchy evaluation steps: For each test data in the visual test data set, determine the corresponding multiple output results in the output result set, and obtain the format hierarchy test result corresponding to each test data according to the model weight value set and the multiple output results, wherein the multiple output results correspond one-to-one with the multiple test models in the preset test model set; Resolution level evaluation steps: For each data resolution in the visual test dataset, aggregate the test results of multiple format levels under each data resolution, and obtain the resolution level test result corresponding to each data resolution based on the multiple format level test results; The visual test results of the chip under test include at least the format level test results corresponding to each data format in the preset data format set and the resolution level test results corresponding to each data resolution in the preset data resolution set.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the visual data reasoning capability testing method for the chip as claimed in any one of claims 1 to 8 when executing the computer program.

Images