Chip verification analysis method and device, electronic equipment and computer storage medium

Abstract

The present disclosure provides a chip verification analysis method and device, electronic equipment and computer storage medium, belonging to the chip verification technical field. The chip verification analysis method comprises: obtaining a message attribute set and an expected occurrence parameter corresponding to the message attribute set in the chip verification process, wherein the message attribute set comprises at least one message attribute; obtaining the actual occurrence frequency of the message attribute set in the chip verification process; and generating an analysis report according to the expected occurrence parameter and the actual occurrence frequency of the message attribute set. The scheme provided by the present disclosure can realize automatic quantitative verification of various feature messages in chip verification, and does not need to separately write coverage model code for various messages, nor does it need to manually check verification logs line by line and statistically count coverage data by type, thereby improving the execution efficiency and result accuracy of chip verification analysis, and ensuring the comprehensiveness and reliability of chip verification.

Description

Technical Field

[0001] This disclosure relates to the field of chip verification technology, and in particular to a chip verification analysis method, apparatus, electronic device, and computer storage medium. Background Technology

[0002] With the development of network communication technology, the types of communication protocols that network chips are compatible with are constantly increasing, and each type of communication protocol integrates a large number of configuration attributes, data fields, and differentiated operating modes. During the chip verification phase, verification engineers need to ensure verification quality through test case design, regression testing, coverage analysis, and other methods.

[0003] Currently, verification personnel typically develop corresponding functional coverage models based on preset test plans and for various combinations of message attributes in different protocols in order to collect and analyze the coverage of chip verification.

[0004] However, developing separate functional coverage models for different combinations of message attributes not only involves a large amount of code writing and debugging work, but also results in long model development cycles, high iteration and maintenance costs, and makes it difficult to efficiently adapt to verification scenarios with a wide variety of protocol types and complex attribute fields. Summary of the Invention

[0005] In view of the above, embodiments of this disclosure provide a chip verification and analysis method, apparatus, electronic device, and computer storage medium to at least partially solve the above problems.

[0006] According to a first aspect of the present disclosure, a chip verification analysis method is provided, comprising: acquiring a message attribute set and an expected occurrence parameter corresponding to the message attribute set during chip verification, wherein the message attribute set includes at least one message attribute; acquiring the actual occurrence number of the message attribute set during chip verification; and generating an analysis report based on the expected occurrence parameter and the actual occurrence number of the message attribute set.

[0007] In one possible implementation, obtaining the actual occurrence count of the message attribute set during chip verification includes: obtaining the message attribute set corresponding to a message appearing at a preset node during chip verification, wherein the preset node includes at least one of an excitation input node, a message output node, and a message discard node; indexing the message attribute set corresponding to the message appearing at the preset node in a dynamic data structure, wherein the dynamic data structure is used to store the mapping relationship between the message attribute set and the actual occurrence count corresponding to the message attribute set; if the message attribute set corresponding to the message appearing at the preset node is found in the dynamic data structure, the actual occurrence count of the message attribute set corresponding to the message appearing at the preset node is incremented by 1.

[0008] In one possible implementation, generating an analysis report based on the expected occurrence parameter and the actual occurrence count of the message attribute set includes: obtaining the actual occurrence count corresponding to the message attribute set in the dynamic data structure; generating an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set, wherein the expected occurrence parameter includes an expected occurrence threshold range and / or an expected occurrence percentage threshold range.

[0009] In one possible implementation, the expected occurrence parameter includes an expected occurrence threshold range. Generating an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set includes: generating the analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence threshold range corresponding to the message attribute set; if the actual occurrence count corresponding to the message attribute set does not belong to the expected occurrence threshold range corresponding to the message attribute set, then the occurrence count of the message attribute set is marked as abnormal in the analysis report. Specifically, if the actual occurrence count corresponding to the message attribute set is 0, and the expected occurrence threshold range corresponding to the message attribute set does not include 0, then the occurrence count of the message attribute set is marked as abnormal and not occurring in the analysis report.

[0010] In one possible implementation, the expected occurrence parameter includes an expected occurrence percentage threshold range. Generating an analysis report based on the actual occurrence count corresponding to each message attribute set and the expected occurrence parameter corresponding to that message attribute set includes: calculating the sum of the actual occurrence counts for all message attribute sets based on the actual occurrence counts for each message attribute set; calculating the actual occurrence percentage of each message attribute set based on the sum of the actual occurrence counts for each message attribute set and the expected occurrence percentage threshold range corresponding to that message attribute set; generating the analysis report based on the actual occurrence percentage of the message attribute set and the expected occurrence percentage threshold range corresponding to that message attribute set; if the actual occurrence percentage of the message attribute set does not belong to the expected occurrence percentage threshold range corresponding to that message attribute set, then the message attribute set is marked as having an abnormal occurrence percentage in the analysis report. Specifically, if the actual occurrence percentage of the message attribute set is 0, and the expected occurrence percentage threshold range corresponding to that message attribute set does not include 0, then the message attribute set is marked as having an abnormal occurrence percentage and not occurring in the analysis report.

[0011] In one possible implementation, the method further includes: if the message attribute set is abnormal during multiple chip verification processes, adjusting the incentive constraints of the test cases corresponding to the message attribute set, and / or adding test cases, wherein the abnormality of the message attribute set during multiple chip verification processes includes: the fluctuation range of the actual occurrence ratio of the message attribute set during multiple chip verification processes is greater than a preset fluctuation threshold, and / or the actual occurrence ratio of the message attribute set is 0 in a preset number of chip verification rounds, and the expected occurrence ratio threshold range corresponding to the message attribute set does not include 0.

[0012] In one possible implementation, the method further includes generating a visual display result based on the analysis report of multiple chip verification processes.

[0013] According to a second aspect of the present disclosure, a chip verification analysis apparatus is provided, comprising: a first acquisition unit, configured to acquire a message attribute set and an expected occurrence parameter corresponding to the message attribute set during chip verification, wherein the message attribute set includes at least one message attribute; a second acquisition unit, configured to acquire the actual occurrence number of the message attribute set during chip verification; and an analysis unit, configured to generate an analysis report based on the expected occurrence parameter and the actual occurrence number of the message attribute set.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to execute the chip verification and analysis method of the first aspect.

[0015] According to a fourth aspect of the present disclosure, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the chip verification and analysis method of the first aspect.

[0016] According to the chip verification analysis method provided in this disclosure, by obtaining a pre-configured set of message attributes to be monitored and their corresponding expected occurrence parameters, the actual occurrence frequency of the message attribute set is obtained during the chip verification process. Based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set, an analysis report is generated. This method can achieve automated quantitative verification of various feature messages in chip verification, and eliminates the need to write coverage model code separately for each type of message, as well as the need for manual line-by-line verification log checking and category-by-category coverage data statistics. This can improve the execution efficiency and accuracy of chip verification analysis, and ensure the comprehensiveness and reliability of chip verification. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of a chip verification and analysis method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram illustrating a visualization of the results provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of a chip verification and analysis device provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art should fall within the protection scope of this disclosure.

[0020] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0022] As mentioned earlier, with the development of network communication technology, the types of communication protocols that network chips are compatible with are constantly increasing, and each type of communication protocol integrates a large number of configuration attributes, data fields, and differentiated operating modes. During the chip verification phase, verification engineers need to ensure verification quality through test case design, regression testing, and coverage analysis. Currently, verification personnel typically develop corresponding functional coverage models based on preset test plans for various combinations of message attributes in different protocols to collect and analyze chip verification coverage. However, developing independent functional coverage models for different combinations of message attributes not only involves a large workload of code writing and debugging, but also results in long model development cycles, high iterative maintenance costs, and difficulty in efficiently adapting to verification scenarios with numerous protocol types and complex attribute fields.

[0023] This disclosure provides a chip verification analysis method. By acquiring a pre-configured set of message attributes to be monitored and their corresponding expected occurrence parameters, the method obtains the actual occurrence frequency of the message attribute set during chip verification. Based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set, an analysis report is generated. This method can achieve automated quantitative verification of various characteristic messages in chip verification. Furthermore, it eliminates the need to write coverage model code separately for each type of message, and also eliminates the need for manual line-by-line verification log checking and category-by-category coverage data statistics. This improves the execution efficiency and accuracy of chip verification analysis, ensuring the comprehensiveness and reliability of chip verification.

[0024] The chip verification and analysis method provided in this disclosure is illustrated below through examples.

[0025] First, some of the nouns or terms that appear in the description of the embodiments of this disclosure are explained as follows.

[0026] Chip verification: The verification stage in the chip design process, located after the design is completed and before tape-out. Through simulation verification, prototype testing and other means, the actual operation scenarios of the chip are simulated to verify whether the chip's functional logic, performance indicators and other aspects meet the design specifications. This is used to identify design defects in advance and ensure the reliability of the final mass production of the chip.

[0027] Stimulus: Test signals or test data input to the chip under test in the chip verification environment. These signals are used to simulate the input scenarios in the chip's actual working state and drive the chip's internal functional logic to perform corresponding processing operations.

[0028] A packet is a structured data unit encapsulated according to a specific communication protocol specification. It includes a header, payload, and other formatted fields, and serves as a data carrier during data transmission and forwarding between chip interfaces and internal data paths. In chip verification, stimuli are often constructed in the form of packets. By generating packets that conform to the protocol specification as input stimuli, real network interaction scenarios are simulated, thereby driving the chip interface or internal logic to perform the expected operations.

[0029] Functional coverage: A quantitative indicator used in the field of chip verification to evaluate the completeness of verification. It determines whether the coverage of verification tests meets the standard by statistically analyzing the degree to which various preset functional scenarios and attribute combinations are actually covered during the chip verification process.

[0030] Figure 1 This is a flowchart of a chip verification and analysis method provided in this embodiment. This chip verification and analysis method can be applied to verification scenarios for various chips with message transmission, forwarding, or processing functions, such as network chips, interface chips, and data processing chips, for automated quantitative analysis of chip verification results. Figure 1 As shown, the chip verification and analysis method includes the following steps 101 to 103: Step 101: Obtain the message attribute set and the expected parameters of the message attribute set during the chip verification process.

[0031] Message attributes are configurable dimensional information used to characterize message features, such as message protocol type, message length, source / destination address, and service tag. A message attribute set consists of at least one message attribute and is used to define a class of messages with uniform characteristics, serving as the target unit for monitoring and statistics during chip verification. Each message attribute set can be flexibly configured according to the needs of different verification scenarios. A message attribute set can include only a single message attribute to define a coarse-grained statistical dimension, or it can include multiple message attributes to define fine-grained feature groupings, thus adapting to different granularity analysis needs.

[0032] The expected occurrence parameters are quantitative verification benchmarks pre-defined according to the chip design specifications and the test objectives of the verification cases, corresponding to each set of message attributes. They are used to verify whether the occurrence of the corresponding characteristic messages meets the design expectations. The expected occurrence parameters can be flexibly configured according to the needs of different verification scenarios to adapt to various verification objectives such as functional verification, stress testing, and boundary scenario testing.

[0033] The aforementioned message attribute set and its corresponding expected occurrence parameters can be stored in a structured configuration table. The configuration process requires no additional code development, resulting in low implementation costs. Furthermore, statistical objects and verification rules can be flexibly added, deleted, and adjusted according to the needs of different verification scenarios. Verification personnel do not need to write separate coverage model code for each type of message attribute set; they only need to complete the configuration definition of the message attribute set and its corresponding expected parameters to determine the monitoring and statistical objects and verification standards. This decouples statistical rules from execution logic at the entry point, providing a basis for subsequent automated statistical analysis.

[0034] Step 102: During the chip verification process, obtain the actual number of occurrences of the message attribute set.

[0035] During the execution of chip verification test cases, the messages processed by the chip are monitored and their features are analyzed in real time. Messages that match preset message attribute sets are identified, and the number of times each matching message attribute set actually appears during the verification process is accumulated. The actual number of occurrences can be converted into formatted statistical data by a statistical component and recorded in the simulation log.

[0036] It should be noted that in this step, the verification environment can be started in the usual way for simulation or regression testing. There is no need to enable the additional code coverage collection option. The statistical process is executed synchronously with the main simulation process, which will not increase the additional simulation running overhead. Moreover, it can truly and objectively reflect the actual occurrence of the corresponding feature messages during the chip verification process.

[0037] Step 103: Generate an analysis report based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set.

[0038] After a single test case or a complete regression verification is completed, the actual occurrence count of each message attribute set can be obtained. The actual occurrence count of each message attribute set is then compared and analyzed with the pre-configured expected occurrence parameters to determine whether the actual occurrence pattern of various messages meets the preset design requirements.

[0039] The system identifies message attribute sets that do not actually appear or whose frequency of appearance deviates from the expected range. Based on the comparison results, it generates a lightweight structured analysis report, which may include actual statistical data for each message attribute set, comparison results between expected and actual values, and alerts for abnormal situations.

[0040] This step, through automated report generation, can replace the traditional manual work mode of checking verification logs line by line and calculating coverage data category by category, reducing the repetitive workload of verification personnel, and also reducing the omissions and errors that are prone to occur in manual statistics.

[0041] In this embodiment of the disclosure, by obtaining a pre-configured set of message attributes to be monitored and their corresponding expected occurrence parameters, the actual occurrence frequency of the message attribute set is obtained during the chip verification process. Based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set, an analysis report is generated. This enables automated quantitative verification of various feature messages in chip verification. Furthermore, it eliminates the need to write coverage model code separately for each type of message, and also eliminates the need for manual line-by-line verification log checking and category-by-category statistical coverage data. This improves the execution efficiency and accuracy of chip verification analysis, ensuring the comprehensiveness and reliability of chip verification.

[0042] In one possible implementation, when obtaining the actual occurrence count of a message attribute set, the message attribute set corresponding to a message appearing at a preset node can be obtained during chip verification. The preset node includes at least one of an excitation input node, a message output node, and a message discard node. The message attribute set corresponding to a message appearing at the preset node is indexed in a dynamic data structure, which stores the mapping relationship between the message attribute set and the actual occurrence count of the corresponding message attribute set. If the message attribute set corresponding to a message appearing at the preset node is found in the dynamic data structure, the actual occurrence count of the message attribute set corresponding to the message appearing at the preset node is incremented by 1.

[0043] In the chip verification environment, pre-determine preset nodes for packet monitoring. These preset nodes can be one or more of the following: stimulus input nodes, packet output nodes, and packet discarding nodes. The stimulus input node corresponds to the injection port where packets enter the chip, and is used to statistically analyze the actual injection of various characteristic packets in the verification stimulus. The packet output node corresponds to the outflow port of the packets after chip processing, and is used to statistically analyze the characteristic distribution of packets normally forwarded by the chip. The packet discarding node corresponds to the processing node where the chip discards packets due to verification anomalies, route misses, buffer congestion, etc., and is used to statistically analyze the discarding of various types of packets under abnormal scenarios.

[0044] In one example, the chip verification environment adopts the Universal Verification Methodology (UVM) verification environment. The stimulus input node corresponds to the transaction sending execution position in the UVM verification environment, the message output node corresponds to the result decision execution position in the UVM verification environment, and the message drop node corresponds to the execution position of message drop statistics logic in the UVM verification environment.

[0045] The verification environment can embed lightweight statistical components at the aforementioned preset nodes. During chip verification, when a packet is received at a preset node, the statistical component performs feature parsing on the packet, obtains the corresponding packet attribute set, and uses the parsed packet attribute set as an index key to perform matching indexing in a dynamic data structure. The dynamic data structure is a storage unit deployed in the verification environment's runtime memory, storing the packet attribute set and its corresponding actual occurrence count in a key-value mapping relationship. It can be implemented using hash tables, associative arrays, dictionaries, etc.

[0046] If a matching message attribute set entry is found in the dynamic data structure, the actual occurrence count of that entry is incremented by 1. If no corresponding entry is found, the corresponding statistical entry can be ignored or added according to a pre-configured strategy. This statistical process is executed synchronously with the sending of stimuli and the processing of messages. The count is updated immediately each time a message passes through a node, without waiting for the entire simulation to finish, and the statistical results are real-time.

[0047] At the designated node of the chip verification simulation or after the execution of a single test case, the statistics component can export the counting results stored in the dynamic data structure according to the set structured format and write them to the simulation log, forming a traceable statistical data record, providing a data source for subsequent comparative analysis and report generation.

[0048] It should be noted that although existing Electronic Design Automation (EDA) tools also have code coverage analysis options, EDA tools require enabling dedicated compilation and execution options for coverage acquisition. The tools add acquisition logic to various parts of the design or verification code, recording the execution status of code and logic branches segment by segment, which usually introduces significant simulation overhead. This solution, however, performs message feature frequency statistics by obtaining the message attribute set corresponding to messages appearing at preset nodes. It does not rely on the native coverage acquisition framework of EDA tools, does not require enabling additional coverage-related compilation and execution options, and does not modify the underlying design code. Therefore, it does not introduce the high additional overhead of tool-side coverage acquisition, only adding a very light simulation burden. It can complete the quantitative statistics of message features while basically maintaining the original simulation efficiency.

[0049] In this embodiment, by selecting at least one of the excitation input node, message output node, and message discard node as a preset node during the chip verification process, and obtaining the message attribute set corresponding to the message appearing at the preset node, and performing index matching in the dynamic data structure, if the matching is successful, the actual occurrence count of the message attribute set is incremented by 1. This enables feature statistics of multiple links in the configurable message processing link, and the statistical process is executed synchronously with the main simulation process, introducing only very low additional operating overhead. This ensures the real-time and traceability of the statistical results, providing accurate and multi-dimensional statistical data sources for subsequent comparison analysis and structured report generation, while flexibly adapting to the node monitoring and counting needs of different verification scenarios.

[0050] In one possible implementation, when generating an analysis report based on the expected occurrence parameters and the actual occurrence count of the message attribute set, the actual occurrence count corresponding to the message attribute set in the dynamic data structure can be obtained. Based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameters corresponding to the message attribute set, an analysis report is generated. The expected occurrence parameters include the expected occurrence threshold range and / or the expected occurrence percentage threshold range.

[0051] After a single test case or a complete regression verification is completed, the actual occurrence count corresponding to the message attribute set is obtained from the dynamic data structure that stores the message attribute set and its corresponding actual occurrence count in a key-value mapping relationship. Since the dynamic data structure synchronously updates the count as the message flows through preset nodes during chip verification, the statistical data has been structurally stored, eliminating the need for backtracking and secondary statistics on the original simulation logs, and can be directly used as the data source for comparison and analysis.

[0052] In practice, one can either directly read the dynamic data structure in memory during the simulation to obtain real-time statistical data, or extract the corresponding values ​​from the formatted statistical data recorded in the simulation log after the simulation ends, to adapt to different scenario requirements of online real-time analysis and offline archive analysis.

[0053] The actual occurrence counts of each extracted message attribute set are compared with the pre-configured expected occurrence parameters, and a structured analysis report is generated based on the comparison results. The expected occurrence parameters support two types of verification dimensions: one is the expected occurrence threshold range, used to verify whether the occurrence count of the corresponding message meets the design requirements from an absolute quantity perspective; the other is the expected occurrence percentage threshold range, used to verify whether the distribution percentage of the corresponding message within the statistical range meets the design requirements from a relative proportion perspective. These two verification dimensions can be selected individually or in combination according to the actual needs of the verification scenario to adapt to verification analysis objectives of different granularities.

[0054] In this embodiment, by extracting the actual occurrence count of the message attribute set from the dynamic data structure and comparing it with multi-dimensional expected parameters such as the expected occurrence threshold range and the expected occurrence percentage threshold range, an analysis report is generated. This eliminates the need for secondary parsing and statistics of the original simulation logs, improving the efficiency of analysis execution. It also supports two types of verification dimensions: absolute quantity and relative proportion, which can flexibly adapt to different verification analysis scenarios, improve the versatility and configuration flexibility of verification analysis, and thus improve the automation level and reliability of chip verification analysis results.

[0055] In one possible implementation, the expected occurrence parameter includes an expected occurrence threshold range. When generating an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set, the analysis report can be generated based on the actual occurrence count corresponding to the message attribute set and the expected occurrence threshold range corresponding to the message attribute set. If the actual occurrence count corresponding to the message attribute set does not fall within the expected occurrence threshold range corresponding to the message attribute set, the occurrence count of the message attribute set is marked as abnormal in the analysis report. Specifically, if the actual occurrence count corresponding to the message attribute set is 0, and the expected occurrence threshold range corresponding to the message attribute set does not include 0, the occurrence count of the message attribute set is marked as abnormal and not occurring in the analysis report.

[0056] The expected occurrence threshold range is a pre-defined reasonable range of occurrences for each message attribute set. It can be flexibly configured based on factors such as the test objectives of the verification test cases, the scale of incentive deployment, and the proportion of business scenarios. It is used to verify whether the occurrence of corresponding characteristic messages meets the design expectations from an absolute quantity perspective. For example, both the upper and lower limits of the occurrences can be configured simultaneously to constrain the reasonable range of occurrences, or only the lower limit can be configured to ensure the minimum verification coverage of the corresponding characteristic messages, or only the upper limit can be configured to control the maximum allowed number of abnormally discarded messages and erroneous messages.

[0057] The actual occurrence count of each message attribute set is matched against the corresponding expected occurrence threshold range. The comparison results are integrated into the analysis report to form a standardized quantitative verification result. When the actual occurrence count falls within the expected occurrence threshold range, the occurrence count of this type of message is determined to be in line with expectations, and it is marked as normal in the report. When the actual occurrence count falls outside the expected occurrence threshold range, the occurrence count of this type of message is determined to be unexpected, and it is marked as abnormal in the report.

[0058] Specifically, if the actual occurrence count of the message attribute set is 0, and the value range of its corresponding expected occurrence threshold interval does not include 0, it means that this type of feature message was not triggered at all during this verification process, and belongs to the missing item of verification coverage. It is then further marked as having an abnormal occurrence count and not appearing, forming a hierarchical abnormality prompt.

[0059] In this embodiment, by using the expected occurrence threshold range of the absolute number dimension as the verification benchmark, the actual occurrence frequency of each message attribute set is matched and verified. The abnormal situations of quantity deviation and no occurrence are classified and marked. This can not only intuitively and quantitatively verify whether the absolute occurrence scale of various messages meets the test target and adapt to the control requirements of various targeted verification scenarios, but also help verification personnel quickly distinguish the severity of abnormalities through hierarchical abnormality identification, and prioritize the location of high-priority issues with missing verification coverage. In this way, the automation level of verification analysis can be improved, while the efficiency of verification problem investigation and the interpretability of results can be improved.

[0060] In one possible implementation, the expected occurrence parameter includes an expected occurrence percentage threshold range. When generating an analysis report based on the actual occurrence count corresponding to each message attribute set and the expected occurrence parameter corresponding to that set, the sum of the actual occurrence counts for all message attribute sets can be calculated based on the actual occurrence count for each set. The actual occurrence percentage of each message attribute set is calculated based on the sum of the actual occurrence counts for all message attribute sets. An analysis report is generated based on the actual occurrence percentage of each message attribute set and the expected occurrence percentage threshold range corresponding to that set. If the actual occurrence percentage of a message attribute set does not fall within the expected occurrence percentage threshold range, the message attribute set is marked as having an abnormal occurrence percentage in the analysis report. Specifically, if the actual occurrence percentage of a message attribute set is 0, and the expected occurrence percentage threshold range does not include 0, the message attribute set is marked as having an abnormal occurrence percentage and not occurring in the analysis report.

[0061] Within the same statistical scope, the actual occurrence counts of various message attribute sets to be counted are summed to obtain the total message count for that statistical dimension. This total count serves as the baseline denominator for the percentage calculation, reflecting the overall message scale in the current verification scenario. The statistical scope can correspond to different dimensions such as a single type of preset node, a single test case, or an entire round of regression testing.

[0062] For each type of message attribute set, the actual occurrence frequency of the message is divided by the total number of messages to obtain the actual occurrence ratio of that type of message within the corresponding statistical range. This ratio is used to characterize the distribution ratio of the corresponding characteristic message in the overall message count, so as to reflect the traffic structure and distribution pattern of the messages.

[0063] The expected occurrence percentage threshold range is a pre-defined reasonable occurrence percentage range for each message attribute set. It can be flexibly configured based on factors such as the test objectives of the verification test cases, the scale of incentive deployment, and the proportion of business scenarios. It is used to verify whether the occurrence of corresponding characteristic messages meets the design expectations from a relative proportion dimension. For example, both the upper and lower limits of the proportion can be configured simultaneously to constrain the reasonable occurrence percentage range, or only the lower limit can be configured to ensure the minimum verification coverage percentage of the corresponding characteristic messages, or only the upper limit can be configured to control the maximum allowed percentage of abnormally discarded messages and erroneous messages.

[0064] The actual occurrence percentage of each message attribute set is matched against the corresponding expected occurrence percentage threshold range. The comparison results are integrated into the analysis report to form standardized quantitative verification results. When the actual occurrence percentage falls within the expected occurrence percentage threshold range, the occurrence percentage of that type of message is determined to be in line with expectations and is marked as normal in the report. When the actual occurrence percentage falls outside the expected occurrence percentage threshold range, the occurrence percentage of that type of message is determined to be unsatisfactory and is marked as abnormal in the report.

[0065] Specifically, if the actual occurrence rate of the message attribute set is 0, and the corresponding expected occurrence rate threshold range does not include 0, it means that this type of feature message was not triggered at all during this verification process. It belongs to the missing item of verification coverage and is further marked as having an abnormal occurrence rate and not appearing, forming a hierarchical abnormality prompt.

[0066] In this embodiment of the disclosure, by first counting the total number of actual occurrences of all message attribute sets, then calculating the actual occurrence percentage for each category, and finally comparing and verifying with the expected percentage threshold range, it is possible to analyze whether the traffic distribution structure of various messages during chip verification meets the design requirements. At the same time, combined with the hierarchical marking mechanism of abnormal percentages and complete absence, it can help verification personnel quickly distinguish the severity of anomalies, thereby more accurately assessing the rationality of the verification scenario coverage and improving the completeness and refinement of chip verification analysis.

[0067] In one possible implementation, if the message attribute set is abnormal during multiple chip verification processes, the incentive constraints of the test cases corresponding to the message attribute set are adjusted, and / or test cases are added. The abnormality of the message attribute set during multiple chip verification processes includes: the fluctuation range of the actual occurrence ratio of the message attribute set during multiple chip verification processes is greater than a preset fluctuation threshold, and / or the actual occurrence ratio of the message attribute set is 0 in a preset number of chip verification rounds.

[0068] The analysis reports of each message attribute set obtained after each round of chip verification are archived and stored in a historical database, forming a historical statistical data set covering multiple rounds of regression verification. Based on the actual occurrence frequency, actual occurrence percentage, and anomaly analysis results in the analysis reports, a long-term trend analysis of the verification coverage of each message attribute set is performed according to preset rules to determine whether the message attribute set exhibits persistent anomalies in multiple verification processes. Persistent anomalies in message attribute sets during multiple chip verification processes are specifically identified in the following two scenarios: The first scenario involves the actual occurrence percentage of a message attribute set fluctuating beyond a preset fluctuation threshold across multiple chip verification processes. The fluctuation range can be quantified using statistical indicators such as the range and standard deviation of the actual occurrence percentages across multiple rounds, reflecting the controllability of verification stimulus generation. The preset fluctuation threshold is a reasonable fluctuation range for the proportion of various message attribute sets, which can be flexibly configured based on the stability requirements of the verification stage and business scenario. If the actual occurrence percentage of the same message attribute set fluctuates beyond the preset threshold across multiple consecutive verification rounds, it typically reflects insufficient stability in the traffic allocation of verification stimulus, or iterative changes in chip design or the verification environment causing abnormal changes in message processing behavior, indicating an abnormality in the coverage stability of this message attribute set.

[0069] The second scenario involves a message attribute set whose actual occurrence percentage is 0 across a preset number of chip verification rounds, and the expected occurrence percentage threshold range for this message attribute set does not include 0. The preset number is a pre-defined threshold for consecutive verification rounds. If the same message attribute set consistently has an actual occurrence percentage of 0 across multiple verification rounds reaching the preset number, it indicates that the absence of this type of feature message is not an accidental omission in a single verification, but rather a continuous failure to be effectively triggered, suggesting a persistent coverage omission anomaly in this message attribute set.

[0070] When a set of message attributes is determined to exhibit persistent anomalies during multiple chip verification processes, the incentive constraints of the test cases associated with that set of message attributes should be adjusted, and / or new targeted test cases should be added. For fluctuation-type anomalies, the generation ratio of the corresponding messages can be stabilized by adjusting the incentive constraints, thus optimizing the stability of verification traffic. For anomalies with persistent zero coverage, the generation restrictions of the corresponding messages can be released by adjusting the incentive constraints, or targeted test cases can be added to fill the verification coverage gaps. Both adjustment methods can be used individually or in combination.

[0071] In this embodiment, by continuously tracking the changes in the proportion of each message attribute set in multiple rounds of verification, persistent anomalies are identified based on two types of rules: fluctuation amplitude and continuous zero coverage. The incentive constraints or supplementary test cases are then adjusted accordingly. This avoids the accidental deviation of a single verification, more objectively reflects the long-term status and trend of verification quality, and enables rapid quantitative evaluation of regression verification quality. Consequently, a closed-loop verification mechanism from quality analysis to strategy optimization can be formed, enabling continuous iteration and improvement of the chip verification solution, ensuring the sufficiency of verification coverage and the stability of verification results, and adapting to the verification quality control needs of the entire chip development cycle.

[0072] In one possible implementation, a visual representation of the results is generated based on analysis reports from multiple chip verification processes.

[0073] Obtain structured analysis reports generated from multiple rounds of chip verification processes, and extract statistical data such as the actual occurrence frequency, actual occurrence percentage, and anomaly type identifier for each message attribute set in each report. Based on this multi-dimensional statistical data set, generate corresponding visual display results, presenting the message coverage and variation patterns of chip verification in a graphical format.

[0074] In one example, for each set of message attributes, a trend chart is generated showing the actual occurrence frequency and actual occurrence percentage of each attribute in multiple rounds of chip verification, presenting the long-term trend of the verification coverage and distribution ratio of the corresponding feature messages. The trend chart can also mark the expected occurrence threshold range and / or the expected occurrence percentage threshold range to visually reflect whether the actual occurrence frequency and actual occurrence percentage of the message attribute set meet expectations.

[0075] Figure 2 This is a schematic diagram illustrating a visualization of the results provided in an embodiment of this disclosure. Figure 2 The data shows the actual occurrence percentage of message attribute sets 1-4 during the multi-day chip verification process, as well as the expected occurrence percentage threshold range (>1%).

[0076] Furthermore, the visualization results can be output through the graphical interface of the verification platform, allowing verification personnel to filter and view them according to conditions such as message attribute type, verification round range, and monitoring node type.

[0077] In this embodiment of the disclosure, based on the analysis report of multiple chip verification processes, a visual display result is generated. This can intuitively present the changing patterns of the number of times and the proportion of messages appearing during multiple rounds of verification, thereby improving the information transmission efficiency and readability of the verification results. This allows verification personnel to quickly grasp the overall trend of verification quality changes, lowers the threshold for reviewing and analyzing multiple rounds of regression results, and improves the decision-making efficiency and accuracy of verification strategy adjustments.

[0078] Figure 3 This is a schematic diagram of a chip verification and analysis device provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the chip verification and analysis device 200 includes: The first acquisition unit 201 is used to acquire a message attribute set and the expected occurrence parameters of the message attribute set during the chip verification process, wherein the message attribute set includes at least one message attribute. The second acquisition unit 202 is used to acquire the actual number of occurrences of the message attribute set during the chip verification process; Analysis unit 203 is used to generate an analysis report based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set.

[0079] In this embodiment of the disclosure, the first acquisition unit 201 can be used to execute step 101 in the above method embodiment, the second acquisition unit 202 can be used to execute step 102 in the above method embodiment, and the analysis unit 203 can be used to execute step 103 in the above method embodiment.

[0080] In one possible implementation, the second acquisition unit 202 is further configured to acquire, during the chip verification process, a set of message attributes corresponding to messages appearing at a preset node, wherein the preset node includes at least one of an excitation input node, a message output node, and a message discard node; index the set of message attributes corresponding to messages appearing at the preset node in a dynamic data structure, wherein the dynamic data structure is used to store the mapping relationship between the set of message attributes and the actual occurrence count of the message attribute set; if the set of message attributes corresponding to messages appearing at the preset node is indexed in the dynamic data structure, the actual occurrence count of the set of message attributes corresponding to messages appearing at the preset node is incremented by 1.

[0081] In one possible implementation, the analysis unit 203 is further configured to obtain the actual occurrence count corresponding to the message attribute set in the dynamic data structure; and generate an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set, wherein the expected occurrence parameter includes the expected occurrence threshold range and / or the expected occurrence percentage threshold range.

[0082] In one possible implementation, the analysis unit 203 is further configured to generate an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence threshold range corresponding to the message attribute set; if the actual occurrence count corresponding to the message attribute set does not belong to the expected occurrence threshold range corresponding to the message attribute set, then the occurrence count of the message attribute set is marked as abnormal in the analysis report. If the actual occurrence count corresponding to the message attribute set is 0 and the expected occurrence threshold range corresponding to the message attribute set does not include 0, then the occurrence count of the message attribute set is marked as abnormal and not occurring in the analysis report.

[0083] In one possible implementation, the analysis unit 203 is further configured to calculate the sum of the actual occurrence counts of all message attribute sets based on the actual occurrence counts of each message attribute set; calculate the actual occurrence percentage of a message attribute set based on the sum of the actual occurrence counts of all message attribute sets; generate an analysis report based on the actual occurrence percentage of a message attribute set and the expected occurrence percentage threshold range corresponding to the message attribute set; if the actual occurrence percentage of a message attribute set does not fall within the expected occurrence percentage threshold range corresponding to the message attribute set, then the message attribute set is marked as having an abnormal occurrence percentage in the analysis report. Specifically, if the actual occurrence percentage of a message attribute set is 0, and the expected occurrence percentage threshold range corresponding to the message attribute set does not include 0, then the message attribute set is marked as having an abnormal occurrence percentage and not occurring in the analysis report.

[0084] In one possible implementation, the chip verification analysis device 200 is further configured to adjust the incentive constraints of the test cases corresponding to the message attribute set and / or add test cases if the message attribute set is abnormal in multiple chip verification processes. The abnormality of the message attribute set in multiple chip verification processes includes: the fluctuation range of the actual occurrence ratio of the message attribute set in multiple chip verification processes is greater than a preset fluctuation threshold, and / or the actual occurrence ratio of the message attribute set is 0 in a preset number of chip verification rounds, and the expected occurrence ratio threshold range corresponding to the message attribute set does not include 0.

[0085] In one possible implementation, the chip verification analysis device 200 is also used to generate a visual display result based on the analysis report of multiple chip verification processes.

[0086] In this embodiment, the chip verification analysis device 200 obtains a pre-configured set of message attributes to be monitored and their corresponding expected occurrence parameters. During the chip verification process, it obtains the actual occurrence count of the message attribute set and generates an analysis report based on the expected occurrence parameters and the actual occurrence count of the message attribute set. This enables automated quantitative verification of various feature messages in chip verification without the need to write coverage model code separately for each type of message or manually check the verification log line by line and statistically analyze the coverage data by category. This improves the execution efficiency and accuracy of chip verification analysis and ensures the comprehensiveness and reliability of chip verification.

[0087] Reference Figure 4 This diagram illustrates the structure of an electronic device according to an embodiment of the present disclosure. The specific embodiments of the present disclosure do not limit the specific implementation of the electronic device.

[0088] like Figure 4As shown, the electronic device may include: a processor 302, a communications interface 304, a memory 306, and a communications bus 308.

[0089] in: The processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308.

[0090] Communication interface 304 is used to communicate with other electronic devices or servers.

[0091] The processor 302 is used to execute program 310, which can specifically execute the relevant steps in the above-described chip verification and analysis method embodiment.

[0092] Specifically, program 310 may include program code that includes computer operation instructions.

[0093] Processor 302 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure. The electronic device may include one or more processors of the same type, such as one or more CPUs; one or more GPUs; or it may include processors of different types, such as one or more CPUs, one or more GPUs, and one or more ASICs.

[0094] Memory 306 is used to store program 310. Memory 306 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0095] Specifically, program 310 can be used to cause processor 302 to execute the chip verification analysis method in any of the foregoing embodiments.

[0096] The specific implementation of each step in program 310 can be found in the corresponding steps and units described in any of the aforementioned chip verification and analysis method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0097] In this embodiment of the disclosure, by obtaining a pre-configured set of message attributes to be monitored and their corresponding expected occurrence parameters, the actual occurrence frequency of the message attribute set is obtained during the chip verification process. Based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set, an analysis report is generated. This enables automated quantitative verification of various feature messages in chip verification. Furthermore, it eliminates the need to write coverage model code separately for each type of message, and also eliminates the need for manual line-by-line verification log checking and category-by-category statistical coverage data. This improves the execution efficiency and accuracy of chip verification analysis, ensuring the comprehensiveness and reliability of chip verification.

[0098] This disclosure also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described plurality of method embodiments.

[0099] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this disclosure can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this disclosure.

[0100] The methods described above according to embodiments of this disclosure can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded over a network. Thus, the methods described herein can be stored as software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the chip verification and analysis methods described herein are implemented. Furthermore, when a general-purpose computer accesses the code used to implement the chip verification and analysis methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the chip verification and analysis methods shown herein.

[0101] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments disclosed herein.

[0102] It should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0103] The above embodiments are only used to illustrate the embodiments of this disclosure, and are not intended to limit the embodiments of this disclosure. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this disclosure. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this disclosure, and the patent protection scope of the embodiments of this disclosure should be defined by the claims.

Claims

1. A chip verification and analysis method, characterized in that, include: Obtain a message attribute set and the expected occurrence parameters of the message attribute set during the chip verification process, wherein the message attribute set includes at least one message attribute; During chip verification, the actual number of occurrences of the message attribute set is obtained; An analysis report is generated based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set.

2. The method according to claim 1, characterized in that, The step of obtaining the actual occurrence count of the message attribute set during chip verification includes: During chip verification, the set of message attributes corresponding to the messages appearing at a preset node is obtained, wherein the preset node includes at least one of an excitation input node, a message output node, and a message discard node; The dynamic data structure indexes the set of message attributes corresponding to the messages appearing at the preset node, wherein the dynamic data structure is used to store the mapping relationship between the set of message attributes and the actual number of occurrences corresponding to the set of message attributes; If the message attribute set corresponding to the message appearing at the preset node is indexed in the dynamic data structure, the actual occurrence count of the message attribute set corresponding to the message appearing at the preset node is incremented by 1.

3. The method according to claim 2, characterized in that, An analysis report is generated based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set, including: Obtain the actual occurrence count corresponding to the message attribute set in the dynamic data structure; An analysis report is generated based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set. The expected occurrence parameter includes the expected occurrence threshold range and / or the expected occurrence percentage threshold range.

4. The method according to claim 3, characterized in that, The expected occurrence parameter includes an expected occurrence threshold range. The step of generating an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set includes: The analysis report is generated based on the actual occurrence count corresponding to the message attribute set and the expected occurrence threshold range corresponding to the message attribute set; If the actual occurrence count of the message attribute set does not fall within the expected occurrence threshold range of the message attribute set, then the occurrence count of the message attribute set is marked as abnormal in the analysis report. If the actual occurrence count of the message attribute set is 0 and the expected occurrence threshold range of the message attribute set does not include 0, then the occurrence count of the message attribute set is marked as abnormal and not occurring in the analysis report.

5. The method according to claim 3, characterized in that, The expected occurrence parameter includes an expected occurrence percentage threshold range. The step of generating an analysis report based on the actual occurrence count corresponding to the message attribute set and the expected occurrence parameter corresponding to the message attribute set includes: Calculate the sum of the actual occurrence counts corresponding to all message attribute sets based on the actual occurrence counts corresponding to each message attribute set; The actual occurrence percentage of each message attribute set is calculated based on the actual occurrence count corresponding to the set of message attributes and the sum of the actual occurrence counts corresponding to all the sets of message attributes. The analysis report is generated based on the actual occurrence percentage of the message attribute set and the expected occurrence percentage threshold range corresponding to the message attribute set. If the actual occurrence percentage of the message attribute set does not fall within the expected occurrence percentage threshold range corresponding to the message attribute set, then the occurrence percentage of the message attribute set is marked as abnormal in the analysis report. If the actual occurrence percentage of the message attribute set is 0, and the expected occurrence percentage threshold range corresponding to the message attribute set does not include 0, then the occurrence percentage of the message attribute set is marked as abnormal and not occurring in the analysis report.

6. The method according to claim 5, characterized in that, The method further includes: If the message attribute set exhibits anomalies during multiple chip verification processes, the incentive constraints of the test cases corresponding to the message attribute set are adjusted, and / or test cases are added. The anomalies in the message attribute set during multiple chip verification processes include: the actual occurrence percentage of the message attribute set fluctuates more than a preset fluctuation threshold during multiple chip verification processes, and / or the actual occurrence percentage of the message attribute set is 0 in a preset number of chip verification rounds, and the expected occurrence percentage threshold range corresponding to the message attribute set does not include 0.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Based on the analysis reports of multiple chip verification processes, a visual representation of the results is generated.

8. A chip verification and analysis device, characterized in that, include: The first acquisition unit is used to acquire a message attribute set and the expected occurrence parameters of the message attribute set during the chip verification process, wherein the message attribute set includes at least one message attribute. The second acquisition unit is used to acquire the actual number of occurrences of the message attribute set during the chip verification process; The analysis unit is used to generate an analysis report based on the expected occurrence parameters and the actual occurrence frequency of the message attribute set.

9. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the chip verification and analysis method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the chip verification and analysis method as described in any one of claims 1-7.

Images