Fault-tolerant methods and electronic devices for convolutional neural networks

Abstract

This invention provides a fault-tolerant method and electronic device for convolutional neural networks, relating to the field of computer technology. The method includes: calculating a first checksum based on an input first feature map matrix; concatenating the first checksum to the end of the first feature map matrix to obtain a second feature map matrix; calculating a second checksum based on a first weight matrix; concatenating the second checksum to the end of the first weight matrix to obtain a second weight matrix; calculating the first convolution result of the second feature map matrix and the second weight matrix; determining the first row checksum, the second row checksum, the first column checksum, and the second column checksum; calculating the row checksum pair difference and the column checksum pair difference; determining the error location based on the row checksum pair difference and the column checksum pair difference; if multiple errors are detected in the row and column indices of a block at the secondary block row and column, setting potentially erroneous elements within the block to 0 for error correction; and obtaining the corrected second convolution result. This invention improves the fault tolerance performance of convolutional neural networks under high error rates.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a fault-tolerant method for convolutional neural networks and an electronic device. Background Technology

[0002] Algorithm-Based Fault Tolerance (ABFT) refers to fault tolerance achieved by introducing redundancy and error detection mechanisms at the computational algorithm level. This allows the system to continue providing correct results even when errors or failures occur during execution. The advantages of this approach lie in its efficiency and flexibility, making it widely applicable in computing environments requiring high reliability and performance.

[0003] In the field of software testing, it is necessary to distinguish between three terms: Fault (or defect), Error, and Failure. The following definitions are from the ISO 26262 automotive functional safety standard:

[0004] 1) Fault: An abnormal condition that can cause an element or an item to fail. Permanent, intermittent, and transient faults (especially soft faults) all fall into this category.

[0005] 2) Error: discrepancy between a computed, observed, or measured value or condition, and the true, specified, or theoretically correct value or condition.

[0006] 3) Failure: ermination of an intended behavior of an element or anitem due to a fault manifestation.

[0007] The relationship among the three is as follows: a fault may lead to an error, and an error may lead to failure. In this invention, specifically, a fault refers to the change of certain bits of the values ​​of certain memory elements from 0 to 1 or from 1 to 0 during the calculation of certain convolutional layers in a neural network (which can be used to simulate single-event flip errors in a space environment), an error refers to the change in the convolution result in the neural network due to the change of the values ​​of certain elements, and a failure refers to the inconsistency between the classification result output by the neural network and the expected result.

[0008] In research on neural network fault-tolerant technology, existing methods typically design a fault-tolerant testing framework or a fault-tolerant enhancement model. The testing framework can load multiple models and evaluate their robustness and recovery capabilities under different fault conditions. Fault injection technology is widely used to simulate various possible system failures, such as data loss, corrupted model parameters, and abnormal computing resources. When optimizing models and designing fault-tolerant mechanisms, developers use fault injection to compare the performance differences between the original model before modification and the model after introducing the fault-tolerant mechanism under various fault scenarios. By quantitatively analyzing key indicators (such as accuracy, response time, and resource consumption), developers can verify the effectiveness of the fault-tolerant effect. This process not only helps developers identify potential vulnerabilities but also ensures the reliability and stability of the final model in real-world application scenarios, thereby improving the model's adaptability to abnormal conditions and overall performance.

[0009] Existing convolutional neural network fault-tolerance techniques are only suitable for low error rate scenarios, and there is an urgent need to provide a convolutional neural network fault-tolerance method that is compatible with high error rate environments. Summary of the Invention

[0010] This invention provides a convolutional neural network fault-tolerant method and electronic device to address the shortcomings of existing fault-tolerant methods that are only applicable to low error rate environments, and to provide a convolutional neural network fault-tolerant method that is compatible with high error rate environments.

[0011] This invention provides a fault-tolerant method for convolutional neural networks, comprising the following steps: based on the input first feature map matrix Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. ;in, , The first layer of the fault-tolerant layer i Each feature map This indicates the number of feature maps input to the fault-tolerant layer. , According to the first weight matrix Calculate the second checksum , the second checksum Connected to the first weight matrix The second weight matrix is ​​obtained at the end. ;in, ), This indicates the number of weight matrices in the fault-tolerant layer; ; The first layer of the fault-tolerant layer j A weight matrix, ; Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. ;in, = , Indicates the convolution operation; determines the checksum of the first row. The second row checksum First column checksum and the second column checksum ;in, ; Second line checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; ; Second column checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is The secondary block; calculates the row checksum and the difference. Column checksums and differences ;in, , ;Based on the row checksum and the difference Determine the first location of the error According to the column checksum, the difference is checked. Determine the second error location Wherein, the first error location middle, These represent the secondary block column index, the intra-block row index, and the intra-block column index, respectively; the second error location. middle, These represent the secondary block row index, intra-block row index, and intra-block column index, respectively; based on the first error location... and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection status; in response to the error detection status, the intra-block row index and intra-block column index are determined. If multiple errors are detected in both the row and column of the secondary block, the positions of elements within the block where errors may occur are obtained, and error correction is performed by setting the corresponding elements within the block to 0; the corrected second convolution result is then obtained; wherein, the second convolution result is the corrected first convolution result. Remove the remaining parts of the last row and last column of subblocks.

[0012] According to a convolutional neural network fault-tolerant method provided by the present invention, in the step of determining the first error position and the second error location Determine intra-block row indexes and intra-block column indexes After detecting the error, the method further includes: in response to determining the intra-block row index and intra-block column index based on the error detection. If only one error is detected in a secondary block row, the value of the secondary block element with the detected error is obtained, and the difference between the secondary block element value and the column checksum is calculated. Addition for error correction; in response to the error detection, determine the intra-block row index and intra-block column index. If only one error is detected in a secondary block column, the value of the secondary block element with the detected error is obtained, and the difference between the secondary block element value and the row checksum is calculated. Add them together to correct errors.

[0013] According to the present invention, a convolutional neural network fault-tolerant method is provided, wherein the fault is determined based on the first error location. and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection includes: based on the first error location Building the first dictionary ; where the first dictionary The keys are the intra-block row index and the intra-block column index. The first dictionary The key value is the secondary block column index. According to the second error location Create a second dictionary ; where the second dictionary The keys are the intra-block row index and the intra-block column index. The second dictionary The key value is the secondary block row index. ; responds to iterating through the first dictionary in a loop and the second dictionary Know the intra-block row index and intra-block column index Corresponding secondary block row index If unique, then the intra-block row index and intra-block column index are determined. Only one error is detected on the second block line; this is in response to iterating through the first dictionary. and the second dictionary Know the intra-block row index and intra-block column index Corresponding secondary block column index If unique, then the intra-block row index and intra-block column index are determined. Only one error was detected on the secondary block column; this was in response to iterating through the first dictionary. and the second dictionary Know the intra-block row index and intra-block column index Corresponding secondary block row index and secondary block column indexes If neither is unique, then determine the intra-block row index and intra-block column index. Multiple errors were detected in both the secondary block row and the secondary block column.

[0014] According to the present invention, a fault-tolerant method for convolutional neural networks is provided, wherein the method is based on the input first feature map matrix. Calculate the first checksum Previously, the method further included: obtaining the first feature map matrix for the batch-normalized fault-tolerant layer input. The feature values ​​that are not within the preset error range of the mean of the batch normalization results are truncated to a preset value within the preset error range.

[0015] According to the fault-tolerant method for convolutional neural networks provided by the present invention, the batch normalization result approximately satisfies a normal distribution, and the preset error range is [missing information]. , , Let represent the mean and standard deviation of the batch normalization result, respectively. The preset integer represents the value when the feature value is less than the mean of the batch normalization result. When the feature value is greater than the mean of the batch normalization result, the preset value is... Alternatively, the preset value can be uniformly set to the mean of the batch normalization results.

[0016] According to a convolutional neural network fault-tolerant method provided by the present invention, the difference is calculated based on the row checksum. Determine the first location of the error Previously, the method also included: performing a row check and a difference... The elements in the list are traversed in response to the existence of at least one element whose value is not in the list. Within the range, then perform the operation based on the row checksum and the difference. Determine the first location of the error The action; among them, , These represent the pre-obtained row check and the difference value, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer; the difference is checked according to the column checksum. Determine the second error location Previously, the method also included: validating the column and checking the difference. The elements in the list are traversed in response to the existence of at least one element whose value is not in the list. Within the range, then perform the checksum based on the column and the difference. Determine the second error location The action; among them, , These represent the pre-obtained column validation and the difference values, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer.

[0017] According to a convolutional neural network fault-tolerant method provided by the present invention, in the row check and the difference... Before iterating through the elements in the dataset, the method further includes: replacing all convolutional layers of the convolutional neural network with the fault-tolerant layers, and setting a fault-tolerant control switch for each fault-tolerant layer to control whether the fault-tolerant function is enabled; under error-free conditions, using the replaced convolutional neural network to perform inference on the test set data, and calculating the row checksum difference. mean of elements and standard deviation And calculate the difference between the column checksums. mean of elements and standard deviation .

[0018] According to a fault-tolerant method for convolutional neural networks provided by the present invention, before the convolutional neural network receives input data for inference, the method further includes: controlling the opening or closing of the fault-tolerant control switch of each fault-tolerant layer according to the planning result of the number of fault-tolerant layers with fault-tolerant function enabled and the distribution of the fault-tolerant layers with fault-tolerant function enabled; wherein, the planning result of the distribution of fault-tolerant layers includes prioritizing the opening of the fault-tolerant control switch of the fault-tolerant layer with a larger convolutional kernel size; for fault-tolerant layers with the same priority, the fault-tolerant control switch is opened uniformly according to the network depth.

[0019] According to a convolutional neural network fault-tolerance method provided by the present invention, after obtaining the second convolution result after error correction, the method further includes: in response to the cycle time of reaching the test phase execution, performing inference using a test set and calculating an accuracy rate, and calculating a bit error rate based on the accuracy rate; in response to the bit error rate being greater than a first threshold, enabling the fault-tolerance control switch of more fault-tolerance layers; in response to the bit error rate being less than a second threshold, disabling the fault-tolerance control switch of more fault-tolerance layers; wherein the second threshold is less than the first threshold; in response to the bit error rate being less than or equal to the first threshold and greater than or equal to the second threshold, maintaining the setting of the fault-tolerance control switch of the fault-tolerance layer unchanged.

[0020] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the convolutional neural network fault-tolerant method as described above.

[0021] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the convolutional neural network fault-tolerant method as described above.

[0022] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the convolutional neural network fault-tolerant method as described above.

[0023] The convolutional neural network fault-tolerant method and electronic device provided by this invention, by using the input first feature map matrix Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. According to the first weight matrix Calculate the second checksum , the second checksum Connect to the first weight matrix The second weight matrix is ​​obtained at the end. Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. Determine the checksum of the first row. The second row checksum First column checksum and the second column checksum Calculate the difference between the row checksum and the checksum. Column checksums and differences Based on row checksum and difference Determine the first location of the error According to column checksum, the difference is checked. Determine the second error location According to the first error location Second error location Determine the corresponding intra-block row index and intra-block column index. Error detection results are used to determine the intra-block row index and intra-block column index based on the error detection results. If multiple errors are detected in both the secondary block rows and columns, the positions of elements within the block where errors may occur are obtained, and the corresponding elements within the block are set to 0 to correct the errors. The corrected second convolution result is then obtained, which improves the fault tolerance of the convolutional neural network under high error rates, thereby enhancing the full-scene adaptability and fault tolerance performance of the convolutional neural network. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is one of the flowcharts of the convolutional neural network fault-tolerant method provided by the present invention.

[0026] Figure 2 This is one of the schematic diagrams illustrating the principle of the convolutional neural network fault-tolerant method provided by this invention.

[0027] Figure 3 This is the second schematic diagram illustrating the principle of the convolutional neural network fault-tolerant method provided by this invention.

[0028] Figure 4 This is the second flowchart of the convolutional neural network fault-tolerant method provided by the present invention.

[0029] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0031] Figure 1 This is one of the flowcharts illustrating the fault-tolerant method for convolutional neural networks provided by the present invention. The convolutional neural network receives input data and performs inference. During forward propagation, the fault-tolerant method is executed once for each fault-tolerant layer with its fault-tolerant function enabled. The fault-tolerant layer is a convolutional layer protected by the fault-tolerant method. If the fault-tolerant function is enabled, the fault-tolerant method can be executed and play its protective role. Figure 1 As shown, the fault-tolerance method includes:

[0032] Step S1: Based on the input first feature map matrix Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. ;in, , The first layer of the fault-tolerant layer i Each feature map This indicates the number of feature maps input to the fault-tolerant layer. , .

[0033] This invention utilizes the distributive law of convolution operations to generate checksums. Specifically, the following relationships exist:

[0034] (1)

[0035] Similarly, there are:

[0036] (2)

[0037] Figure 2 This is one of the schematic diagrams illustrating the principle of the convolutional neural network fault-tolerant method provided by this invention. For example... Figure 2 As shown, secondary block Its size is denoted as R×R, where R can be calculated using the input feature map size H, convolution kernel size F, padding P, and stride S, as shown in the formula:

[0038]

[0039] In the fault-tolerance method of the present invention, a first checksum is added. That is, the parenthetical part in the right half of formula (1). And the first checksum... Connected to the first feature map matrix The end yields the second feature map matrix. .

[0040] Step S2: Based on the first weight matrix Calculate the second checksum , the second checksum Connected to the first weight matrix The second weight matrix is ​​obtained at the end. ;in, ), This indicates the number of weight matrices in the fault-tolerant layer; ; The first layer of the fault-tolerant layer j A weight matrix, .

[0041] In the fault-tolerance method of the present invention, a second checksum is added. That is, the parenthetical part in the right half of formula (2). And the second checksum... Connect to the first weight matrix The second weight matrix is ​​obtained at the end. .

[0042] The execution order of steps S1 and S2 can be interchanged, or they can be executed simultaneously.

[0043] Step S3: Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. ;in, = , This represents the convolution operation.

[0044] Step S4: Determine the checksum of the first row. The second row checksum First column checksum and the second column checksum ;in, ; Second line checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; ; Second column checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is The secondary block.

[0045] From the first convolution result Obtain the first row checksum This is the result of the first convolution. Remove the portion consisting of all the last rows of subblocks in the last column of subblocks. Or, based on... Calculate the checksum of the first row .according to Calculated The elements in the table are used to obtain the checksum of the second row. The second line checksum Each element is the result of the first convolution. The result of summing the subblocks by row after removing the last column and the last row of subblocks.

[0046] From the first convolution result Obtain the first column checksum That is Remove the portion consisting of the last column of all sub-blocks in the last row of the sub-block. Or based on The checksum of the first column is calculated. .according to Calculated The elements in the table are used to obtain the second column of checksum. The second column checksum Each element is the result of the first convolution. The result of summing the subblocks by column after removing the last column and the last row of subblocks.

[0047] Step S5: Calculate the row checksum and the difference. Column checksums and differences ;in, , .

[0048] First line checksum Second row checksum Construct a row checksum pair, with the first column checksum as the checksum. and the second column checksum This constitutes a column checksum. If no errors occur, there should be... as well as Therefore, the difference is calculated by checking the row checksum. Column checksums and differences ,Will , and , Comparisons can detect, locate, and correct soft errors under certain conditions.

[0049] Step S6: Calculate the difference based on the row checksum. Determine the first location of the error According to the column checksum, the difference is checked. Determine the second error location Wherein, the first error location middle, These represent the secondary block column index, the intra-block row index, and the intra-block column index, respectively; the second error location. middle, These represent the secondary block row index, the intra-block row index, and the intra-block column index, respectively.

[0050] In high-error-rate environments, multiple errors are highly likely to occur in a single convolutional layer, potentially affecting the first convolution result. There are multiple single elements in different rows and columns, or multiple entire rows and columns of elements that are corrupted.

[0051] Record the result of the first convolution. The index of a certain element in () is ( ),in and It is a secondary block index, and The element is in the secondary block The index within. Then when the row checksum is... and When discrepancies exist, the erroneous secondary block can be obtained. column index and the index of elements within the secondary block and Similarly, when column checksums are... and When discrepancies exist, the erroneous secondary block can be obtained. row index and the index of elements within the secondary block and .

[0052] Step S7: Based on the first error location and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection status.

[0053] Based on the index of the element within the same second-order block and First error location Secondary block column index The quantity, and based on the index of the element corresponding to the element within the same second-order block. and The second error location Secondary block row index The quantity is determined corresponding to the intra-block row index and intra-block column index. Error detection status.

[0054] Step S8: In response to the error detection, determine the intra-block row index and intra-block column index. If multiple errors are detected in both the secondary block row and the secondary block column, the positions of the elements within the block where the errors may occur are obtained, and the errors are corrected by setting the corresponding elements within the block to 0.

[0055] Elements with different indices within a secondary block have independent checksums, allowing for the detection and correction of multiple soft errors at different positions within the secondary block. Correction simply involves adding back the difference in checksums. However, when errors at the same position occur in multiple secondary blocks, the following three scenarios emerge:

[0056] 1) Errors with the same location in multiple blocks occur in secondary blocks in the same row but different columns;

[0057] 2) Errors with the same location in multiple blocks occur in secondary blocks of different rows but the same column;

[0058] 3) Errors with the same location in multiple blocks occur in secondary blocks in different rows and columns.

[0059] In the first two scenarios, the error accumulates at the same position in either the row or column checksum, but remains at different positions within either checksum, thus allowing for normal detection and correction. However, in the third scenario, the error accumulates at the same position in both the row and column checksums, leading to two problems:

[0060] First, it cannot distinguish the secondary block index that actually caused the error.

[0061] Figure 3 This is the second schematic diagram illustrating the principle of the convolutional neural network fault-tolerant method provided by this invention. For the error location... When the row checksum detects two (or more) error locations and Meanwhile, the column checksum also detected two (or more) error locations. and ,Right now Figure 3 In the first row checksum Checksum with the first column At the same internal index (3,3), the first row checksum Checksum with the first column Two errors were detected in both cases. Therefore, the first convolution result... There are four possible error locations in the code, namely... Figure 3 The parts marked 1, 2, 3, and 4, taking the detection of 2 errors in both rows and columns as an example, have 7 error scenarios: ① Block 1 and 4 error; ② Block 2 and 3 error; ③ Block 1, 2, and 4 error; ④ Block 1, 3, and 4 error; ⑤ Block 1, 2, and 3 error; ⑥ Block 2, 3, and 4 error; ⑦ Block 1, 2, 3, and 4 error.

[0062] This situation is more likely to occur when the batch size is larger, the feature map size is smaller, and the number of channels is larger. In common convolutional neural networks, the later convolutional layers need to capture higher-level and more abstract features, so the number of channels is increased to encode more information. At the same time, due to pooling operations or convolutions with a stride greater than 1, the size of the feature map continues to decrease, so this situation must be considered.

[0063] If an error results in errors in at most one row or one column, these errors will accumulate in the first row checksum or the first column checksum at different positions. Error correction can be achieved simply by adding the errors back to the original value. When multiple errors accumulate in both the first row and first column checksums, obtaining the correct value becomes a linear algebra problem: extract these errors and construct a separate matrix. Assuming N column checksum pairs and M row checksum pairs are detected, we have an N x M matrix with the sum of all elements in each row and column, and we need to find the value of each element in the matrix. A necessary but not sufficient condition for this problem to have a solution is N = M. This condition is not always satisfied, and even if it is, there may be multiple solutions, so it is not always possible to accurately find the original value. Furthermore, at high error rates, the error value will expand to infinity after only a few layers, making it impossible to estimate the original value.

[0064] Since experiments have shown that precise restoration of the original value is unnecessary, this invention employs a zeroing method for error correction, that is, based on the error detection results, the intra-block row index and intra-block column index are determined. When multiple errors are detected in both the secondary block row and the secondary block column, the positions of the elements within the block where the errors may occur are obtained, and the errors are corrected by setting the corresponding elements within the block to 0. By suppressing the propagation and accumulation of error values, the network inference accuracy can be restored well.

[0065] Step S9: Obtain the corrected second convolution result; wherein, the second convolution result is the corrected first convolution result. Remove the remaining parts of the last row and last column of subblocks.

[0066] After error correction, the first convolution result is... The remaining part after removing the last row and the last column of the sub-blocks is the final corrected second convolution result.

[0067] The convolutional neural network fault-tolerant method provided by this invention, by using the input first feature map matrix... Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. According to the first weight matrix Calculate the second checksum , the second checksum Connect to the first weight matrix The second weight matrix is ​​obtained at the end. Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. Determine the checksum of the first row. The second row checksum First column checksum and the second column checksum Calculate the difference between the row checksum and the checksum. Column checksums and differences Based on row checksum and difference Determine the first location of the error According to column checksum, the difference is checked. Determine the second error location According to the first error location Second error location Determine the corresponding intra-block row index and intra-block column index. Error detection results are used to determine the intra-block row index and intra-block column index based on the error detection results. If multiple errors are detected in both the secondary block rows and columns, the positions of elements within the block where errors may occur are obtained, and the corresponding elements within the block are set to 0 to correct the errors. The corrected second convolution result is then obtained, which improves the fault tolerance of the convolutional neural network under high error rates, thereby enhancing the full-scene adaptability and fault tolerance performance of the convolutional neural network.

[0068] According to a convolutional neural network fault-tolerant method provided by the present invention, in the step of determining the first error position and the second error location Determine intra-block row indexes and intra-block column indexes After detecting the error, the method further includes: in response to determining the intra-block row index and intra-block column index based on the error detection. If only one error is detected in a secondary block row, the value of the secondary block element with the detected error is obtained, and the difference between the secondary block element value and the column checksum is calculated. Addition for error correction; in response to the error detection, determine the intra-block row index and intra-block column index. If only one error is detected in a secondary block column, the value of the secondary block element with the detected error is obtained, and the difference between the secondary block element value and the row checksum is calculated. Add them together to correct errors.

[0069] If the block row index and block column index are known based on the error detection results Only one error is detected on a row in a secondary block, while one or more errors can be detected on a column in a secondary block. The difference can be calculated using column checksums. Error correction is performed. Specifically, the values ​​of the secondary block elements that were found to have errors are obtained, and the differences between these secondary block element values ​​and the column checksums are calculated. Correct the errors.

[0070] If the block row index and block column index are known based on the error detection results Only one error is detected on a column of a secondary block, while one or more errors can be detected on a row of a secondary block. The difference can be calculated using row checksums. Error correction is performed. Specifically, the values ​​of the secondary block elements that detected errors are obtained, and the differences between these secondary block element values ​​and the row checksums are calculated. Add them together to correct errors.

[0071] The convolutional neural network fault-tolerant method provided by this invention obtains the intra-block row index and intra-block column index in response to error detection. If only one error is detected in the secondary block row, then obtain the value of the secondary block element that detected the error, and calculate the difference between the secondary block element value and the column checksum. Error correction is performed, responding by determining the intra-block row index and intra-block column index based on error detection. If only one error is detected in the secondary block column, obtain the value of the secondary block element that detected the error, and calculate the difference between the secondary block element value and the row checksum. Adding the results together for error correction improves the ease and efficiency of error correction even with low error rates.

[0072] According to the present invention, a convolutional neural network fault-tolerant method is provided, wherein the fault is determined based on the first error location. and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection includes: based on the first error location Building the first dictionary ; where the first dictionary The keys are the intra-block row index and the intra-block column index. The first dictionary The key value is the secondary block column index. According to the second error location Create a second dictionary ; where the second dictionary The keys are the intra-block row index and the intra-block column index. The second dictionary The key value is the secondary block row index. ; responds to iterating through the first dictionary in a loop and the second dictionary Know the intra-block row index and intra-block column index Corresponding secondary block row index If unique, then the intra-block row index and intra-block column index are determined. Only one error is detected on the second block line; this is in response to iterating through the first dictionary. and the second dictionary Know the intra-block row index and intra-block column index Corresponding secondary block column index If unique, then the intra-block row index and intra-block column index are determined. Only one error was detected on the secondary block column; this was in response to iterating through the first dictionary. and the second dictionary Know the intra-block row index and intra-block column index Corresponding secondary block row index and secondary block column indexes If neither is unique, then determine the intra-block row index and intra-block column index. Multiple errors were detected in both the secondary block row and the secondary block column.

[0073] Based on the first error location Second error location Determine intra-block row indexes and intra-block column indexes When detecting errors, based on the first error location Building the first dictionary First Dictionary The keys are the intra-block row index and the intra-block column index. First Dictionary The key value is a secondary block column index. According to the second error location Create a second dictionary Second dictionary The keys are the intra-block row index and the intra-block column index. Second dictionary The key value is the secondary block row index. .

[0074] If we iterate through the first dictionary using a loop Second Dictionary Know the intra-block row index and intra-block column index Corresponding secondary block row index If unique, then the intra-block row index and intra-block column index are determined. If only one error is detected on a secondary block line, the difference can be added back for error correction.

[0075] If we iterate through the first dictionary using a loop Second Dictionary Know the intra-block row index and intra-block column index Corresponding secondary block column index If unique, then the intra-block row index and intra-block column index are determined. If only one error is detected on the secondary block column, the difference can be added back for error correction.

[0076] If we iterate through the first dictionary using a loop Second Dictionary Know the intra-block row index and intra-block column index Corresponding secondary block row index and secondary block column indexes If neither is unique, then determine the intra-block row index and intra-block column index. Multiple errors were detected in both the second-order block rows and columns. For all and Iterate through all its combinations and identify all possible error locations. , , ... Set the corresponding element within the block to 0.

[0077] The convolutional neural network fault-tolerant method provided by this invention establishes a first dictionary. and the second dictionary By iterating through the first dictionary and the second dictionary Determine the corresponding intra-block row index and intra-block column index. Error detection has improved the performance of intra-block row indexes and intra-block column indexes. The efficiency of obtaining error detection information.

[0078] According to the present invention, a fault-tolerant method for convolutional neural networks is provided, wherein the method is based on the input first feature map matrix. Calculate the first checksum Previously, the method further included: obtaining the first feature map matrix for the batch-normalized fault-tolerant layer input. The feature values ​​that are not within the preset error range of the mean of the batch normalization results are truncated to a preset value within the preset error range.

[0079] This invention leverages the batch normalization feature of networks like ResNet before convolutional layers. By truncating the input to the convolutional layers according to their parameters, it suppresses error propagation and accumulation during computation. The batch normalization process can include calculating the mean and variance of features within each mini-batch of data, and using these statistics to standardize the input, giving it zero mean and unit variance.

[0080] A preset error range can be set based on the mean and standard deviation of the batch normalization results. When performing numerical truncation, the first feature map matrix is ​​obtained for the batch normalized fault-tolerant layer input. The feature values ​​that are not within the preset error range of the mean of the batch normalization result are truncated to the preset value within the preset error range.

[0081] This invention can also be applied to other networks that use batch normalization after convolutional layers, such as Inception-v2 / v3, DenseNet, MobileNet, ShuffleNet, EfficientNet, etc.

[0082] The convolutional neural network fault-tolerant method provided by this invention obtains the first feature map matrix by taking the batch-normalized input of the fault-tolerant layer. The system identifies eigenvalues ​​that are outside the preset error range of the mean of the batch normalization results and truncates them to preset values ​​within the preset error range. This reduces the impact of error propagation caused by large absolute values ​​and further improves the system's fault tolerance performance.

[0083] According to the fault-tolerant method for convolutional neural networks provided by the present invention, the batch normalization result approximately satisfies a normal distribution, and the preset error range is [missing information]. , , Let represent the mean and standard deviation of the batch normalization result, respectively. Represents a preset integer;

[0084] When the eigenvalue is less than the mean of the batch normalization result, the preset value is... When the eigenvalue is greater than the mean of the batch normalization result, the preset value is... Alternatively, the default value can be uniformly set to the mean of the batch normalization results.

[0085] In this embodiment, the batch normalization result approximately follows a normal distribution. If the batch normalization result approximately follows a standard normal distribution, then the mean of the batch normalization result is... =0, standard deviation of batch normalization results =1. The preset error range is... .

[0086] When the eigenvalue is less than the mean of the batch normalization result, the preset value can be... Or it can also be [ Other values ​​between ]; when the eigenvalue is greater than the mean of the batch normalization result, the preset value is Or it can also be [0, Other values ​​between ]

[0087] To avoid the propagation of erroneous values, the eigenvalues ​​can be set directly to the mean of the batch normalization result when the eigenvalue is less than or greater than the mean of the batch normalization result, such as setting the eigenvalues ​​to 0 in both cases.

[0088] in, Given a preset integer, experiments were conducted. When the integer is less than or equal to 16, good results can be obtained.

[0089] The convolutional neural network fault-tolerant method provided by this invention further improves fault tolerance performance by reasonably setting preset values ​​for truncation of eigenvalues ​​when they are less than the mean of the batch normalization result and when they are greater than the mean of the batch normalization result.

[0090] According to a convolutional neural network fault-tolerant method provided by the present invention, the difference is calculated based on the row checksum. Determine the first location of the error Previously, the method also included: performing a row check and a difference... The elements in the list are traversed in response to the existence of at least one element whose value is not in the list. Within the range, then perform the operation based on the row checksum and the difference. Determine the first location of the error The action; among them, , These represent the pre-obtained row check and the difference value, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer; the difference is checked according to the column checksum. Determine the second error location Previously, the method also included: validating the column and checking the difference. The elements in the list are traversed in response to the existence of at least one element whose value is not in the list. Within the range, then perform the checksum based on the column and the difference. Determine the second error location The action; among them, , These represent the pre-obtained column validation and the difference values, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer.

[0091] Based on row validation and the difference Determine the first location of the error Previously, it was necessary to determine the difference based on row validation. An error location was detected. The row checksum difference is then checked. The method for detecting error locations is to perform row checks and check the differences. Iterate through the elements in the array, check the row checksum, and compare the differences. Are the elements in Within the range; if the row check sum is within the range of the difference All elements in Within the range, it indicates that the difference is based on row checksum. No error location detected; if the row checksum is different. There are at least one element whose value is not in the [element name]. Within the range, it indicates that the difference is based on row checksum. If an error location is detected, then the difference is calculated based on the row checksum. Determine the first location of the error The action. , These represent the pre-obtained row check and the difference value, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer.

[0092] Based on column validation and difference analysis Determine the second error location Previously, it was necessary to determine the difference based on column validation. Error location detected. Check column checksum against the difference. The method for detecting error locations is: check the column and check the difference. Iterate through the elements in the column, check the column checksum and compare the differences. Are the elements in Within the range; if the column checksum is within the range of differences All elements in Within the range, it means that the difference is calculated based on the column checksum. No error location detected; if column checksums match the difference There are at least one element whose value is not in the [element name]. If it is within the range, it means that the difference is checked according to the column checksum. If an error is detected, perform a checksum based on the column to recalculate the difference. Determine the second error location The action. Among them, , These represent the pre-obtained column validation and the difference values, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer.

[0093] The convolutional neural network fault-tolerant method provided by this invention performs row verification and difference value verification. The elements in the table are iterated through to determine whether they fall within a preset range determined by the mean and variance, and by checking the columns and the differences. The elements in the table are iterated through to determine whether they fall within a preset range determined by the mean and variance, improving the efficiency of row validation and difference analysis. Determine the first location of the error And based on column checksums and differences Determine the second error location Reliability.

[0094] According to a convolutional neural network fault-tolerant method provided by the present invention, in the row check and the difference... Before iterating through the elements in the dataset, the method further includes: replacing all convolutional layers of the convolutional neural network with the fault-tolerant layers, and setting a fault-tolerant control switch for each fault-tolerant layer to control whether the fault-tolerant function is enabled; under error-free conditions, using the replaced convolutional neural network to perform inference on the test set data, and calculating the row checksum difference. mean of elements and standard deviation And calculate the difference between the column checksums. mean of elements and standard deviation .

[0095] Row checksum and difference mean of elements and standard deviation And calculate the column checksum for the difference. mean of elements and standard deviation This can be determined using a test set before the model is applied.

[0096] In row validation and difference analysis Before traversing the elements in the convolutional neural network, all convolutional layers are replaced with fault-tolerant layers, and a fault-tolerant control switch is set for each fault-tolerant layer to control whether the fault-tolerant function is enabled.

[0097] Under error-free conditions, the replaced convolutional neural network is used to perform inference on the test set data, calculating the row checksum and the difference. mean of elements and standard deviation And calculate the column checksum for the difference. mean of elements and standard deviation Since the operation is performed under error-free conditions, the fault tolerance control switches of each fault tolerance layer can be turned off to reduce time overhead.

[0098] The convolutional neural network fault-tolerant method provided by this invention replaces all convolutional layers of a convolutional neural network with fault-tolerant layers. A fault-tolerant control switch is set for each fault-tolerant layer to control whether the fault-tolerant function is enabled. Under fault-free conditions, the replaced convolutional neural network is used to perform inference on the test set data, calculating row checksums and pairwise differences. mean of elements and standard deviation And calculate the column checksum for the difference. mean of elements and standard deviation This provides a basis for determining the location of the error.

[0099] According to a fault-tolerant method for convolutional neural networks provided by the present invention, before the convolutional neural network receives input data for inference, the method further includes: controlling the opening or closing of the fault-tolerant control switch of each fault-tolerant layer according to the planning result of the number of fault-tolerant layers with fault-tolerant function enabled and the distribution of the fault-tolerant layers with fault-tolerant function enabled; wherein, the planning result of the distribution of fault-tolerant layers includes prioritizing the opening of the fault-tolerant control switch of the fault-tolerant layer with a larger convolutional kernel size; for fault-tolerant layers with the same priority, the fault-tolerant control switch is opened uniformly according to the network depth.

[0100] The number of fault-tolerant layers to enable fault tolerance can be pre-planned; this number is called the fault-tolerant layer number. Furthermore, the specific fault-tolerant layers to enable fault tolerance can also be pre-planned. Therefore, before the convolutional neural network receives input data for inference, the fault tolerance control switches of each fault-tolerant layer are controlled to be turned on or off based on the planned distribution of the number and type of fault-tolerant layers.

[0101] It is preferable to enable the fault tolerance switch of fault-tolerant layers with larger kernel sizes, because these layers undertake more computational tasks; for fault-tolerant layers of the same priority, they should be selected uniformly according to the network depth, because the number of errors is exponentially related to the number of propagation layers, so averaging the error propagation path length can minimize the number of errors.

[0102] The convolutional neural network fault-tolerant method provided by this invention further improves fault tolerance performance by rationally planning the number and distribution of fault-tolerant layers activated according to the fault-tolerant function, and controlling the opening or closing of the fault-tolerant control switch of each fault-tolerant layer.

[0103] According to a convolutional neural network fault-tolerance method provided by the present invention, after obtaining the second convolution result after error correction, the method further includes: in response to the cycle time of reaching the test phase execution, performing inference using a test set and calculating an accuracy rate, and calculating a bit error rate based on the accuracy rate; in response to the bit error rate being greater than a first threshold, enabling the fault-tolerance control switch of more fault-tolerance layers; in response to the bit error rate being less than a second threshold, disabling the fault-tolerance control switch of more fault-tolerance layers; wherein the second threshold is less than the first threshold; in response to the bit error rate being less than or equal to the first threshold and greater than or equal to the second threshold, maintaining the setting of the fault-tolerance control switch of the fault-tolerance layer unchanged.

[0104] Most existing ABFT methods for neural network fault tolerance only propose a verification and construction scheme and design corresponding experiments for effectiveness analysis. In reality, various fault tolerance methods are only effective and feasible within a certain error rate range. In practical applications, fault tolerance effectiveness is affected by various factors, including the environment, hardware, and the fault tolerance capability of the model itself. For example, wafer-level chips, due to their complex manufacturing process and large area, have a much higher rate of manufacturing defects and bit errors than traditional chips; different hardware (such as FPGAs and ASICs) also have different error rates; at the same time, different model structures and different layers of the model have different fault tolerance capabilities. When the error rate is very high, most fault tolerance methods, even if they can correct errors normally, will not be able to obtain correct results in a timely manner due to excessive overhead. In this case, sacrificing some error coverage to reduce overhead may be reasonable, but currently there is a lack of methods that can flexibly balance the two.

[0105] This invention designs a workflow at the network level that dynamically adjusts the number of fault-tolerant layers based on the detection error rate. After the convolutional neural network completes one or more batch inference tasks, it determines whether to enter the testing phase. If the set period has been reached, it enters the testing phase; otherwise, it continues executing the inference task. The testing phase process includes:

[0106] a) Perform inference on the test set and observe the accuracy. The correspondence between the accuracy and bit error rate on the test set has been obtained through fault injection experiments in a fault-free environment. The bit error rate can be estimated based on the network output accuracy on the test set in a fault-prone environment (this may not equal the actual bit error rate, but the estimated bit error rate can be used to decide whether to strengthen the fault tolerance). Regarding fault injection tools, MLIR fault injection tool or other PyTorch-based fault injection tools such as OPFI and PyTorchFI can be used.

[0107] b) If the estimated bit error rate is too high, set the fault tolerance control switch of more fault tolerance layers to true (i.e., on); if the estimated bit error rate is too low, set the fault tolerance control switch of more fault tolerance layers to false (i.e., off). If the estimated bit error rate is within an acceptable range, keep the fault tolerance control switch settings of the fault tolerance layers unchanged.

[0108] c) If the error rate is extremely high, try truncating the convolution input values ​​to ±3 and setting them to zero. This can further reduce inference time in some cases of high error rates.

[0109] The convolutional neural network fault-tolerance method provided by this invention performs inference and calculates the accuracy using the test set in response to the cycle time of reaching the test phase. Based on the accuracy, it calculates the bit error rate. If the bit error rate is greater than a first threshold, it activates the fault-tolerance control switch for more fault-tolerance layers. If the bit error rate is less than a second threshold, it deactivates the fault-tolerance control switch for more fault-tolerance layers. If the bit error rate is less than or equal to the first threshold and greater than or equal to the second threshold, it keeps the fault-tolerance control switch setting unchanged. The fault-tolerance function of the fault-tolerance layers can be adjusted to enable or disable based on real-time fault-tolerance performance, introducing a flexible fault-tolerance level. This approach aims to restore network output accuracy as much as possible and reduce overhead in high error rate environments, achieving a dynamic trade-off between classification accuracy and overhead.

[0110] Figure 4 This is the second flowchart illustrating the fault-tolerant convolutional neural network method provided by this invention. This invention first performs initialization in a fault-free environment, and then periodically runs the fault-tolerant phase and the testing phase in a fault-prone environment. The overall flowchart is as follows: Figure 4 As shown, the method includes:

[0111] 1. Initialization phase:

[0112] The system accepts a pre-trained convolutional neural network model and a portion of the test set of the dataset used for its training. It does not need to use the entire test set; only about 100 images from it are required.

[0113] Step 1: The system replaces all convolutional layers in the network model with convolutional layers protected by the ABFT fault-tolerance method, called fault-tolerant layers. Fault-tolerant layers add checksum calculations and checks before and after convolution calculation during forward propagation. In addition, compared to ordinary convolutional layers, they have four more attributes: the mean and standard deviation of the differences between row checksum pairs and column checksum pairs. A switch controls whether fault tolerance is enabled; if disabled, the convolution result is returned directly, just like ordinary convolutional layers.

[0114] Step 2 involves using the replaced convolutional neural network to perform inference on a small amount of test data. The goal is to compute the row checksum difference under error-free conditions. The mean and standard deviation Used to set the error detection threshold And calculate the column checksum for the difference. The mean and standard deviation Used to set the error detection threshold The default parameter k=4, and each convolutional layer has an independent... , , , , and .

[0115] 2. Fault Tolerance Phase:

[0116] After initialization, the fault-tolerant convolutional neural network receives data and performs inference in a fault-prone environment (referring to the actual application environment of the model). During forward propagation, the following fault-tolerant process is executed once for each fault-tolerant layer with fault tolerance enabled:

[0117] a) For the convolutional layer input after batch normalization (generally the parameters are set to mean of 0 and variance of 1 by default), all values ​​not in the range of [-3,3] are truncated to ±3.

[0118] b) For the input feature map matrix ,calculate And connect to the end of the matrix to obtain For the weight matrix calculate And connect to the end of the matrix to obtain ;

[0119] c) Calculation and The convolution of is denoted as . ;

[0120] d) From Obtain row checksum That is Remove the portion consisting of all the last rows of sub-blocks in the last column of the sub-block. Calculate the checksum of another row. , No. Each element is represented as That is After removing the last column and the last row of sub-blocks, sum the sub-blocks row by row.

[0121] from Obtain column checksum That is Remove the portion consisting of the last column of all sub-blocks in the last row of the sub-block. Calculate the checksum of the other column. , No. Each element is represented as That is After removing the last column and the last row of sub-blocks, sum the sub-blocks column by column.

[0122] e) Calculate the difference between the row checksums and the checksums. Difference from column checksum pair ;

[0123] f) To Iterate through the elements and determine if their values ​​are within the threshold. If the value is outside this range, it is considered an error, and the error location is recorded. Iterate through the elements and determine if their values ​​are within the threshold. If the value is outside this range, it is considered an error, and the location of the error is recorded. Error location The meaning is (secondary block column index, intra-block row index, intra-block column index). Error location The meaning is (secondary block row index, intra-block row index, intra-block column index);

[0124] g) According to respectively and The error location is determined by the intra-block row index and intra-block column index. Create two dictionaries for the keys. and Iterate through the blocks and determine whether multiple errors are detected in the rows / columns of the sub-blocks at a given index.

[0125] h) Use nested loops to key Traversal and ,for and :like or This indicates that the index within this block only detected one error in the row or column sub-block, and the difference can be added back directly for error correction;

[0126] like and This indicates that the index within this block detected multiple errors in both the row and column sub-blocks. and Iterate through all its combinations and check all possible error locations. , , ... Set to zero to suppress erroneous output;

[0127] i) Return the corrected convolution result, i.e. The remaining part after removing the last row and the last column of sub-blocks.

[0128] 3. Testing Phase

[0129] After the convolutional neural network completes one or more batch inference tasks, it determines whether to enter the testing phase. If the set period has elapsed, it enters the testing phase; otherwise, it continues executing the inference task. The testing phase process is as follows:

[0130] a) Perform inference on the test set and observe the accuracy. The correspondence between the accuracy of the test set and the bit error rate has been obtained through fault injection experiments in a fault-free environment. The bit error rate can be estimated based on the network output accuracy on the test set in a fault-prone environment (it may not be equal to the actual bit error rate, but the estimated bit error rate can be used to decide whether to strengthen the fault tolerance).

[0131] (b) If the estimated error rate is too high, set the fault tolerance switch of more fault-tolerant layers to true; if the estimated error rate is too low, set the fault tolerance switch of more fault-tolerant layers to false. Fault-tolerant layers are selected with different priorities. It is preferable to enable the fault tolerance switch of fault-tolerant layers with larger kernel sizes, as these layers handle more computational tasks. For fault-tolerant layers of the same priority, they are selected evenly according to network depth, because the number of errors is exponentially related to the number of propagation layers; therefore, averaging the error propagation path length can minimize the number of errors.

[0132] c) If the error rate is extremely high, try truncating the convolution input values ​​to ±3 and setting them to zero. This can further reduce inference time in some cases of high error rates.

[0133] The following experimental data further illustrates the fault-tolerant performance of the convolutional neural network fault-tolerant method provided by this invention.

[0134] Table 1 is a summary table of experimental data. For the sake of simplicity, only a portion of the average data is extracted for different key points.

[0135] The experimental data was based on ResNet50, with datasets CIFAR10 and CIFAR100, and the test set containing all 10,000 images. The fault injection tool was MRFI, the fault injection type was activation, the fault injection selector was RandomPositionByRate, and the fault model was a FloatRandomBitFlip-float32 type numerical value.

[0136] Table 1

[0137]

[0138] Table 1 - Continued

[0139]

[0140] Here, "over 3 set to 3" means that if the eigenvalue is in the range [0,3], it is set to 3; if the eigenvalue is in the range [-3,0], it is set to -3.

[0141] Key point 1: The optimal number of fault-tolerant layers varies for different bit error rates.

[0142] Example tables are shown in Table 2.

[0143] Table 2

[0144]

[0145] In low error rate environments (bit error rate = 1e-6), only 4 layers of fault tolerance are needed to restore the network classification accuracy to over 99%. Using more fault tolerance layers will only increase inference time.

[0146] In environments with higher error rates, 4-layer fault tolerance has an absolute advantage in terms of overhead, with inference time nearly identical to that without fault tolerance across all three error rate levels, even at an in-situ error rate of 10. -4 This approach can restore the network classification accuracy, which is equivalent to random guessing, to over 70%. However, to achieve even higher classification accuracy, it's necessary to increase the number of fault-tolerant layers rather than using three 4-layer fault-tolerant networks for voting. For example, in the 10-class classification task CIFAR10, with 4 layers of fault tolerance, the network classification accuracy is 75%. Using three 4-layer fault-tolerant networks for voting, the one with the most votes becomes the final result. If the three networks produce different results, one is randomly selected. Simple calculations show a final classification accuracy of approximately 85.8%, still lower than 16 layers of fault tolerance, and the difference is even greater for 100-class classification tasks. Therefore, in high-error-rate environments, increasing the number of fault-tolerant layers is still necessary to achieve high accuracy. Excessive overhead can be significantly reduced by truncating error values ​​from the convolutional layer inputs, as discussed in the next key point.

[0147] Beneficial technical effect: The appropriate number of fault-tolerant layers can be selected based on the estimated bit error rate, overhead, and classification accuracy requirements.

[0148] Key Point 2: Error truncation can reduce error correction time and improve accuracy.

[0149] Example tables are shown in Table 3.

[0150] Table 3

[0151]

[0152] It can be seen that the more fault-tolerant layers there are, the more error values ​​can be truncated from the convolutional input, thus reducing error correction time and improving accuracy, with an in-place error rate as high as 10%. -4 At that time, using 53 layers of fault tolerance + error value truncation can achieve the best recovery effect. Although the time cost is still more than 10 times that of 4 layers of fault tolerance, it can achieve a higher classification accuracy than using 10 4-layer fault tolerance networks for voting (the latter has a final accuracy of about 92.8%).

[0153] Beneficial technical effects: Error value truncation can be used when the error rate is high, reducing time overhead and improving accuracy.

[0154] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a convolutional neural network fault-tolerant method, which includes: based on the input first feature map matrix... Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. ;in, , The first layer of the fault-tolerant layer i Each feature map This indicates the number of feature maps input to the fault-tolerant layer. , According to the first weight matrix Calculate the second checksum , the second checksum Connected to the first weight matrix The second weight matrix is ​​obtained at the end. ;in, ), This indicates the number of weight matrices in the fault-tolerant layer; ; The first layer of the fault-tolerant layer j A weight matrix, ; Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. ;in, = , Indicates the convolution operation; determines the checksum of the first row. The second row checksum First column checksum and the second column checksum ;in, ; Second line checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; ; Second column checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is The secondary block; calculates the row checksum and the difference. Column checksums and differences ;in, , ;Based on the row checksum and the difference Determine the first location of the error According to the column checksum, the difference is checked. Determine the second error location Wherein, the first error location middle, These represent the secondary block column index, the intra-block row index, and the intra-block column index, respectively; the second error location. middle, These represent the secondary block row index, intra-block row index, and intra-block column index, respectively; based on the first error location... and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection status; in response to the error detection status, the intra-block row index and intra-block column index are determined. If multiple errors are detected in both the row and column of the secondary block, the positions of elements within the block where errors may occur are obtained, and error correction is performed by setting the corresponding elements within the block to 0; the corrected second convolution result is then obtained; wherein, the second convolution result is the corrected first convolution result. Remove the remaining parts of the last row and last column of subblocks.

[0155] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the convolutional neural network fault-tolerant method provided by the above methods, the method including: based on the input first feature map matrix... Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. ;in, , The first layer of the fault-tolerant layer i Each feature map This indicates the number of feature maps input to the fault-tolerant layer. , According to the first weight matrix Calculate the second checksum , the second checksum Connected to the first weight matrix The second weight matrix is ​​obtained at the end. ;in, ), This indicates the number of weight matrices in the fault-tolerant layer; ; The first layer of the fault-tolerant layer j A weight matrix, ; Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. ;in, = , Indicates the convolution operation; determines the checksum of the first row. The second row checksum First column checksum and the second column checksum ;in, ; Second line checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; ; Second column checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is The secondary block; calculates the row checksum and the difference. Column checksums and differences ;in, , ;Based on the row checksum and the difference Determine the first location of the error According to the column checksum, the difference is checked. Determine the second error location Wherein, the first error location middle, These represent the secondary block column index, the intra-block row index, and the intra-block column index, respectively; the second error location. middle, These represent the secondary block row index, intra-block row index, and intra-block column index, respectively; based on the first error location... and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection status; in response to the error detection status, the intra-block row index and intra-block column index are determined. If multiple errors are detected in both the row and column of the secondary block, the positions of elements within the block where errors may occur are obtained, and error correction is performed by setting the corresponding elements within the block to 0; the corrected second convolution result is then obtained; wherein, the second convolution result is the corrected first convolution result. Remove the remaining parts of the last row and last column of subblocks.

[0157] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the convolutional neural network fault-tolerant method provided by the methods described above, the method comprising: based on an input first feature map matrix Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. ;in, , The first layer of the fault-tolerant layer i Each feature map This indicates the number of feature maps input to the fault-tolerant layer. , According to the first weight matrix Calculate the second checksum , the second checksum Connected to the first weight matrix The second weight matrix is ​​obtained at the end. ;in, ), This indicates the number of weight matrices in the fault-tolerant layer; ; The first layer of the fault-tolerant layer j A weight matrix, ; Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. ;in, = , Indicates the convolution operation; determines the checksum of the first row. The second row checksum First column checksum and the second column checksum ;in, ; Second line checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; ; Second column checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is The secondary block; calculates the row checksum and the difference. Column checksums and differences ;in, , ;Based on the row checksum and the difference Determine the first location of the error According to the column checksum, the difference is checked. Determine the second error location Wherein, the first error location middle, These represent the secondary block column index, the intra-block row index, and the intra-block column index, respectively; the second error location. middle, These represent the secondary block row index, intra-block row index, and intra-block column index, respectively; based on the first error location... and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection status; in response to the error detection status, the intra-block row index and intra-block column index are determined. If multiple errors are detected in both the row and column of the secondary block, the positions of elements within the block where errors may occur are obtained, and error correction is performed by setting the corresponding elements within the block to 0; the corrected second convolution result is then obtained; wherein, the second convolution result is the corrected first convolution result. Remove the remaining parts of the last row and last column of subblocks.

[0158] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fault-tolerant method for a convolutional neural network, wherein the convolutional neural network receives input data for inference, and executes the fault-tolerant method once during forward propagation for each fault-tolerant layer with fault-tolerant functionality enabled, wherein the fault-tolerant layer is a convolutional layer protected by the fault-tolerant method, characterized in that, The fault-tolerance method includes: Based on the input first feature map matrix Calculate the first checksum , the first checksum Connected to the first feature map matrix The end yields the second feature map matrix. ;in, , The first layer of the fault-tolerant layer i Each feature map This indicates the number of feature maps input to the fault-tolerant layer. , ; According to the first weight matrix Calculate the second checksum , the second checksum Connected to the first weight matrix The second weight matrix is ​​obtained at the end. ;in, ), This indicates the number of weight matrices in the fault-tolerant layer; ; The first layer of the fault-tolerant layer j A weight matrix, ; Calculate the second feature map matrix Second weight matrix The first convolution result is obtained by performing a convolution. ;in, = , This represents the convolution operation; Determine the checksum of the first row The second row checksum First column checksum and the second column checksum ;in, ; Second line checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; ; Second column checksum The first in Each element is represented as , Indicates the result of the first convolution. Bank of China code , column number is Secondary blocks; Calculate the difference between row checksums and pairs. Column checksums and differences ;in, , ; Based on the row checksum and the difference Determine the first location of the error. According to the column checksum, the difference is checked. Determine the second error location Wherein, the first error location middle, These represent the secondary block column index, the intra-block row index, and the intra-block column index, respectively; the second error location. middle, These represent the secondary block row index, the intra-block row index, and the intra-block column index, respectively. According to the first error location and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection status; In response to the error detection, the intra-block row index and intra-block column index are determined. If multiple errors are detected in both the row and column of the secondary block, the positions of the elements within the block where the errors may occur are obtained, and the errors are corrected by setting the corresponding elements within the block to 0. Obtain the error-corrected second convolution result; wherein, the second convolution result is the error-corrected first convolution result. Remove the remaining parts of the last row and last column of subblocks from the middle; The input data is an image, and the convolutional neural network is used to perform a classification task.

2. The convolutional neural network fault-tolerant method according to claim 1, characterized in that, According to the first error location and the second error location Determine intra-block row indexes and intra-block column indexes Following the error detection, the method further includes: In response to the error detection, the intra-block row index and intra-block column index are determined. If only one error is detected in a secondary block row, the value of the secondary block element with the detected error is obtained, and the difference between the secondary block element value and the column checksum is calculated. Addition for error correction; In response to the error detection, the intra-block row index and intra-block column index are determined. If only one error is detected in a secondary block column, the value of the secondary block element with the detected error is obtained, and the difference between the secondary block element value and the row checksum is calculated. Add them together to correct errors.

3. The convolutional neural network fault-tolerant method according to claim 2, characterized in that, According to the first error location and the second error location Determine the corresponding intra-block row index and intra-block column index. Error detection scenarios include: According to the first error location Building the first dictionary ; where the first dictionary The keys are the intra-block row index and the intra-block column index. The first dictionary The key value is the secondary block column index. ; According to the second error location Create a second dictionary ; where the second dictionary The keys are the intra-block row index and the intra-block column index. The second dictionary The key value is the secondary block row index. ; In response to iterating through the first dictionary and the second dictionary Knowing the intra-block row index and intra-block column index Corresponding secondary block row index If unique, then the intra-block row index and intra-block column index are determined. Only one error was detected on the secondary block line; In response to iterating through the first dictionary and the second dictionary Knowing the intra-block row index and intra-block column index Corresponding secondary block column index If unique, then the intra-block row index and intra-block column index are determined. Only one error was detected on the secondary block column; In response to iterating through the first dictionary and the second dictionary Knowing the intra-block row index and intra-block column index Corresponding secondary block row index and secondary block column indexes If neither is unique, then determine the intra-block row index and intra-block column index. Multiple errors were detected in both the secondary block row and the secondary block column.

4. The convolutional neural network fault-tolerant method according to claim 1, characterized in that, According to the input first feature map matrix Calculate the first checksum Previously, the method also included: For the batch-normalized fault-tolerant layer input, obtain the first feature map matrix. The feature values ​​that are not within the preset error range of the mean of the batch normalization result are truncated to a preset value within the preset error range.

5. The convolutional neural network fault-tolerant method according to claim 4, characterized in that, The batch normalization result approximately satisfies a normal distribution, and the preset error range is... , , Let represent the mean and standard deviation of the batch normalization result, respectively. Represents a preset integer; When the eigenvalue is less than the mean of the batch normalization result, the preset value is... When the feature value is greater than the mean of the batch normalization result, the preset value is... Alternatively, the preset value can be uniformly set to the mean of the batch normalization results.

6. The convolutional neural network fault-tolerant method according to claim 1, characterized in that, The difference is calculated based on the row checksum. Determine the first location of the error. Previously, the method also included: The row checksum and the difference The elements in the list are traversed in response to the existence of at least one element whose value is not in the list. Within the range, then perform the operation based on the row checksum and the difference. Determine the first location of the error. The action; among them, , These represent the pre-obtained row check and the difference value, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer; The difference is calculated based on the column checksum. Determine the second error location Previously, the method also included: The column checksum and the difference The elements in the list are traversed in response to the existence of at least one element whose value is not in the list. Within the range, then perform the checksum and difference calculation based on the column. Determine the second error location The action; among them, , These represent the pre-obtained column validation and the difference values, respectively. The mean and standard deviation of the elements in the middle. Represents a preset integer.

7. The convolutional neural network fault-tolerant method according to claim 6, characterized in that, The row checksum and the difference Before iterating over the elements in the array, the method further includes: Replace all convolutional layers of the convolutional neural network with the fault-tolerant layer, and set a fault-tolerant control switch for each fault-tolerant layer to control whether the fault-tolerant function is enabled. Under error-free conditions, the replaced convolutional neural network is used to perform inference on the test set data, and the row checksum difference is calculated. mean of elements and standard deviation And calculate the difference between the column checksums. mean of elements and standard deviation .

8. The convolutional neural network fault-tolerant method according to claim 7, characterized in that, Before the convolutional neural network receives input data for inference, the method further includes: The fault tolerance control switch of each fault tolerance layer is turned on or off based on the planning results of the number of fault tolerance layers and the distribution of the fault tolerance layers. The planning results for the distribution of fault-tolerant layers include prioritizing the activation of the fault-tolerant control switches for fault-tolerant layers with larger convolutional kernel sizes; for fault-tolerant layers of the same priority, the fault-tolerant control switches are activated uniformly according to the network depth.

9. The convolutional neural network fault-tolerant method according to claim 7, characterized in that, After obtaining the error-corrected second convolution result, the method further includes: In response to the cycle time of reaching the test phase execution, inference is performed using the test set and the accuracy is calculated, and the bit error rate is calculated based on the accuracy. In response to the bit error rate being greater than a first threshold, the fault tolerance control switch for more fault tolerance layers is activated; In response to the bit error rate being less than a second threshold, the fault tolerance control switch for more fault tolerance layers is turned off; wherein the second threshold is less than the first threshold; In response to the bit error rate being less than or equal to the first threshold and greater than or equal to the second threshold, the setting of the fault tolerance control switch of the fault tolerance layer remains unchanged.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the convolutional neural network fault-tolerant method as described in any one of claims 1 to 9.

Images