Neural network acceleration core for reducing single pixel error and intelligent driving chip
By designing a neural network acceleration core with parallel computing and polling verification modules in the intelligent driving chip, combined with software logic judgment, the problem of traffic accidents caused by single pixel errors in high-level autonomous driving systems is solved, achieving efficient and safe target object recognition and fault coverage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 北京辉羲智能科技有限公司
- Filing Date
- 2024-11-04
- Publication Date
- 2026-05-08
AI Technical Summary
In advanced autonomous driving systems, a single pixel error caused by random hardware failure may lead to a traffic accident. Existing technologies are unable to effectively identify and deal with both temporary and permanent failures.
A neural network acceleration core design is adopted to reduce single-pixel errors. Through parallel computing modules and polling verification modules, the consistency of computing modules is ensured. Combined with software logic judgment, temporary and permanent failures are covered, reducing target object recognition failures caused by single-pixel errors.
It achieves efficient and safe target object recognition in high-level autonomous driving systems, reduces the risk of traffic accidents caused by single pixel errors, and meets the requirements of high safety and high efficiency in intelligent driving.
Smart Images
Figure CN121997995A_ABST
Abstract
Description
[Technical Field]
[0001] This invention relates to the field of automotive electronics technology, specifically to a neural network acceleration core and intelligent driving chip for reducing single-pixel errors. [Background Technology]
[0002] Automotive electronic chips are designed to achieve high reliability and comply with the functional safety standard ISO 26262. ISO 26262 is an international functional safety standard specifically for electrical and / or electronic systems in road vehicles, including those involving safety-related functions.
[0003] Chips inevitably experience random hardware failures due to environmental factors and aging. These failures can result in errors in a single pixel, such as a logic error in an adder or multiplier. Even a change in a single pixel can significantly impact image recognition and processing systems, potentially leading to incorrect image recognition or processing.
[0004] For autonomous driving, the industry has conducted some fault injection experiments. Injecting single-pixel errors into different layers of the neural network may change the number, position, or size of targets after the error is injected, affecting target detection and distance judgment.
[0005] In advanced autonomous driving systems, an error caused by a single pixel can lead to a traffic accident.
[0006] There are two existing classic neural network accelerator safety designs.
[0007] One approach is to implement it through the logic part DCLS (Dual-core Lockstep Architecture). However, since neural network computation requires a lot of computing power, the implementation of DCLS results in too much loss in terms of power consumption and chip area.
[0008] Secondly, it is identified through online BIST. However, online BIST can only identify permanent failures and cannot identify temporary failures. Furthermore, online BIST places high demands on software scheduling and chip design. BIST, on the other hand, is a technique that embeds relevant functional circuits into the circuit during the design phase to provide self-test capabilities, thereby reducing the reliance on automated test equipment (ATE) for device testing.
[0009] Temporary malfunctions in high-level autonomous driving systems, even errors caused by a single pixel, can lead to traffic accidents. In one Uber accident, the US National Telecommunications Standards Board (NTSB) released an investigation report detailing the timeline leading up to the collision. For example, 2.7–3.8 seconds before the collision, the system's identification of the pedestrian fluctuated between "vehicle" and "other"; 2.6 seconds before the collision, the system identified the pedestrian and her bicycle as a "bicycle"; then 1.5 seconds before the collision, the pedestrian was identified as "unknown"; and 1.2 seconds before the collision, the identification returned to "bicycle." This back-and-forth switching ultimately led to the accident.
[0010] This invention addresses the technical problem in high-level autonomous driving chips where random hardware failures lead to logic errors in adders and multipliers, which in turn cause single-pixel errors and traffic accidents. It provides technical improvements to neural network acceleration cores and intelligent driving chips. [Summary of the Invention]
[0011] The purpose of this invention is to provide an efficient and safe internal neural network acceleration core for intelligent driving main control chips that addresses the problem of random hardware failures causing logic errors in adders and multipliers, which in turn lead to single-pixel errors and traffic accidents.
[0012] To achieve the above objectives, the technical solution adopted by the present invention is a neural network acceleration kernel that reduces single-pixel errors. It includes several fast calculation modules (Cal) for implementing multiplication-then-addition convolution. The fast calculation modules (Cal) are divided into m groups of calculation kernels. Each group of calculation kernels includes n fast calculation modules (Cal) and a polling verification calculation module (Cal pair). The polling verification calculation module (Cal pair) is used to take turns inputting the same value to the n fast calculation modules (Cal) in the same group and comparing the calculation results to ensure consistency, thereby ensuring that the fast calculation modules (Cal) are error-free and preventing permanent failure of the fast calculation modules (Cal).
[0013] Preferably, each set of computing cores further includes a first switcher for switching the input of the same set of fast computing modules Cal to the input of the polling verification computing module Cal pair in turn during each clock cycle, and a second switcher for switching the comparison between the output of the same set of fast computing modules Cal and the output of the polling verification computing module Cal pair in turn during each clock cycle.
[0014] Preferably, the configuration ratio of the fast calculation module Cal and the polling verification calculation module Cal pair in each set of calculation cores is N:1, N<=FDTI / F, where FDTI is the fault detection interval that meets the road vehicle functional safety standard ISO26262, and F is the image acquisition time per frame of the image processed by the neural network acceleration core.
[0015] Preferably, the polling verification calculation module Cal pair does not perform comparison verification with the fast calculation module Cal, which is set to be security-independent.
[0016] Preferably, the neural network acceleration core for reducing single-pixel errors further includes a separate verification calculation module, which inputs the same value to the fast calculation module Cal, which needs to identify temporary failures, in each clock cycle and compares whether the calculation results are consistent, so as to ensure that the fast calculation module Cal has no errors and avoid permanent failure of the fast calculation module Cal.
[0017] Another objective of this invention is to provide a highly efficient and safe intelligent driving main control chip that addresses the problem of random hardware failures leading to logic errors in adders and multipliers, which in turn can result in single-pixel errors and traffic accidents.
[0018] To achieve the above-mentioned objective, the technical solution adopted by the present invention is an intelligent driving chip that reduces single-pixel errors, which includes the aforementioned neural network acceleration core for reducing single-pixel errors.
[0019] Preferably, the intelligent driving chip for reducing single-pixel errors includes a processor, wherein the processor executes the intelligent driving program to address convolutional layer failure caused by a single-pixel error, comprising the following steps:
[0020] S1. Use convolution to extract feature values for recognition, without considering the possibility of the same error occurring twice.
[0021] S2, A is detected;
[0022] S3, B is identified;
[0023] S4. Select A or B, which identifies a low level of safety hazard.
[0024] Preferably, when the processor executes the intelligent driving program to address convolutional layer failure caused by a single pixel error in a target object recognition scenario, the process includes the following steps:
[0025] S1. Use convolution to extract feature values for target recognition, without considering the possibility of the same error occurring twice.
[0026] S2. Unexpected target object detected;
[0027] S3. No target object detected;
[0028] S4. Select and identify unexpected targets.
[0029] Preferably, when the processor executes the intelligent driving program to handle convolutional layer failure caused by a single pixel error in a traffic light scenario, it includes the following steps:
[0030] S1. Use convolution to extract feature values for traffic light recognition, without considering the possibility of the same error occurring twice.
[0031] S2, Red light detected;
[0032] S3, Green light detected;
[0033] S4, Select "Red light detected".
[0034] The beneficial effects of this invention, a neural network acceleration core and intelligent driving chip for reducing single-pixel errors, are as follows: By using two methods together, the target object recognition failure caused by a single pixel in high-level intelligent driving is reduced, thereby reducing the possibility of traffic accidents. 1. The neural network acceleration core used in the main control chip of a high-reliability intelligent driving controller is divided into multiple hardware acceleration units to achieve parallel computation of multiplication and addition, meeting high safety requirements while being highly efficient, and capable of covering and dealing with permanent hardware failures; 2. It provides a low-hazard principle that, under uncertainty, regardless of which of the two possibilities, A or B, is correct, selects the low-hazard method to deal with temporary failures. The judgment principle of ensuring safety through software logic judgment covers temporary failures, thereby dealing with the potential road hazards caused by convolutional layer failures due to single-pixel errors in intelligent driving. [Attached Image Description]
[0035] Figure 1 This is a schematic diagram illustrating the principle of convolution calculation.
[0036] Figure 2 This is a schematic diagram of a neural network acceleration kernel that reduces single-pixel errors.
Detailed Implementation Methods
[0037] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific configurations and algorithms presented below, but covers any modifications, substitutions, and improvements to elements, components, and algorithms without departing from the inventive concept. In the accompanying drawings and the following description, well-known structures and techniques are not shown in order to avoid unnecessarily obscuring the invention.
[0038] Example 1
[0039] This embodiment implements a neural network acceleration kernel that reduces single-pixel errors.
[0040] This embodiment provides a chip hardware design that can cover permanent failures, and a method that combines software logic judgment to ensure safety and cover temporary failures, in order to address the potential road hazards caused by convolutional layer failures due to single pixel errors in intelligent driving.
[0041] Principle 1: Parallel computation of convolution is implemented using chip hardware to extract feature values.
[0042] Figure 1 This is a schematic diagram illustrating the principle of convolution calculation. (For example...) Figure 1 As shown, the function to be achieved by the convolutional layer of the intelligent driving neural network is to extract feature values using the principle of convolution. The function to be achieved is to extract feature values quickly using parallel computing. Since the target to be extracted in intelligent driving is composed of different kernels, a large number of convolution calculations need to be completed.
[0043] In this embodiment, the acceleration core has multiple fast calculation modules Cal to accelerate the calculation of convolutional layers in intelligent driving neural networks. It also has a Cal pair polling and verification fast calculation module Cal. The configuration ratio of Cal and Cal pair is N:1, where N<=15.
[0044] Figure 2 This is a schematic diagram of a neural network acceleration kernel that reduces single-pixel errors. (Example) Figure 2 As shown, the acceleration core (hardware part) in this embodiment has many fast calculation modules Cal, which are used to realize the fast calculation of convolution by multiplication followed by addition.
[0045] Preferably, in addition to the fast calculation of the modules Cal0-Caln, there is also a Cal pair, which is compared with a Calx that calculates the same value to ensure that the Calx module is error-free, thereby avoiding permanent failure in Calx.
[0046] In this embodiment, the acceleration core has a switch that can control the Cal pair to be verified with one of the Calx values from Cal0 to Caln within one clock cycle. In the next clock cycle, the Cal pair can be verified with Calx+1. The configuration ratio of Cal to Cal pair is N:1, where N<=15.
[0047] One image typically takes 16.66ms, and polling and comparing all cal modules results in an FDTI < 250ms (15 x 16.66ms). FDTI (Fault Detection Interval) is an important time parameter in functional safety, especially in ISO 26262 (Road Vehicle Functional Safety Standard). It refers to the time interval from the occurrence of an internal system fault to its detection or identification by the system.
[0048] Preferably, depending on the application, it can be set which Cal pairs are security-related and which are security-independent. For the security-independent parts, Cal pair verification is not required.
[0049] Preferably, for some Cal computing units that need to identify temporary failures, a separate Cal pair can be configured, and comparison and verification can be performed in each clock cycle.
[0050] The acceleration core (hardware design) in this embodiment provides a highly flexible and reliable configuration foundation for the algorithm software.
[0051] Example 2
[0052] This embodiment implements an intelligent driving chip that reduces single-pixel errors.
[0053] This embodiment of the intelligent driving chip includes a chip hardware design from Embodiment 1 that can cover permanent failures. Combined with the intelligent driving algorithm (software) logic judgment principle of this embodiment to ensure safety, it covers temporary failures to deal with the potential road hazards caused by convolutional layer failure due to a single pixel error in intelligent driving.
[0054] Principle 2: The feature values extracted by convolution are used for target object recognition. However, due to random hardware failure, a single-bit temporary fault occurs, leading to a calculation error. The resulting failure effect is: 1. The target object is unexpectedly identified, or 2. The target object is not identified. Since it is a temporary fault, we do not consider the same error occurring twice. Therefore, when a temporary fault occurs, there will only be one calculation that identifies the target object and one calculation that does not identify the target object.
[0055] In such situations, the unexpected detection of a target object results in deceleration. Since autonomous driving systems avoid excessive, unexpected deceleration—meaning deceleration remains within a reasonable range to prevent rear-end collisions—and brake lights illuminate during deceleration, decelerating within a reasonable range is safer. Unexpectedly detecting a target object is safer than losing it. Therefore, to address temporary failures, if a target object is unexpectedly detected once and not detected once, it is assumed to exist.
[0056] The acceleration core (hardware part) in Example 1 is a module inside the intelligent driving chip of this embodiment. For this intelligent driving chip, there is also an AoU (using assumptions; AoU is defined as an Assumption of Use in ISO 26262). Since the Cal pair of the acceleration core in Example 1 cannot detect all temporary failures when using polling mode, the intelligent driving algorithm on this intelligent driving chip needs to set the following rules for single-bit temporary failures when using polling with this Cal pair. The principle of setting the rules is that when the calculation results of two consecutive frames are opposite, their failure effect is analyzed, and the calculation result with the smaller failure effect is selected.
[0057] The specific implementation is as follows:
[0058] 1. In target object recognition scenarios
[0059] Due to a random hardware failure, a single-bit temporary fault occurred, leading to a calculation error. However, the resulting failure effect is: 1. the target object is unexpectedly identified, and 2. the target object is not identified. Since it is a temporary fault, we do not consider the same error occurring twice. Therefore, when a temporary fault occurs, there will only be one calculation that identifies the target object and one calculation that does not identify the target object.
[0060] In such situations, the unexpected detection of a target object results in deceleration. Since autonomous driving systems avoid excessive, unexpected deceleration—meaning deceleration is always within a reasonable range—to prevent rear-end collisions, and brake lights will illuminate during deceleration. Therefore, decelerating unexpectedly within a reasonable range is safer. Unexpectedly detecting a target object is safer than losing the target object. Therefore, to address temporary failures, if a target object is detected once and not detected once, it is assumed to exist.
[0061] 2. In a traffic light scenario
[0062] Due to a random hardware failure, a single-bit temporary fault occurred, resulting in a calculation error. However, the resulting failure effect is: 1. a red light is judged as a green light, and 2. a green light is judged as a red light.
[0063] Since a green light is interpreted as a red light, vehicles will not move forward and will not cause a traffic accident. Therefore, when two consecutive calculations show one as a red light and one as a green light, it is judged as a red light.
[0064] The following is a hypothetical scenario: if the color is red-green-red-red-red, the output will be red-red-red-red-red.
[0065] If the output is red-green-green-green, then the output will be red-red-green-green-green. Since the calculation time is generally on the order of tens of milliseconds, the user will not notice it.
[0066] In summary, the intelligent driving chip in this embodiment is:
[0067] 1. Provide a chip computing core that meets high security and high efficiency to cope with permanent hardware failure;
[0068] 2. Provide a method based on the principle of low risk, in situations of uncertainty, to choose the lowest risk approach to deal with temporary failures, regardless of which of the two possibilities, A or B, is correct.
[0069] 3. By combining the two methods mentioned above, we can reduce the target object recognition failures caused by single pixels in high-level intelligent driving, thereby reducing the occurrence of traffic accidents.
[0070] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).
[0071] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and additions without departing from the principle of the present invention, and these improvements and additions should also be considered within the scope of protection of the present invention.
Claims
1. A neural network acceleration kernel for reducing single-pixel errors, comprising several fast computation modules Cal for implementing multiplication-then-addition convolution, characterized in that: The fast calculation module Cal is divided into m groups of calculation cores; each group of calculation cores includes n fast calculation modules Cal and a polling verification calculation module Cal pair. The polling verification calculation module Cal pair is used to take turns inputting the same value to the n fast calculation modules Cal in the same group and comparing whether the calculation results are consistent, so as to ensure that the fast calculation module Cal has no errors and avoid the fast calculation module Cal from becoming permanently invalid.
2. The neural network acceleration kernel for reducing single-pixel errors according to claim 1, characterized in that: Each set of computing cores also includes a first switcher for switching the input of the same set of fast computing modules Cal to the input of the polling verification computing module Calpair in turn every clock cycle, and a second switcher for switching the output of the same set of fast computing modules Cal and the output of the polling verification computing module Calpair in turn every clock cycle.
3. The neural network acceleration kernel for reducing single-pixel errors according to claim 2, characterized in that: The configuration ratio of the fast calculation module Cal and the polling verification calculation module Cal pair in each set of computing cores is N:1, where N<=FDTI / F, where FDTI is the fault detection interval that meets the road vehicle functional safety standard ISO26262, and F is the image acquisition time per frame of the image processed by the neural network acceleration core.
4. The neural network acceleration kernel for reducing single-pixel errors according to claim 2, characterized in that: The polling verification calculation module Cal pair does not perform comparison verification with the fast calculation module Cal, which is set to be security-independent.
5. The neural network acceleration kernel for reducing single-pixel errors according to claim 2, characterized in that: It also includes a separate verification calculation module, which inputs the same value to the fast calculation module Cal, which needs to identify temporary failures, every clock cycle and compares the calculation results to ensure that the fast calculation module Cal has no errors and avoids permanent failure of the fast calculation module Cal.
6. A smart driving chip that reduces single-pixel errors, characterized in that: Internally, it includes a neural network acceleration core for reducing single-pixel errors as described in any one of claims 1 to 5.
7. The intelligent driving chip for reducing single-pixel errors according to claim 6, characterized in that... The processor, when executing an intelligent driving program to respond to convolutional layer failure caused by a single pixel error, includes the following steps: S1. Use convolution to extract feature values for recognition, without considering the possibility of the same error occurring twice. S2, A is detected; S3, B is identified; S4. Select A or B, which identifies a low level of safety hazard.
8. The intelligent driving chip for reducing single-pixel errors according to claim 7, characterized in that, When the processor executes the intelligent driving program to address convolutional layer failure caused by a single pixel error in an object recognition scenario, the following steps are included: S1. Use convolution to extract feature values for target recognition, without considering the possibility of the same error occurring twice. S2. Unexpected target object detected; S3. No target object detected; S4. Select and identify unexpected targets.
9. The intelligent driving chip for reducing single-pixel errors according to claim 7, characterized in that, When the processor executes the intelligent driving program to handle convolutional layer failure caused by a single pixel error in a traffic light scenario, the following steps are included: S1. Use convolution to extract feature values for traffic light recognition, without considering the possibility of the same error occurring twice. S2, Red light detected; S3, Green light detected; S4, Select "Red light detected".