Calculation system

Abstract

In a computing system, it is possible to suppress the occurrence of a waste in a system configuration and to detect the presence or absence of an abnormality. A computing system (100) that executes computing processing using a neural network model is provided with: a plurality of processor cores (PC1-PC8); a calculation allocation unit (11) that determines a calculation kernel on the basis of the calculation load of the calculation process and causes the calculation kernel to execute the calculation process; and an abnormality detection unit (12) that detects the presence or absence of an abnormality in the processor core. The operation allocation unit causes at least some of the processor cores, which are not the operation cores, to execute the abnormality detection operation in parallel with the basic operation. The abnormality detection unit determines that there is no abnormality in the calculation core and the abnormality detection core when the difference between a first calculation result, which is a calculation result of the basic calculation, and a second calculation result, which is a calculation result of the abnormality detection calculation, satisfies a predetermined permission condition, and determines that there is no abnormality in the calculation core and the abnormality detection core when the difference does not satisfy the permission condition. It is determined that there is an abnormality in at least some of the operation core and the abnormality detection core.

Description

Technical Field

[0001] The present disclosure relates to a computing system. Background Art

[0002] In the past, in order to ensure the functional safety of a computing system equipped with a processor, a method for detecting an abnormality when an abnormality occurs in the computing system is required. Patent document 1 discloses a system that has a processor core for abnormality detection in addition to a processor core used in normal times, and compares the calculation results of the two processor cores. In addition, patent document 2 discloses a system that compares the calculation results of the processor core at system startup with the calculation results stored in advance in a memory, in order to detect abnormalities in the system.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: U.S. Patent No. 6065135

[0006] Patent Document 2: U.S. Patent Application Publication No. 2006 / 0062046 Summary of the invention

[0007] The same structure can also be applied to a computing system with an NPU (Neural Network Processing Unit) that performs computing processing using a neural network. However, in Patent Document 1, since it has a processor core for abnormality detection in addition to the processor core used in normal times, there is a problem of waste in the system structure. In addition, in Patent Document 2, only the computing results pre-stored in the memory can be compared when the system is started, and there is a problem that abnormalities cannot be detected for newly output computing results.

[0008] The present disclosure can be implemented in the following ways.

[0009] According to one embodiment of the present disclosure, there is provided a computing system that performs computing processing using a neural network model. The computing system includes: a plurality of processor cores that perform the computing processing; a computing allocation unit that determines a processor core that performs the computing processing, i.e., a computing core, among the plurality of processors based on the computing load of the computing processing, and causes the computing core to perform the computing processing; and an abnormality detection unit that detects whether the processor core has an abnormality; the computing allocation unit causes at least a portion of the processor cores that are not the computing cores among the plurality of processors to perform a computing obtained based on the basic computing, i.e., an abnormality detection computing, in parallel with the computing processing, i.e., a basic computing, performed by the computing core; the abnormality detection unit determines that the computing core and the processor core that performs the abnormality detection computing, i.e., the abnormality detection core, have no abnormality when the difference between a first computing result that is a computing result of the basic computing and a second computing result that is a computing result of the abnormality detection computing satisfies a predetermined permission condition, and determines that at least a portion of the computing core and the abnormality detection core have an abnormality when the difference does not satisfy the permission condition.

[0010] According to the control device of this method, the operation cores among the plurality of processors are determined according to the operation load of the operation processing, and it is determined whether the operation result of the basic operation executed in the operation core is consistent with the operation result of the abnormality detection operation executed in the plurality of abnormality detection cores, thereby detecting the presence or absence of abnormality. Therefore, compared with a structure having a processor core for abnormality detection in addition to a processor core used in normal times, it is possible to suppress waste in the system structure and detect the presence or absence of abnormality. In addition, the presence or absence of abnormality can be detected using the newly output operation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a block diagram showing a schematic configuration of a computing system according to the present embodiment.

[0012] Figure 2 This is an explanatory diagram showing an example of the structure of a neural network model.

[0013] Figure 3 This is a flowchart showing the procedure of the abnormality detection process according to the first embodiment.

[0014] Figure 4 This is a flowchart showing the procedure of the abnormality detection process according to the first embodiment.

[0015] Figure 5 This is an explanatory diagram showing an example of allocation of calculation processing in the first embodiment.

[0016] Figure 6This is a flowchart showing the procedure of the abnormality detection process according to the second embodiment.

[0017] Figure 7 This is a flowchart showing the procedure of the abnormality detection process according to the second embodiment.

[0018] Figure 8 It is an explanatory diagram showing an example of allocation of calculation processing in the second embodiment.

[0019] Fig. 9 This is a flowchart showing the procedure of the abnormality detection process according to the third embodiment.

[0020] Fig.10 It is an explanatory diagram showing an example of allocation of calculation processing in the third embodiment.

[0021] Fig.11 This is an explanatory diagram showing an example of allocation of arithmetic processing in another embodiment. DETAILED DESCRIPTION

[0022] A. First Implementation Method:

[0023] A-1. System structure:

[0024] The operation system 100 of this embodiment uses a neural network model to perform operation processing. In this embodiment, the operation system 100 uses the image data captured by the camera mounted on the vehicle as an object and performs operation processing for identifying pedestrians, structures, etc. contained in the image data. In this embodiment, the neural network model is configured as a multi-category classifier, which outputs the probability that each object contained in the image data belongs to the category for each category in a plurality of predetermined categories based on the image data. In addition, the operation system 100 can also perform operation processing on sound data and text data as objects, not limited to image data.

[0025] Figure 2 An example of a neural network model is shown in . Figure 2 The neural network model shown has a convolution layer L1, a pooling layer L2, a convolution layer L3, a pooling layer L4, and a fully connected layer L5. The operation system 100 performs operations in each layer in this order. In addition, the operation system 100 can perform operation processing using a neural network model of any structure, and is not limited to Figure 2 The neural network model of the structure shown. In addition, the operation system 100 can perform operation processing that uses a plurality of neural network models separately in sequence, and is not limited to the operation processing that uses a single type of neural network model.

[0026] like Figure 1As shown, the operation system 100 includes processor cores PC1 to PC8, an operation control unit 10, and a semiconductor storage device 20. The processor cores PC1 to PC8, the operation control unit 10, and the semiconductor storage device 20 are connected to each other via a bus BS so as to be able to communicate with each other. In addition to the above-mentioned structure, the operation system 100 may also include a cache memory, a register, etc.

[0027] In the present embodiment, the processor cores PC1 to PC8 are configured as an NPU (Neural Network Processing Unit) that performs computational processing using a neural network model. The specific structure of the processor cores PC1 to PC8 is the same as that of a known NPU, so a detailed description is omitted. In the present embodiment, the processor cores PC1 to PC8 are input with a single image data, and different regions in the image data are shared by a plurality of processor cores, and computational processing is performed in parallel with each other. In addition, any one of the processor cores PC1 to PC8 can also perform computational processing on a single image data.

[0028] The number of processor cores included in the computing system 100 is not limited to eight, but may be any number greater than two, as long as the computing processing with the largest computing load pre-conceived can be completed in the computing system 100 within the predetermined allowed time. The "allowed time" is set, for example, to the upper limit of the time allowed for distinguishing pedestrians, structures, etc. that exist around the vehicle. In the following description, the computing load of computing processing that requires all processor cores to operate in parallel in order to complete the computing processing within the allowed time is referred to as the "maximum computing load". In addition, the "computational load of computing processing" is determined according to the type of neural network model used in the computing processing, more specifically, the number of layers that constitute the neural network model, the type of operations performed in each layer, etc.

[0029] The calculation control unit 10 includes a calculation allocation unit 11 and an abnormality detection unit 12. The calculation allocation unit 11 determines the processor core (hereinafter also referred to as "calculation core") that performs the calculation processing among the processor cores PC1 to PC8 according to the calculation load of the calculation processing, and causes the calculation core to perform the calculation processing. In the following description, the calculation processing performed in the calculation core is also referred to as "basic calculation". In the present embodiment, the calculation allocation unit 11 determines the minimum number of processor cores (hereinafter also referred to as "minimum number of cores") required to complete the basic calculation within the allowed time as the calculation core. The minimum number of cores is preset according to the type of basic calculation to be performed.

[0030] In addition, the operation allocation unit 11 causes the processor cores that are not operation cores among the processor cores PC1 to PC8 to perform an operation for abnormality detection. "Anomaly detection operation" is an operation process performed to detect whether there is an abnormality in the processor core, and refers to an operation process obtained based on the basic operation. The operation allocation unit 11 determines the number of processor cores (hereinafter also referred to as "abnormality detection cores") that perform abnormality detection operations based on the operation load of the basic operation. In the present embodiment, the operation allocation unit 11 causes the abnormality detection core to perform the same operation as the basic operation as the abnormality detection operation. The specific method of allocating the abnormality detection operation to the abnormality detection core by the operation allocation unit 11 will be described in detail later.

[0031] The abnormality detection unit 12 detects whether the processor core has an abnormality by determining whether the difference between the calculation result of the basic operation and the calculation result of the abnormality detection operation satisfies a predetermined permission condition. In addition, the abnormality detection unit 12 can be a functional unit implemented in software by executing a program, or a functional unit implemented in hardware by a logic circuit. The specific processing of the abnormality detection unit 12 will be described later.

[0032] The semiconductor storage device 20 stores various data input and output via the bus BS in the calculation processing executed by the processor cores PC1 to PC8, such as image data, coefficients used in the calculation processing, calculation results, etc. In the present embodiment, the semiconductor storage device 20 is configured as a DRAM (Dynamic Random Access Memory).

[0033] A-2. Abnormal detection and processing:

[0034] The calculation control unit 10 executes Figure 3 and Figure 4 The abnormality detection process shown is parallel to the basic operation of the operation core, and the abnormality detection core performs the abnormality detection operation according to the operation load of the basic operation to detect whether the processor core has an abnormality. The abnormality detection process is executed every time the basic operation is started in the operation system 100. In addition, in the following description, the operation process to be executed is based on image data.

[0035] exist Figure 3 In step S10 shown, the calculation allocation unit 11 determines whether the calculation load of the basic calculation is less than 1 / 2 of the maximum calculation load. When it is determined that the calculation load is greater than 1 / 2 of the maximum calculation load (step S10: No), the calculation allocation unit 11 executes the basic calculation in the calculation core and ends the abnormality detection process. That is, in this case, the abnormality detection unit 12 does not perform abnormality detection.

[0036] On the other hand, when it is determined that the calculation load is less than 1 / 2 of the maximum calculation load (step S10: Yes), the calculation allocating unit 11 Figure 4 In step S20 shown, it is determined whether the calculation load of the basic calculation is less than 1 / 3 of the maximum calculation load.

[0037] When it is determined that the calculation load is less than 1 / 3 of the maximum calculation load (step S20: Yes), in step S30, the calculation allocation unit 11 allocates calculation processing to the processor core so that in addition to executing the basic calculation, it also executes two calculation processings identical to the basic calculation in parallel as abnormality detection calculations.

[0038] Reference Figure 5 The example shown is a more specific description of the allocation of the calculation process in step S30. Figure 5 In the example shown, the minimum number of cores for the basic operation is set to 2. In this case, the operation allocation unit 11 allocates the basic operation to the processor cores PC1 and PC2, allocates the first abnormality detection operation to the processor cores PC3 and PC4, and allocates the second abnormality detection operation to the processor cores PC5 and PC6. Figure 5 In the example shown, the processor cores PC1 and PC2 correspond to the calculation cores, and the processor cores PC3 and PC4 and the processor cores PC5 and PC6 correspond to the abnormality detection cores.

[0039] exist Figure 5 In the example shown, the calculation allocation unit 11 does not allocate calculation processing to the processor cores PC7 and PC8. In the configuration in which calculation processing is not allocated to the processor cores PC7 and PC8, it is possible to suppress an increase in power consumption and processing load in the calculation system 100, compared with a configuration in which calculation processing is allocated to the processor cores PC7 and PC8. Alternatively, the calculation allocation unit 11 may allocate calculation processing to the processor cores PC7 and PC8.

[0040] exist Figure 4 In step S40 shown in FIG. 1 , the abnormality detection unit 12 obtains three intermediate operation results of the basic operation and the two abnormality detection operations output in parallel. The "intermediate operation result" refers to the operation result in the intermediate layer among the multiple layers constituting the neural network model. Figure 2 In the example shown, the convolution layer L1, the pooling layer L2, the convolution layer L3, and the pooling layer L4 are equivalent to the intermediate layers. In this embodiment, the calculation result in the pooling layer L2 is obtained as the intermediate calculation result. In addition, the abnormality detection unit 12 can also obtain the calculation results in two or more intermediate layers as the intermediate calculation result, rather than being limited to one of the multiple intermediate layers. Figure 5In the example shown, the basic operation and the two abnormality detection operations are respectively executed on the same number of processor cores, so that the intermediate operation results for mutually corresponding pixels are output in parallel with each other.

[0041] The abnormality detection unit 12 obtains each intermediate calculation result each time the intermediate calculation results of the basic calculation and the two abnormality detection calculations are output in parallel for the corresponding pixels. More specifically, the abnormality detection unit 12 directly obtains each intermediate calculation result output from the calculation core and the abnormality detection core. In addition, the abnormality detection unit 12 may also obtain each intermediate calculation result output from the calculation core and the abnormality detection core and temporarily stored in the semiconductor storage device 20 from the semiconductor storage device 20.

[0042] exist Figure 4 In step S50 shown in the figure, as a permission condition, the abnormality detection unit 12 determines whether the three intermediate operation results of the basic operation and the two abnormality detection operations are consistent with each other. Since the abnormality detection unit 12 uses the intermediate operation results for determination, it does not need to wait for the output of the final operation result described later, and can detect the abnormality in the processor core at an early stage, which can suppress the increase in the time required for the abnormality detection process.

[0043] The abnormality detection unit 12 may perform the determination in step S50 with respect to all pixels in the image data as objects, or may perform the determination in step S50 with respect to a predetermined portion of pixels in the image data as objects. In the method of performing the determination with respect to all pixels as objects, the presence or absence of abnormality can be detected with higher accuracy. On the other hand, in the method of performing the determination with respect to a portion of pixels as objects, the number of times the determination is performed can be suppressed, thereby suppressing the increase in the processing load and processing time in the abnormality detection unit 12. As to which determination method to adopt, it can be arbitrarily determined in consideration of the length of the above-mentioned allowed time, the required accuracy of abnormality detection, etc.

[0044] When it is determined that the three intermediate calculation results are inconsistent with each other (step S50: No), in step S84, the abnormality detection unit 12 determines that at least a part of the calculation core and the abnormality detection core has an abnormality. "The three intermediate calculation results of the basic calculation and the two abnormality detection calculations are inconsistent with each other" means that at least one of the three intermediate calculation results is inconsistent with the other intermediate calculation results. Since the abnormality detection unit 12 determines whether the three intermediate calculation results are consistent with each other, when the three intermediate calculation results are inconsistent with each other, it is possible to determine the processor core with a higher probability of abnormality. For example, in Figure 5In the example shown, when the intermediate operations of the first abnormality detection operation and the second abnormality detection operation are consistent with each other and only the intermediate operation results of the basic operation are different, it can be determined that the possibility of an abnormality occurring in the processor core PC1 or the processor core PC2 is high. After step S84 is completed, the abnormality detection process ends.

[0045] On the other hand, Figure 4 In the case where it is determined that the three intermediate operation results are consistent with each other (step S50: Yes), in step S60, the abnormality detection unit 12 obtains the three final operation results of the basic operation and the two abnormality detection operations output in parallel. The "final operation result" refers to the operation result in the last layer of the multiple layers constituting the neural network model. Figure 2 In the example shown, L5 corresponds to the last layer. For the same reason as in the case of the intermediate calculation results described above, the final calculation results for the pixels corresponding to each other are output in parallel with each other.

[0046] As in the case of the above-mentioned intermediate calculation results, the abnormality detection unit 12 obtains each final calculation result each time the final calculation results of the basic calculation and the two abnormality detection calculations are output in parallel for the corresponding pixels. More specifically, the abnormality detection unit 12 directly obtains each final calculation result output from the calculation core and the abnormality detection core. In addition, the abnormality detection unit 12 may also obtain each final calculation result output from the calculation core and the abnormality detection core and temporarily stored in the semiconductor storage device 20 from the semiconductor storage device 20.

[0047] exist Figure 4 In step S70 shown, as a permission condition, the abnormality detection unit 12 determines whether the three final calculation results of the basic calculation and the two abnormality detection calculations are consistent with each other. Since the abnormality detection unit 12 makes a determination using the final calculation result, the reliability of the final calculation result can be ensured.

[0048] When it is determined that the three final calculation results are consistent with each other (step S70: Yes), in step S80, the abnormality detection unit 12 determines that there is no abnormality in the calculation core and the abnormality detection core. After step S80 is completed, the abnormality detection process ends.

[0049] On the other hand, when it is determined that the three final operation results are inconsistent with each other (step S70: No), in step S84, the abnormality detection unit 12 determines that at least a part of the operation core and the abnormality detection core has an abnormality. "The three final operation results of the basic operation and the two abnormality detection operations are inconsistent with each other" means that at least one of the three final operation results is inconsistent with the other final operation results. Since the abnormality detection unit 12 determines whether the three final operation results are consistent with each other, when the three final operation results are inconsistent with each other, it is possible to determine the processor core with a higher possibility of an abnormality. The method of determining the processor core with a higher possibility of an abnormality is the same as the case where the determination result of the above-mentioned step S50 is No. After step S84 ends, the abnormality detection process ends.

[0050] Returning to the judgment of the above-mentioned step S20, when it is determined that the calculation load of the basic operation is greater than 1 / 3 of the maximum calculation load (step S20: No), in step S32, the calculation allocation unit 11 allocates calculation processing to the processor core so that in addition to executing the basic operation, it also executes an operation processing identical to the basic operation in parallel as an operation for abnormality detection.

[0051] In step S42, the abnormality detection unit 12 acquires two intermediate calculation results of the basic calculation and the abnormality detection calculation output in parallel. The intermediate level of the intermediate calculation results and the timing of acquiring the intermediate calculation results are the same as those in step S40 described above.

[0052] In step S52, as a permission condition, the abnormality detection unit 12 determines whether the two intermediate calculation results of the basic calculation and the abnormality detection calculation are consistent with each other. When it is determined that the two intermediate calculation results are inconsistent with each other (step S52: No), in step S84, the abnormality detection unit 12 determines that at least a part of the calculation core and the abnormality detection core has an abnormality. After step S84 is completed, the abnormality detection process ends.

[0053] On the other hand, if it is determined that the two intermediate calculation results are consistent with each other (step S52: Yes), in step S62, the abnormality detection unit 12 obtains the two final calculation results of the basic calculation and the abnormality detection calculation output in parallel. The timing of obtaining the final calculation results is the same as that of the above-mentioned step S60.

[0054] In step S72, as a permission condition, the abnormality detection unit 12 determines whether the two final operation results of the basic operation and the abnormality detection operation are consistent with each other. When it is determined that the two final operation results are consistent with each other (step S72: Yes), in step S82, the abnormality detection unit 12 determines that there is no abnormality in the operation kernel and the abnormality detection kernel. After step S82 is completed, the abnormality detection process ends.

[0055] On the other hand, if it is determined that the two final calculation results do not match each other (step S72: No), in step S84, the abnormality detection unit 12 determines that at least one of the calculation core and the abnormality detection core has an abnormality. After step S84, the abnormality detection process ends.

[0056] According to the computing system 100 of the first embodiment described above, the computing cores among the plurality of processors are determined according to the computing load of the computing process, and it is determined whether the computing result of the basic computing executed in the computing core is consistent with the computing result of the abnormality detection computing executed in the abnormality detection core, thereby detecting the presence or absence of an abnormality. Therefore, compared with a configuration that includes a processor core for abnormality detection in addition to a processor core used in normal times, it is possible to suppress waste in the system configuration and detect the presence or absence of an abnormality.

[0057] Furthermore, since the abnormality detection unit 12 detects the presence of abnormality using the calculation results output in parallel with each other, the presence of abnormality can be detected in real time.

[0058] In addition, the operation allocation unit 11 causes the abnormality detection kernel to perform the same operation processing as the basic operation as the abnormality detection operation, and the abnormality detection unit 12 detects the presence or absence of an abnormality by determining whether the operation result of the basic operation is consistent with the operation result of the abnormality detection operation, thereby easily detecting an abnormality.

[0059] In addition, the abnormality detection unit 12 uses the intermediate calculation result to determine whether there is an abnormality, so there is no need to wait for the output of the final calculation result, and the abnormality can be detected at an earlier stage, which can suppress the increase in the time required for abnormality detection. In addition, the abnormality detection unit 12 uses the final calculation result to determine whether there is an abnormality, so the reliability of the final calculation result can be ensured.

[0060] B. Second Implementation Method:

[0061] like Figure 6 As shown, the operation system 100 of the second embodiment replaces Figure 3 Step S12 shown in FIG. Figure 7 The processing shown is different from the operation system 100 of the first embodiment in this respect. The system structure of the operation system 100 of the second embodiment and other processes in the abnormality detection processing are the same as those of the operation system 100 of the first embodiment, so the same reference numerals are attached to the same structure and the same process, and their detailed description is omitted.

[0062] If it is determined in step S10 that the calculation load of the basic calculation is less than 1 / 2 of the maximum calculation load (step S10: No), Figure 7In step S122 shown, the arithmetic processing is allocated to the processor cores so that, in addition to executing the basic arithmetic processing, an arithmetic processing similar to the basic arithmetic processing is executed in parallel as an arithmetic processing for abnormality detection.

[0063] Reference Figure 8 The example shown is a more specific description of the allocation of the calculation processing in step S122. Figure 8 In the example shown, the minimum number of cores for basic operations is set to 6. In this case, the operation allocation unit 11 allocates basic operations to the processor cores PC1 to PC6, and allocates abnormality detection operations to the processor cores PC7 and PC8. Figure 8 In the example shown, the processor cores PC1 to PC6 correspond to the calculation cores, and the processor cores PC7 and PC8 correspond to the abnormality detection cores.

[0064] exist Figure 7 In step S124 shown in FIG. 1 , the abnormality detection unit 12 obtains the intermediate calculation result of the basic calculation and the intermediate calculation result of the abnormality detection calculation outputted with a delay compared to the intermediate calculation result. The intermediate level of obtaining the intermediate calculation result is the same as that of step S40 described above. Figure 8 In the example shown, the number of processor cores executing the abnormality detection operation is less than the number of processor cores executing the basic operation. Therefore, the time until the operation in the intermediate layer is completed in the abnormality detection operation is longer than the time until the operation in the intermediate layer is completed in the basic operation. In this way, the intermediate operation result of the abnormality detection operation is outputted with a delay compared with the intermediate operation result of the basic operation for the corresponding pixels.

[0065] The abnormality detection unit 12 obtains each intermediate calculation result when outputting the intermediate calculation result of the abnormality detection operation after outputting the intermediate calculation result of the basic operation for each corresponding pixel. More specifically, the abnormality detection unit 12 directly obtains the intermediate calculation result of the abnormality detection operation output from the abnormality detection core, and obtains the intermediate calculation result of the basic operation output from the calculation core and temporarily stored in the semiconductor storage device 20 from the semiconductor storage device 20. In addition, the abnormality detection unit 12 may also obtain the intermediate calculation result of the abnormality detection operation output from the abnormality detection core and temporarily stored in the semiconductor storage device 20 from the semiconductor storage device 20.

[0066] exist Figure 7In step S126 shown in the figure, as a permission condition, the abnormality detection unit 12 determines whether the two intermediate calculation results of the basic calculation and the abnormality detection calculation are consistent with each other. When it is determined that the two intermediate calculation results are inconsistent with each other (step S126: No), in step S134, the abnormality detection unit 12 determines that at least a part of the calculation core and the abnormality detection core has an abnormality. After step S134 is completed, the abnormality detection process ends.

[0067] On the other hand, when it is determined that the two intermediate calculation results are consistent with each other (step S126: Yes), in step S128, the abnormality detection unit 12 obtains the final calculation result of the basic calculation and the final calculation result of the abnormality detection calculation output with a delay compared with the final calculation result. For the same reason as in the case of the intermediate calculation result, the final calculation result of the abnormality detection calculation is output with a delay compared with the final calculation result of the basic calculation for the corresponding pixels.

[0068] As in the case of the above-mentioned intermediate calculation results, the abnormality detection unit 12 obtains each final calculation result when outputting the final calculation result of the abnormality detection calculation after outputting the final calculation result of the basic calculation for each corresponding pixel. More specifically, the abnormality detection unit 12 directly obtains the final calculation result of the abnormality detection calculation output from the abnormality detection core, and obtains the final calculation result of the basic calculation output from the calculation core and temporarily stored in the semiconductor storage device 20 from the semiconductor storage device 20. In addition, the abnormality detection unit 12 may also obtain the final calculation result of the abnormality detection calculation output from the abnormality detection core and temporarily stored in the semiconductor storage device 20 from the semiconductor storage device 20.

[0069] In step S130, as a permission condition, the abnormality detection unit 12 determines whether the two final operation results of the basic operation and the abnormality detection operation are consistent with each other. When it is determined that the two final operation results are consistent with each other (step S130: Yes), in step S132, the abnormality detection unit 12 determines that there is no abnormality in the operation core and the abnormality detection core. After step S132 is completed, the abnormality detection process ends.

[0070] On the other hand, if it is determined that the two final calculation results do not match each other (step S130: No), in step S134, the abnormality detection unit 12 determines that at least one of the calculation core and the abnormality detection core has an abnormality. After step S134, the abnormality detection process ends.

[0071] According to the operation system 100 of the second embodiment described above, the presence of an abnormality is detected by using the operation result of the basic operation and the operation result of the abnormality detection operation output after the operation result. Therefore, even if the operation load of the basic operation is large and the operation result of the basic operation and the operation result of the abnormality detection operation cannot be output in parallel in the processor core, the presence of an abnormality can be detected.

[0072] C. Third Implementation Method:

[0073] The operation system 100 of the third embodiment replaces Figure 7 The processing shown is performed Fig. 9 The processing shown is different from the operation system 100 of the second embodiment in this respect. The system structure of the operation system 100 of the third embodiment and other processes in the abnormality detection processing are the same as those of the operation system 100 of the second embodiment, so the same reference numerals are attached to the same structure and the same process, and their detailed description is omitted.

[0074] In step S122A, the operation distribution unit 11 distributes operation processing to the processor core so that in addition to the basic operation, two lightweight operations are performed in parallel as abnormality detection operations. "Lightweight operation" refers to operation processing using a neural network model that has been pre-learned to suppress the operation load compared to the basic operation. As a method of suppressing the operation load, for example, known lightweight techniques such as Pruning and Sparsity can be applied to suppress the operation load by replacing the values ​​of some coefficients that have a low contribution to the operation result with 0.

[0075] Reference Fig.10 The example shown in FIG. 1 is a more specific description of the allocation of the calculation processing in step S122A. Fig.10 In the example shown, the minimum number of cores for basic operations is set to 6. In this case, the operation allocation unit 11 allocates basic operations to the processor cores PC1 to PC6. Fig.10 In the example shown, as an abnormality detection operation, the operation allocation unit 11 allocates a first lightweight operation whose operation load is suppressed to 1 / 6 of the operation load of the basic operation to the processor core PC7, and allocates a second lightweight operation that is the same operation as the first lightweight operation to the processor core PC8. Fig.10 In the example shown, the processor cores PC1 to PC6 correspond to the calculation cores, and the processor core PC7 and the processor core PC8 correspond to the abnormality detection cores.

[0076] exist Fig. 9In step S124A shown in FIG. 1 , the abnormality detection unit 12 obtains three intermediate operation results of the basic operation and two lightweight operations output in parallel. Fig.10 In the example shown, the number of processor cores executing each lightweight operation is 1 / 6 of the number of processor cores executing the basic operation, and the computational load of the lightweight operation is also lightweighted to 1 / 6 of the computational load of the basic operation. Therefore, each intermediate computation result for mutually corresponding pixels is output in parallel.

[0077] In step S126A, as the permission condition, in the present embodiment, the abnormality detection unit 12 determines whether all of the following conditions (1) to (3) are satisfied. In addition, the threshold values ​​in the following conditions (2) and (3) can be values ​​determined in advance in experiments, for example, such as the difference between each intermediate operation result when the basic operation and the lightweight operation are respectively executed in the processor core confirmed to have no abnormality.

[0078] (1) The intermediate calculation results of the first lightweight calculation and the second lightweight calculation are consistent with each other

[0079] (2) The difference between the intermediate calculation results of the basic calculation and the first lightweight calculation is less than a predetermined threshold.

[0080] (3) The difference between the intermediate calculation results of the basic calculation and the second lightweight calculation is less than a predetermined threshold.

[0081] If it is determined that at least one of the above conditions (1) to (3) is not satisfied (step S128A: No), in step S134, the abnormality detection unit 12 determines that at least one of the calculation core and the abnormality detection core has an abnormality. After step S134, the abnormality detection process ends.

[0082] On the other hand, when it is determined that all of the above conditions (1) to (3) are satisfied (step S126A: Yes), in step S128A, the abnormality detection unit 12 obtains three final operation results of the basic operation and the two lightweight operations output in parallel. For the same reason as in the case of the above intermediate operation results, the final operation results for the corresponding pixels are output in parallel.

[0083] In step S130A, as the permission condition, in the present embodiment, the abnormality detection unit 12 determines whether all of the following conditions (4) to (6) are satisfied. In addition, the thresholds in the following conditions (5) and (6) can be predetermined by the same method as the thresholds in the above conditions (2) and (3).

[0084] (4) The final calculation results of the first lightweight calculation and the second lightweight calculation are consistent with each other

[0085] (5) The difference between the final calculation results of the basic calculation and the first lightweight calculation is less than a predetermined threshold.

[0086] (6) The difference between the final calculation results of the basic calculation and the second lightweight calculation is less than a predetermined threshold.

[0087] When it is determined that all of the above conditions (4) to (6) are satisfied (step S130A: Yes), in step S132, the abnormality detection unit 12 determines that there is no abnormality in the calculation core and the abnormality detection core. After step S132, the abnormality detection process ends.

[0088] On the other hand, when it is determined that at least one of the above conditions (4) to (6) is not satisfied (step S130A: No), in step S134, the abnormality detection unit 12 determines that at least a part of the calculation core and the abnormality detection core has an abnormality. After step S134 is completed, the abnormality detection process ends.

[0089] According to the computing system 100 of the third embodiment described above, a lightweight computing is performed in the abnormality detection core to determine whether the permitted conditions including the difference between the computing result of the basic computing and the computing result of the lightweight computing is smaller than a predetermined threshold value are satisfied. Therefore, in the computing core and the abnormality detection core, even if the computing load of the basic computing is large and the computing processing identical to the basic computing cannot be performed in parallel as the abnormality detection computing, by performing the basic computing and the lightweight computing in parallel, the presence or absence of abnormality can be detected using the computing results output in parallel with each other, and the presence or absence of abnormality can be detected in real time.

[0090] D. Fourth Implementation Method:

[0091] The operation system 100 of the fourth embodiment is different from the operation system 100 of the third embodiment in that the permission condition in step S130A of the abnormality detection process is different. The system structure of the operation system 100 of the fourth embodiment and other processes in the abnormality detection process are the same as those of the operation system 100 of the third embodiment, so the same reference numerals are attached to the same structure and the same process, and their detailed description is omitted.

[0092] exist Fig. 9 In step S130A shown in the figure, as the permission condition, in the present embodiment, the abnormality detection unit 12 determines whether the following condition (7) is satisfied and at least one of the condition (8) and the condition (9) is satisfied.

[0093] (7) In the final calculation results of the first lightweight calculation and the second lightweight calculation, the top five categories to which the objects contained in the image data belong with a high probability are consistent with each other.

[0094] (8) In each of the final calculation results of the basic calculation and the first lightweight calculation, there is at least one mutually consistent category among the top five categories with a higher probability that the object contained in the image data belongs to.

[0095] (9) In each final calculation result of the basic calculation and the second lightweight calculation, there is at least one mutually consistent category among the top five categories with a higher probability that the object contained in the image data belongs to.

[0096] In the above conditions (7) to (9), the number of top categories to be compared is not limited to 5. In the above conditions (8) and (9), the lower limit of the number of categories required to match each other may be 2 or more.

[0097] According to the operation system 100 of the fourth embodiment described above, the abnormality detection unit 12 can also detect the presence or absence of an abnormality by determining whether the permissible conditions including the following conditions are satisfied: whether at least one of the top five categories with a higher probability that the object contained in the image data belongs to in the operation result of the basic operation and the top five categories with a higher probability that the object contained in the image data belongs to in the operation result of the abnormality detection operation are consistent with each other.

[0098] E. Fifth Implementation Method:

[0099] The operation system 100 of the fifth embodiment is different from the operation system 100 of the fourth embodiment in that in step S122A of the abnormality detection process, the detection core performs a bit reduction operation instead of a lightweight operation. The system structure of the operation system 100 of the fifth embodiment and other processes in the abnormality detection process are the same as those of the operation system 100 of the fourth embodiment, so the same reference numerals are attached to the same structure and the same process, and their detailed description is omitted.

[0100] "Bit reduction operation" refers to the operation processing of a neural network model that is pre-learned using weight coefficients with fewer bits than the above-mentioned lightweight operation. In this embodiment, in the lightweight operation, the number of bits of the weight coefficient is set to 8 bits, and in the bit reduction operation, the number of bits of the weight coefficient is set to 4 bits. In addition, the number of bits in the bit reduction operation is not limited to 4 bits, and can be set to any number of bits less than the number of bits of the weight coefficient in the lightweight operation.

[0101] In the method of performing lightweight calculation as the abnormality detection calculation, the weight coefficient used in the basic calculation and the weight coefficient used in the lightweight calculation are different from each other, so compared with the method of performing the basic calculation as the abnormality detection calculation, more storage capacity for storing the weight coefficient is required in the semiconductor storage device 20. In addition, since different weight coefficients are read from the semiconductor storage device 20 in the basic calculation and the lightweight calculation, respectively, the memory bandwidth is squeezed compared with the case where only the weight coefficient used in the basic calculation is read.

[0102] In the bit reduction operation, since weight coefficients with fewer bits are used than in the lightweight operation, it is possible to suppress an increase in the storage capacity for storing weight coefficients in semiconductor storage device 20. In addition, it is possible to suppress the occupation of memory bandwidth when reading weight coefficients from semiconductor storage device 20.

[0103] According to the operation system 100 of the fifth embodiment described above, since a weight coefficient having a smaller number of bits than a lightweight operation is used, it is possible to suppress an increase in the storage capacity for storing the weight coefficient in the semiconductor storage device 20. In addition, it is possible to suppress the occupation of the memory bandwidth when reading the weight coefficient from the semiconductor storage device 20.

[0104] F. Other implementations:

[0105] (F1) In the above embodiment, the abnormality detection unit 12 performs both determination using the intermediate calculation result and determination using the final calculation result, but the present disclosure is not limited thereto. The abnormality detection unit 12 may perform only one of the determination using the intermediate calculation result and the determination using the final calculation result.

[0106] (F2) In the above embodiment, the processor cores PC1 to PC8 are input with a single image data, and different regions in the image data are shared by a plurality of processor cores, and the calculation processing is performed in parallel, but the present disclosure is not limited to this. The processor cores PC1 to PC8 may also be input with a plurality of image data, and different image data may be shared by a plurality of processor cores, and the calculation processing is performed in parallel.

[0107] Reference Fig.11 For more specific explanation. Fig.11As shown, it is also possible that the processor cores PC1 and PC2 perform the first basic operation with the first image data as the object, the processor cores PC3 and PC4 perform the second basic operation with the second image data as the object, and the processor cores PC5 and PC6 perform the third basic operation with the third image data as the object. In addition, in this case, the processor cores PC7 and PC8 can also perform the same operation processing as the first basic operation with the first image data as the object as the operation for abnormality detection in the same way as the processor cores PC1 and PC2. By performing each operation processing in this way, each basic operation can be performed in parallel on the three image data in the processor cores PC1 to PC6, and it is possible to detect whether the processor cores PC1, PC2, PC7, and PC8 have abnormalities. Moreover, after the above-mentioned operation processing is completed, when the basic operation is performed in parallel on the three new image data in the processor cores PC1 to PC6, the processor cores PC7 and PC8 can also replace the processor cores PC1 and PC2 to perform the abnormality detection operation with the same image data as the image data set as the object in the processor cores PC3 and PC4. By executing each operation processing in this way, each time an operation processing is performed on new image data, it is possible to detect in sequence whether an abnormality has occurred in a processor core different from the previous one, each basic operation can be performed on three image data in parallel, and all processor cores can be detected in sequence whether an abnormality has occurred.

[0108] (F3) In the third embodiment, the operation allocation unit 11 allocates operation processing to the processor cores in step S122A of the abnormality detection processing so that the processor cores perform two lightweight operations as abnormality detection operations in addition to the basic operation, but the present disclosure is not limited to this. The operation allocation unit 11 may also allocate operation processing to the processor cores in step S122A so that the processor cores perform one lightweight operation as abnormality detection operation in addition to the basic operation. Fig.10 In the same case as the example shown, the operation allocation unit 11 can allocate one abnormality detection operation to the processor cores PC7 and PC8. In this case, the operation load of the lightweight operation can be reduced to 1 / 3 of the basic operation. In this structure, compared with a structure in which the operation load of the lightweight operation is further reduced, the difference between the operation results of the basic operation and the lightweight operation can be suppressed from becoming larger, and the reduction in the accuracy of abnormality detection can be suppressed.

[0109] (F4) In the third embodiment, as a lightweight operation, the operation allocation unit 11 causes the abnormality detection kernel to perform an operation process in which a known lightweight technology is applied to the basic operation. The known lightweight technology suppresses the operation load by replacing the values ​​of some coefficients that have a low contribution to the operation result with 0, but the present disclosure is not limited to this. As a lightweight operation, the operation allocation unit 11 may also cause the abnormality detection kernel to perform an operation process in which the operation load is suppressed by using a neural network model different from the neural network model used in the basic operation. For example, when ResNet is used as the neural network model in the basic operation, MobileNet may also be used in the lightweight operation. MobileNet is a neural network model that implements image recognition in the same way as ResNet. The layer structure of MobileNet is simpler than that of ResNet, so by using MobileNet in the lightweight operation, the operation load can also be suppressed compared to the basic operation using ResNet.

[0110] The operation system 100 and the method thereof described in the present disclosure may also be implemented by a special-purpose computer as follows: the special-purpose computer is provided by a processor and a memory programmed in a manner to execute one or more functions embodied by a computer program. Alternatively, the operation system 100 and the method thereof described in the present disclosure may also be implemented by a special-purpose computer as follows: the special-purpose computer is provided by constituting a processor using one or more special-purpose hardware logic circuits. Alternatively, the operation system 100 and the method thereof described in the present disclosure may also be implemented by one or more special-purpose computers as follows: the special-purpose computer is constituted by a combination of a processor and a memory programmed to implement one or more hardware and a processor constituted by one or more hardware logic circuits. In addition, the computer program may also be stored as instructions executed by a computer in a computer-readable non-transient tangible recording medium.

[0111] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various structures within the scope of its main purpose. For example, the technical features in each embodiment corresponding to the technical features in the method described in the invention content column can be appropriately replaced or combined in order to solve part or all of the above-mentioned problems or to achieve part or all of the above-mentioned effects. In addition, as long as the technical features are not described as necessary features in this specification, they can be appropriately deleted.

[0112] (Method 1)

[0113] A computing system, which performs computing processing using a neural network model, comprises:

[0114] A plurality of processor cores, which perform the arithmetic processing;

[0115] an operation allocating unit that determines a processor core (i.e., an operation core) among the plurality of processors that executes the operation process based on the operation load of the operation process, and causes the operation core to execute the operation process; and

[0116] an abnormality detection unit, which detects whether the processor core has an abnormality;

[0117] The operation distribution unit causes at least a portion of the processor cores of the plurality of processors that are not the operation cores to perform an operation based on the basic operation, namely, an abnormality detection operation, in parallel with the operation processing, namely, the basic operation, performed by the operation core.

[0118] The abnormality detection unit determines that there is no abnormality in the operation core and the processor core that executes the abnormality detection operation, i.e., the abnormality detection core, when the difference between the first operation result as the operation result of the basic operation and the second operation result as the operation result of the abnormality detection operation satisfies a predetermined permission condition,

[0119] The abnormality detection unit determines that at least part of the calculation core and the abnormality detection core has an abnormality when the difference does not satisfy the allowable condition.

[0120] (Method 2)

[0121] The computing system according to aspect 1, wherein:

[0122] The abnormality detection unit detects the presence or absence of the abnormality using the first calculation result and the second calculation result output in parallel with the first calculation result.

[0123] (Method 3)

[0124] The computing system according to aspect 1, wherein:

[0125] The computing system further includes a semiconductor storage device for storing the first computing result.

[0126] The abnormality detection unit detects the presence or absence of the abnormality using the first calculation result read from the semiconductor storage device and the second calculation result output after the first calculation result.

[0127] (Method 4)

[0128] The computing system according to any one of Modes 1 to 3, wherein:

[0129] The abnormality detection unit uses at least one of the calculation result in the last layer among the multiple layers constituting the neural network model, i.e., the final calculation result, and the calculation result in the intermediate layer among the multiple layers, i.e., the intermediate calculation result, as the first calculation result and the second calculation result to detect the presence or absence of the abnormality.

[0130] (Method 5)

[0131] The computing system according to any one of Modes 1 to 4, wherein:

[0132] The operation allocating unit causes the abnormality detection core to execute the operation processing that is the same as the basic operation as the abnormality detection operation.

[0133] The permission condition includes a condition that the first calculation result and the second calculation result are consistent with each other.

[0134] (Method 6)

[0135] The computing system according to any one of Modes 1 to 4, wherein:

[0136] The calculation allocation unit causes the abnormality detection core to execute, as the abnormality detection calculation, a lightweight calculation that has been learned in advance to reduce the calculation load compared to the basic calculation,

[0137] The permission condition includes a condition that the difference is smaller than a predetermined threshold value.

[0138] (Method 7)

[0139] The computing system according to any one of Modes 1 to 4, wherein:

[0140] The neural network model is configured as a multi-category classifier that outputs a probability that the input data belongs to each of a plurality of predetermined categories based on the input data,

[0141] The calculation allocation unit causes the abnormality detection core to execute, as the abnormality detection calculation, a lightweight calculation that has been learned in advance to reduce the calculation load compared to the basic calculation,

[0142] The permission condition includes a condition that a predetermined number of categories among the top N categories with higher probabilities in the first calculation result and the top N categories with higher probabilities in the second calculation result coincide with each other.

[0143] (Method 8)

[0144] The computing system according to any one of aspects 1 to 7, wherein:

[0145] The calculation assignment unit causes the abnormality detection kernel to execute, as the abnormality detection calculation, a bit number reduction calculation that has been learned in advance using a weight coefficient having a smaller bit number than a predetermined lightweight calculation.

Claims

1. A computing system that uses a neural network model to perform computing processing, characterized in that: have: A plurality of processor cores for executing the arithmetic processing; An operation allocating unit that determines a processor core (i.e., an operation core) among the plurality of processors that executes the operation process based on the operation load of the operation process, and causes the operation core to execute the operation process; as well as An abnormality detection unit, detecting whether the processor core has an abnormality; The operation allocating unit causes at least a portion of the processor cores other than the operation core among the plurality of processors to execute an operation obtained based on the basic operation, namely, an abnormality detection operation, in parallel with the operation processing, namely, the basic operation, performed by the operation core. The abnormality detection unit determines that there is no abnormality in the operation core and the processor core that executes the abnormality detection operation, i.e., the abnormality detection core, when the difference between the first operation result as the operation result of the basic operation and the second operation result as the operation result of the abnormality detection operation satisfies a predetermined permission condition, The abnormality detection unit determines that at least part of the calculation core and the abnormality detection core has an abnormality when the difference does not satisfy the allowable condition.

2. The computing system according to claim 1, characterized in that: The abnormality detection unit detects the presence or absence of the abnormality using the first calculation result and the second calculation result output in parallel with the first calculation result.

3. The computing system according to claim 1, wherein: The computing system further includes a semiconductor storage device for storing the first computing result. The abnormality detection unit detects the presence or absence of the abnormality using the first calculation result read from the semiconductor storage device and the second calculation result output after the first calculation result.

4. The computing system according to claim 1, wherein: The abnormality detection unit uses at least one of the calculation result in the last layer among the multiple layers constituting the neural network model, i.e., the final calculation result, and the calculation result in the intermediate layer among the multiple layers, i.e., the intermediate calculation result, as the first calculation result and the second calculation result to detect the presence or absence of the abnormality.

5. The computing system according to any one of claims 1 to 4, characterized in that: The operation allocating unit causes the abnormality detection core to execute the operation processing that is the same as the basic operation as the abnormality detection operation. The permission condition includes a condition that the first calculation result and the second calculation result are consistent with each other.

6. The computing system according to any one of claims 1 to 4, characterized in that: The calculation allocation unit causes the abnormality detection core to execute, as the abnormality detection calculation, a lightweight calculation that has been previously learned so as to suppress the calculation load compared to the basic calculation. The permission condition includes a condition that the difference is smaller than a predetermined threshold value.

7. The computing system according to any one of claims 1 to 4, characterized in that: The neural network model is configured as a multi-category classifier that outputs a probability that the input data belongs to each of a plurality of predetermined categories based on the input data, The calculation allocation unit causes the abnormality detection core to execute, as the abnormality detection calculation, a lightweight calculation that has been previously learned so as to suppress the calculation load compared to the basic calculation. The permission condition includes a condition that a predetermined number of categories among the top N categories with high probability in the first calculation result and the top N categories with high probability in the second calculation result are identical to each other.

8. The computing system according to any one of claims 1 to 4, characterized in that: The calculation assignment unit causes the abnormality detection kernel to execute, as the abnormality detection calculation, a bit number reduction calculation that has been learned in advance using a weight coefficient having a smaller bit number than a predetermined lightweight calculation.

Images