Processor-implemented neural network methods and neural network devices

Abstract

Processor-implemented neural network methods and neural network devices are disclosed. A processor-implemented neural network method includes receiving an event corresponding to a neural network operation and a control program for performing the neural network operation, detecting a missing event based on the event and the control program, and generating a configuration file for the neural network operation based on a result of the detecting.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0114564, filed on September 8, 2020, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to methods and apparatus with neural network profiling. Background Technology

[0003] In the case of simulator inference, the NPU can be profiled by uploading the register-transfer level (RTL) of the neural processing unit (NPU) to the simulator and the board that executes the simulator and performs the inference, and by downloading the logs after the inference is completed and then performing profiling by parsing the profiling data.

[0004] In the case of target inference, event information can be obtained during inference by connecting the hardware event signals of the NPU and the ARM System Trace Macrocell (STM) to the mobile phone kernel driver.

[0005] Such methods may require data post-processing and consume significant time for profiling due to the large log files. Furthermore, it may be difficult to determine the portion of the neural network currently performing inference, and the profiling data may be inaccurate when event logs are lost. Summary of the Invention

[0006] The present invention is provided in a brief form to introduce the choice of concepts further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0007] In one general aspect, a processor-implemented neural network method includes: receiving an event corresponding to a neural network operation and a control program for performing the neural network operation; detecting a missing event based on the event and the control program; and generating a configuration file of the neural network operation based on the detection result.

[0008] The events may include: start events and end events of neural network operations.

[0009] The control program may include the execution sequence of neural network operations.

[0010] The detection steps may include: determining whether the event matches an execution sequence included in the control program; and detecting a missing event based on the determined result.

[0011] The generation process may include: determining the type of the lost event; and generating a configuration file by compensating for the lost event based on the determined type.

[0012] The steps of generating a configuration file by compensating for lost events based on type may include: in response to a lost event being of type start event, inserting the start event into the configuration file at a time determined by subtracting a first time value from the subsequent event of the lost event.

[0013] The next event can be the ending event.

[0014] The steps of generating a configuration file by compensating for lost events based on type may include: determining whether a neural network operation overlaps with an event corresponding to another operation in response to the type of the lost event being a termination event; and inserting the termination event into the configuration file based on the determined result.

[0015] The step of inserting an end event may include: in response to determining that a neural network operation overlaps with an event corresponding to the other operation, inserting an end event in the portion where the overlap begins.

[0016] The step of inserting a termination event may include: in response to determining that a neural network operation does not overlap with an event corresponding to the other operation, inserting a termination event at a time determined by subtracting a second time amount from the subsequent event of the lost event.

[0017] The method may include: optimizing neural network operations based on a generated configuration file; and performing inference using the optimized neural network operations, wherein the neural network operations may include any one of convolution, padding, pooling, and reformatting.

[0018] A non-transitory computer-readable storage medium stores instructions that, when executed by a processor, configure the processor to perform the method.

[0019] In another general aspect, a neural network device includes: a receiver configured to: receive an event corresponding to a neural network operation and a control program for performing the neural network operation; and a processor configured to: detect a missing event based on the event and the control program; and generate a configuration file for the neural network operation based on the detection result.

[0020] The events may include: start events and end events of neural network operations.

[0021] The control program may include the execution sequence of neural network operations.

[0022] For the purpose of the detection, the processor may be configured to: determine whether the event matches an execution sequence included in the control program; and detect a missing event based on the determined result.

[0023] For the generation, the processor can be configured to: determine the type of the lost event; and generate a configuration file by compensating for the lost event based on the determined type.

[0024] In order to generate a configuration file by compensating for lost events based on type, the processor can be configured to insert the start event into the configuration file at a time determined by subtracting a first time value from the subsequent event of the lost event, in response to a start event of type start event.

[0025] In order to generate a configuration file by compensating for lost events based on type, the processor can be configured to: determine whether a neural network operation overlaps with an event corresponding to another operation in response to the type of the lost event being a termination event; and insert the termination event into the configuration file based on the determined result.

[0026] In order to insert an end event, the processor may be configured to insert an end event in the portion where the overlap begins, in response to determining that a neural network operation overlaps with an event corresponding to the other operation.

[0027] In order to insert an end event, the processor may be configured to insert an end event at a time determined by subtracting a second time amount from the event following the lost event, in response to determining that the neural network operation does not overlap with an event corresponding to the other operation.

[0028] In another general aspect, a processor-implemented neural network method includes: detecting a lost event by determining that an event corresponding to a neural network operation does not match an execution sequence included in a control program used to perform the neural network operation; and generating a configuration file for the neural network operation by inserting lost events into a configuration file based on the type of the lost event.

[0029] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0030] Figure 1 An example of a profiling device is shown.

[0031] Figure 2 An example of a neural network processing system is shown.

[0032] Figure 3 An example of the operation of the profiling equipment is shown.

[0033] Figure 4 An example of an operation performed by the profiling device to compensate for a lost event is shown.

[0034] Figure 5 An example of visualization performed by the profiling device is shown.

[0035] Figure 6 An example of a profiling method performed by a profiling device is shown.

[0036] Throughout the accompanying drawings and specific embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0037] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative, except for those that must occur in a specific order, and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure. Furthermore, for greater clarity and conciseness, descriptions of features known upon understanding this disclosure may be omitted.

[0038] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein have been provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will be clear upon understanding the disclosure of this application.

[0039] Throughout this specification, when a component is described as being "on" another component, "connected to," or "bonded to" another component, it may be directly "connected to" or "bonded to" said other component, or there may be one or more other components in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, there may be no other elements in between. Similarly, similar expressions (e.g., "between" and "immediately between," and "adjacent to" and "closely adjacent to") are interpreted in the same manner. As used herein, the term "and / or" includes any one and any combination of any two or more of the associated listed items.

[0040] Although terms such as "first," "second," and "third" may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts are not limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0041] The terminology used herein is for describing various embodiments only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and as understood based on the disclosure of this application. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the relevant field and in the context of the disclosure of this application, and shall not be interpreted in an idealized or overly formal sense. The use of the term “may” herein with respect to examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature, but not all examples are limited thereto.

[0043] Furthermore, in the description of the exemplary embodiments, descriptions of structures or functions known after understanding the disclosure of this application will be omitted where such detailed descriptions would lead to a vague interpretation of the exemplary embodiments.

[0044] In the following description, examples will be given in detail with reference to the accompanying drawings, and the same reference numerals in the drawings always denote the same elements.

[0045] Figure 1 An example of a profiling device is shown.

[0046] The analysis device 10 can perform neural network analysis. The analysis device 10 can perform analysis associated with the operations performed in the neural network.

[0047] Profiling can be or includes dynamic program analysis that measures the time complexity and space (e.g., memory) of a program, the use of specific instructions, the cycle and frequency of function calls, etc. Profiling information can be used to assist in the optimization of neural networks. Profiling device 10 can perform profiling by analyzing program source code or binary executable files.

[0048] A profile can be or includes data generated through profiling. A profile can indicate events associated with time-based neural network operations (or neural network operations as described below).

[0049] Neural networks can include statistical learning algorithms used in machine learning. A neural network can represent an overall model that possesses problem-solving capabilities because nodes, which are connected via synaptic links, learn to change the strength of these synaptic connections.

[0050] Neural networks can include deep neural networks (DNNs). For example, neural networks can include convolutional neural networks (CNNs), recurrent neural networks (RNNs), perceptrons, feedforward (FF) networks, radial basis function (RBF) networks, deep FF (DFF) networks, long short-term memory (LSTM) networks, gated recurrent units (GRUs), autoencoders (AEs), variational AEs (VAEs), denoising AEs (DAEs), sparse AEs (SAEs), Markov chains (MCs), Hopfield networks (HNs), Boltzmann machines (BMs), restricted BMs (RBMs), deep belief networks (DBNs), deep convolutional networks (DCNs), deconvolutional networks (DNs), deep convolutional inverse graph networks (DCIGNs), generative adversarial networks (GANs), liquid state machines (LSMs), extreme learning machines (ELMs), echo state networks (ESNs), deep residual networks (DRNs), differentiable neural computers (DNCs), neural Turing machines (NTMs), capsule networks (CNs), Kohonen networks (KNs), and / or attention networks (ANs).

[0051] The profiling device 10 can generate a profile based on events associated with neural network operations and visualize the generated profile.

[0052] By generating a configuration file for neural network operations, the profiling device 10 can verify or determine whether computation time is appropriate for hardware specifications predicted or determined in the inference process of the neural network model, and whether neural network operations are performed according to the prediction cycle. Furthermore, the profiling device 10 can use the generated configuration file to detect optimization points in the neural network.

[0053] The profiling device 10 can generate a configuration file for neural network operations by processing information associated with those operations. This information may include events associated with the neural network operations and control programs used to execute them.

[0054] Events can indicate the start and end based on the type of neural network operation. Events can include start and end events for the neural network operation.

[0055] The control program may include a compiler-generated program for performing inference using a neural network. The control program may include an intrinsic sequence of neural network operators (e.g., intrinsic). Here, the term "intrinsic" may refer to the built-in functionality of a neural processing unit (NPU) (e.g., a neural processor) that performs neural network operations. For example, the control program may include an execution sequence of neural network operations.

[0056] Reference Figure 1 The analysis device 10 may include a receiver 100, a processor 200 (e.g., one or more processors) and a memory 300.

[0057] Receiver 100 can receive events associated with neural network operations and control programs used to perform neural network operations.

[0058] Receiver 100 can output received events and received control programs to processor 200. Receiver 100 may include a receiving interface.

[0059] Processor 200 can process data stored in memory 300. Processor 200 can execute computer-readable instructions stored in memory 300 that configure processor 200 to perform operations.

[0060] The processor 200 may be a hardware data processing device having circuitry with a physical structure for performing the desired operation. For example, the desired operation may include code or instructions contained in a program.

[0061] Data processing devices may include, for example, microprocessors, central processing units (CPUs), processor cores, multi-core processors, multiprocessors, application-specific integrated circuits (ASICs), and / or field-programmable gate arrays (FPGAs).

[0062] The processor 200 can detect missing events based on events and control procedures. A missing event can be an event that is assumed or intended to be included in the intrinsics of the control procedure and is executed during the processing of the neural network but is not included in the received events.

[0063] The processor 200 can determine whether an event matches an execution sequence included in the control program, and can detect a missing event based on the determined result.

[0064] Processor 200 can generate a configuration file for neural network operations based on the results of detecting missing events. Processor 200 can determine the type of missing event. Processor 200 can generate the configuration file by compensating for the missing event based on the determined type.

[0065] When the type of the lost event corresponds to (or is determined to correspond to) a start event, the processor 200 may insert the start event at the time obtained in the configuration file by subtracting a first time amount from the time corresponding to the subsequent event of the lost event.

[0066] When the type of the missing event corresponds to (or is determined to correspond to) a termination event, the processor 200 can determine whether the neural network operation overlaps with an event associated with another operation, and can insert a termination event based on the determined result.

[0067] When a neural network operation overlaps (or is determined to overlap) with an event associated with another operation, the processor 200 may insert an end event at the beginning of the overlap. When a neural network operation does not overlap (or is determined not to overlap) with an event associated with another operation, the processor 200 may insert an end event at a time obtained by subtracting a second time amount from the time corresponding to the time of the subsequent event of the lost event.

[0068] The memory 300 may store instructions (or programs) that can be executed by the processor 200. For example, the instructions may include instructions for performing operations of the processor 200 and / or operations of each component of the processor 200.

[0069] The memory 300 can be a volatile or non-volatile memory device.

[0070] Volatile memory devices can be, for example, dynamic random access memory (DRAM), static RAM (SRAM), thyristor RAM (T-RAM), zero-capacitor RAM (Z-RAM), and / or dual-transistor RAM (TTRAM).

[0071] Non-volatile memory devices can be, for example, electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic RAM (MRAM), spin-transfer torque (STT) MRAM (STT-MRAM), conductive bridged RAM (CBRAM), ferroelectric RAM (FeRAM), phase-change RAM (PRAM), resistive RAM (RRAM), nanotube RRAM, polymer RAM (PoRAM), nanofloating gate memory (NFGM), holographic memory, molecular electronic memory device, and / or insulator resistance change memory.

[0072] Figure 2 An example of a neural network processing system is shown.

[0073] Reference Figure 2 In a neural network processing system, profiling device 10 and system components can send and receive information related to neural network operations. System components can perform debugging and performance measurements. For example, system components may include CoreSight. CoreSight can be a conventional system for extracting tracking data from the NPU and may include various components (e.g., ETF, ETR, AMBA tracking bus, aggregator (Funnel), etc.).

[0074] The profiling device 10 may include a processor 200 and a memory 300, and may also include an operator 400. The memory 300 may be DRAM. The memory 300 may store tracking data.

[0075] The operator 400 can be located inside or outside the analysis device 10.

[0076] The operator 400 may include an NPU or a digital signal processor (DSP). The operator 400 may include a combiner. The combiner may predefine events. The combiner may combine events based on one of the predefined sets. Furthermore, NPUCs (e.g., NPUC0 and NPUC1) may be core modules within the NPU and may be real arithmetic units.

[0077] Processor 200 can receive events associated with neural network operations from operator 400. Processor 200 can generate a neural network profile by comparing the received events and compensating for lost events via a control program. CSSYS_STM_MUX_SELECTION can indicate the selection of NPU STM events using a multiplexer.

[0078] Figure 3 Showing the analysis equipment (e.g., Figure 1An example of the operation of the analysis device 10 shown in the figure.

[0079] Reference Figure 3 The analysis device 10 may be included in a host device. The host device may be, for example, a personal computer (PC) or a server. The analysis device 10 may receive event information associated with operations performed in the target device and perform analysis on the neural network operations.

[0080] The host device may include a compiler. In operation 310, the compiler may build a neural network. In operation 320, the compiler may generate a control program. For example, the compiler may generate a network control program (NCP) as an executable file for the NPU.

[0081] The target device can be or includes a device that performs inference using a neural network. The target device can be, for example, an Internet of Things (IoT) device, a machine-type communication device, and a portable electronic device.

[0082] Portable electronic devices may include, for example, laptops, mobile phones, smartphones, tablet PCs, mobile internet devices (MIDs), personal digital assistants (PDAs), enterprise digital assistants (EDAs), digital cameras, digital video cameras, portable multimedia players (PMPs), personal or portable navigation devices (PNDs), handheld game consoles, e-readers, and smart devices. Smart devices may include, for example, smartwatches and smart bracelets.

[0083] The target device may include an NPU. In operation 330, the target device may use an NPU configured to perform operations included in a neural network to perform inference. In operation 340, the target device may generate event information while performing inference. In one or more non-limiting examples, the target device may include a host device.

[0084] Receiver 100 can receive event information and control procedures. In operation 350, processor 200 can perform neural network profiling based on the event information and control procedures. The following will refer to... Figure 4 A non-restrictive example of performing profiling is described in further detail.

[0085] In Operation 360, processor 200 can perform visualization based on the generated configuration file.

[0086] Figure 4 This demonstrates the use of a profiling device (e.g., Figure 1 The example shown is an operation performed by the profiling device 10 to compensate for a lost event.

[0087] Reference Figure 4The processor 200 can detect missing events based on events and control procedures. The compiler can generate and transmit the generated control procedure. The control procedure may include, for example, an NCP.

[0088] NCPs can have groups that act as execution units. Through these groups, network execution points can be estimated.

[0089] exist Figure 4 In the example, the NCP (or intrinsic) generated by the compiler may include the execution sequence of neural network operations. Event information can be generated and sent by the NPU. Event information can be received in the form of data files.

[0090] Event information and control procedures (e.g., NCP (or intrinsic)) may include neural network operations and events associated with those operations. For example, in Figure 4 In the code, "File" indicates a convolution operation, "PU" indicates a fill / pooling operation, and "RU" indicates a reformatting operation. Each operation can have a start event and an end event.

[0091] The processor 200 can determine whether an event matches an execution sequence included in the control program, and can detect a missing event based on the determined result.

[0092] For example, processor 200 can detect a loss event by determining that File operation, PU operation, and RU operation were not performed simultaneously.

[0093] Processor 200 can generate a configuration file for neural network operations based on the results of detecting missing events. Processor 200 can determine the type of missing event. Processor 200 can generate the configuration file by compensating for the missing event based on the determined type.

[0094] For example, when the type of the lost event is a start event, the processor 200 may insert the start event at a time obtained by subtracting a first time amount from the time corresponding to the subsequent event (e.g., the end event) of the lost event.

[0095] Here, when the type of the lost event is a start event, since it may be unknown whether the start event occurs while direct memory access (DMA) is being performed before the start event or immediately after the completion of another operator's event, the processor 200 may insert the start event at the time obtained by subtracting a first time amount from the time corresponding to the end event.

[0096] The first time value can vary depending on the type of operation and the hardware. For example, the first time value could be 10 nanoseconds (ns). Alternatively, the first time value could be predetermined based on the type of operation and / or the hardware.

[0097] When the type of the missing event is a termination event, processor 200 can determine whether the neural network operation overlaps with an event associated with another operation. Processor 200 can insert a termination event based on the determined result.

[0098] For example, when a neural network operation overlaps with an event associated with another operation, the processor 200 may insert a termination event at the beginning of the overlap. When a neural network operation does not overlap with an event associated with another operation, the processor 200 may insert a termination event at a time obtained by subtracting a second time value from the time corresponding to the subsequent event of the lost event. The second time value may vary depending on the operation type and hardware. For example, the second time value may be 10 ns. For example, the second time value may be predetermined based on the operation type and / or hardware.

[0099] When events overlap, processor 200 can determine that the operation in the overlapping portion is invalid. Processor 200 can compensate for the missing event so that it does not have such an overlapping portion.

[0100] For example, in the event that both the start and end events are lost, or three or more events are lost, the processor 200 can perform compensation by inserting the lost events at a time obtained by subtracting a calculated time amount from the time when the initially received event occurred after the lost event. The calculated time amount can be obtained by multiplying a third time amount by a predetermined index. For example, the third time amount and the predetermined index can be determined experimentally.

[0101] The third time value can vary depending on the type of operation and the hardware. For example, the third time value could be 10 ns.

[0102] Figure 5 This shows the analysis equipment (e.g., Figure 1 The example shown is a visualization of the analysis device 10.

[0103] Reference Figure 5 In operation 510, processor 200 can parse the received events. Processor 200 can perform event parsing by verifying the event packet recorded in event information (e.g., an event file). The event packet may include the event's timestamp, event identifier (ID), and event type. The event ID may include, as described above... Figure 4 The described File, PU, ​​and RU, and the event types may include start events and end events.

[0104] In operation 520, processor 200 may determine whether an event matches an execution sequence. For example, processor 200 may determine whether an event matches an execution sequence by determining whether start and end events of a neural network operation have been received based on the execution sequence included in the control program.

[0105] In operation 530, when an event matches the execution sequence of a neural network operation, processor 200 may output an event log. In operation 540, when an event does not match the execution sequence, processor 200 may output a missing event log.

[0106] Processor 200 can generate a configuration file by outputting an event log or a lost event log. In operation 550, after the output of the event log or lost event log is complete, processor 200 can terminate the control program. In operation 560, processor 200 can visualize the generated configuration file after the control program has terminated. If the control program has not terminated, it returns to operation 510, and processor 200 can continue to parse the received events.

[0107] Figure 6 This demonstrates the use of a profiling device (e.g., Figure 1 An example of a profiling method performed by the profiling device 10 shown in the figure.

[0108] Reference Figure 6 In operation 610, receiver 100 may receive events associated with neural network operation and control programs for performing neural network operation.

[0109] In operation 630, processor 200 may detect a lost event based on an event and a control procedure. For example, processor 200 may determine whether an event matches an execution sequence included in the control procedure. Processor 200 may detect a lost event based on the determined result (e.g., processor 200 may detect a lost event in response to an event not matching an execution sequence).

[0110] In operation 650, processor 200 may generate a configuration file for the neural network operation based on the results of detecting missing events. For example, processor 200 may determine the type of missing event. Processor 200 may generate the configuration file by compensating for the missing event based on the determined type.

[0111] For example, when the type of the lost event is a start event, the processor 200 may insert the start event at a time obtained by subtracting a first time amount from the time corresponding to the subsequent event of the lost event.

[0112] When the type of the missing event is a termination event, processor 200 can determine whether the neural network operation overlaps with an event associated with another operation. Processor 200 can insert a termination event based on the determined result.

[0113] When a neural network operation overlaps with an event associated with another operation, the processor 200 may insert a termination event at the beginning of the overlap. When a neural network operation does not overlap with an event associated with another operation, the processor 200 may insert a termination event at a time obtained by subtracting a second time amount from the time corresponding to the time of the subsequent event of the lost event.

[0114] In this regard Figures 1 to 6The described profiling device, receiver, processor, memory, neural network processing system, operator, system component, profiling device 10, receiver 100, processor 200, memory 300, operator 400, and other devices, apparatuses, units, modules, and components are implemented by or represent hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). The processor or computer may be implemented by one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by the processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described herein. The hardware components can also access, manipulate, process, create, and store data in response to the execution of the instructions or software. For brevity, the singular terms "processor" or "computer" are used in the description of the examples described herein; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component, or two or more hardware components, may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. The hardware components can have any one or more different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0115] Perform the operations described in this application Figures 1 to 6The methods illustrated are executed by computing hardware (e.g., one or more processors or a computer), which is implemented to execute instructions or software as described above to perform the operations performed by the methods described in this application. For example, a single operation, or two or more operations, may be executed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation, or two or more operations.

[0116] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written in any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding description used herein, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0117] Instructions or software used to control computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, as well as any associated data, data files, and data structures, may be recorded, stored, or fixed in, or on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), programmable random access read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SSDs), card storage devices (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other devices configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide the instructions or software and any associated data, data files, and data structures to one or more processors or computers, enabling one or more processors or computers to execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0118] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example should be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

Claims

1. A processor-implemented neural network method, comprising: Receive events corresponding to neural network operations and control programs used to execute neural network operations; The missing event is detected based on the aforementioned event and control procedures; and The configuration file for neural network operations is generated based on the detection results. The events include: the start event and the end event of the neural network operation. The generation steps include: Determine the type of the loss event; and The configuration file is generated by compensating for lost events based on a defined type. The steps for generating the configuration file based on type compensation loss events include: In response to a lost event of type "start event", the start event is inserted into the configuration file at a time determined by subtracting a first time value from the time corresponding to the subsequent event of the lost event, where the subsequent event is the end event; or In response to the type of the missing event being a termination event, it determines whether the neural network operation overlaps with an event corresponding to another operation, and inserts the termination event into the configuration file based on the determined result.

2. The method according to claim 1, wherein, The control program includes the execution sequence of neural network operations.

3. The method according to claim 1, wherein, The detection steps include: Determine whether the event matches an execution sequence included in the control procedure; and Detect missing events based on definitive results.

4. The method according to claim 1, wherein, The steps for inserting the end event include: In response to determining that a neural network operation overlaps with an event corresponding to the other operation, an end event is inserted in the portion where the overlap begins.

5. The method according to claim 1, wherein, The steps for inserting the end event include: In response to determining that a neural network operation does not overlap with an event corresponding to the other operation, an end event is inserted at a time determined by subtracting a second time amount from the time corresponding to the subsequent event of the lost event.

6. The method according to any one of claims 1 to 3, further comprising: Optimize neural network operations based on the generated configuration file; and Inference is performed using optimized neural network operations, which include any one of convolution, padding, pooling, and reformatting.

7. A non-transitory computer-readable storage medium for storing instructions, which, when executed by a processor, configure the processor to perform the method according to any one of claims 1 to 6.

8. A neural network device, comprising: The receiver is configured to receive events corresponding to neural network operations and control programs for performing neural network operations. and The processor is configured as follows: The missing event is detected based on the aforementioned event and control procedures; and The configuration file for neural network operations is generated based on the detection results. The events include: the start event and the end event of the neural network operation. For the purpose of this generation, the processor is configured as follows: Determine the type of the loss event; and The configuration file is generated by compensating for lost events based on a defined type. Specifically, in order to generate the configuration file by compensating for lost events based on type, the processor is configured as follows: In response to a lost event of type "start event", the start event is inserted into the configuration file at a time determined by subtracting a first time value from the time corresponding to the subsequent event of the lost event, where the subsequent event is the end event; or In response to the type of the missing event being a termination event, it determines whether the neural network operation overlaps with an event corresponding to another operation, and inserts the termination event into the configuration file based on the determined result.

9. The device according to claim 8, wherein, The control program includes the execution sequence of neural network operations.

10. The device according to claim 8, wherein, For the purpose of the detection, the processor is configured to: Determine whether the event matches an execution sequence included in the control procedure; and Detect missing events based on definitive results.

11. The device according to claim 8, wherein, In order to insert the end event, the processor is configured as follows: In response to determining that a neural network operation overlaps with an event corresponding to the other operation, an end event is inserted in the portion where the overlap begins.

12. The device according to claim 8, wherein, In order to insert the end event, the processor is configured as follows: In response to determining that a neural network operation does not overlap with an event corresponding to the other operation, an end event is inserted at a time determined by subtracting a second time amount from the time corresponding to the subsequent event of the lost event.

13. A processor-implemented neural network method, comprising: Missing events are detected by determining that the event corresponding to a neural network operation does not match the execution sequence included in the control program used to perform the neural network operation; and The configuration file for neural network operations is generated by inserting loss events into the configuration file based on the type of loss event. The events include: the start event and the end event of the neural network operation. The steps for inserting lost events into the configuration file based on the type of lost event include: In response to a lost event of type "start event", the start event is inserted into the configuration file at a time determined by subtracting a first time value from the time corresponding to the subsequent event of the lost event, where the subsequent event is the end event; or In response to the type of the missing event being a termination event, it determines whether the neural network operation overlaps with an event corresponding to another operation, and inserts the termination event into the configuration file based on the determined result.

Images