Apparatus and method for searching for optimal architecture of neural network

Abstract

To provide a method and apparatus for searching for an optimal architecture of a neural network.SOLUTION: An apparatus for searching for an optimal architecture of a neural network includes a processor. The processor is configured to: determine a neural network loss based on parameters of a candidate architecture for the neural network; measure hardware resources required for operation of the neural network with the candidate architecture; predict hardware resources required for operating the neural network with the candidate architecture, using a hardware resource prediction module; determine a hardware resource loss based on the measured hardware resources and the predicted hardware resources; and determine a target architecture of the neural network based on the neural network loss and the hardware resource loss.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The following disclosure relates to techniques for searching for optimal architectures for neural networks. [Background technology]

[0002] Neural architecture search (NAS) is one of the methodologies for automatically searching for the optimal architecture of a neural network for a given purpose. NAS is a method for searching the most appropriate neural network architecture structure and form for solving a specific problem through deep learning. A neural network in NAS may be generated by selecting and combining primitive operations, called a search space, which are composed of predefined operators and functions. Here, examples of operators include convolution, pooling, concatenation, skip connection, etc. Summary of the Invention [Problem to be solved by the invention]

[0003] An object of the present invention is to provide a technique for searching for an optimal architecture for a neural network. [Means for solving the problem]

[0004] In one embodiment, an apparatus for searching for an optimal architecture for a neural network includes a processor, which determines a neural network loss based on parameters for candidate architectures of the neural network, measures hardware resources required when the neural network of the candidate architecture operates, predicts hardware resources required when the neural network of the candidate architecture operates using a hardware resource prediction module, determines a hardware resource loss based on the measured hardware resources and the predicted hardware resources, and determines a target architecture for the neural network based on the neural network loss and the hardware resource loss.

[0005] The hardware resource prediction module may be a neural network that receives parameters of the candidate architecture as input, predicts the hardware resources of the neural network of the candidate architecture based on the input parameters, and outputs a hardware resource prediction value.

[0006] The processor may determine an optimization loss as a weighted sum of the neural network loss due to the candidate architectures and parameters of the candidate architectures and the hardware resource loss due to parameters of the candidate architectures, and determine the target architecture that minimizes the optimization loss.

[0007] The hardware resources may include at least one of power consumption, memory requirements, number of operations, and processing time when the neural network of the candidate architecture operates.

[0008] The processor may determine an optimization loss including the neural network loss and the hardware resource loss, and may determine the target architecture by selecting a target operation that minimizes the optimization loss from among candidate operations of each layer included in the neural network of the candidate architecture.

[0009] An augmented reality providing device according to one embodiment includes a processor that performs processing operations using a neural network having a target architecture, the processor determining a neural network loss based on parameters of the neural network for a candidate architecture, measuring hardware resources required when the neural network of the candidate architecture operates, using a hardware resource prediction module to predict hardware resources required when the neural network of the candidate architecture operates, determining a hardware resource loss based on the measured hardware resources and the predicted hardware resources, and determining the target architecture based on the neural network loss and the hardware resource loss.

[0010] In one embodiment, a method for searching for an optimal architecture for a neural network includes operations of determining a neural network loss based on parameters for candidate architectures of the neural network, measuring hardware resources required when the neural network of the candidate architecture operates, predicting hardware resources required when the neural network of the candidate architecture operates using a hardware resource prediction module, determining a hardware resource loss based on the measured hardware resources and the predicted hardware resources, and determining a target architecture for the neural network based on the neural network loss and the hardware resource loss. Effect of the Invention

[0011] According to one embodiment, it is possible to search for an optimal architecture for a neural network taking into account hardware resource limitations, and optimization can be performed in a short time. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating a search framework for searching for an optimal architecture of a neural network according to one embodiment. [Diagram 2] FIG. 1 is a block diagram showing the configuration of a search device for searching for an optimal architecture of a neural network according to an embodiment. [Diagram 3] FIG. 13 is a diagram illustrating a process of selecting an optimal target operation from among candidate operations of each layer according to an embodiment. [Figure 4] FIG. 1 is a diagram illustrating a search process for determining a target architecture of a neural network according to an embodiment. [Diagram 5] 1 is a flowchart illustrating the operation of a method for searching for an optimal architecture of a neural network according to an embodiment. [Figure 6] FIG. 1 is a diagram illustrating a configuration of an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The specific structural or functional description of the embodiments is disclosed for the purpose of illustration only, and may be modified in various forms. Therefore, the embodiments are not limited to the specific disclosed forms, and the scope of the present specification includes modifications, equivalents, or alternatives within the technical spirit.

[0014] Although terms such as first or second may be used to describe multiple components, such terms should be construed only for the purpose of distinguishing one component from the other components. For example, a first component may be named a second component, and similarly, a second component may be named a first component.

[0015] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that although it is directly coupled or connected to the other component, there may be other components in between.

[0016] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, the terms "comprise" or "have" and the like indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood as not precluding the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0017] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Commonly used predefined terms should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as ideal or overly formal unless expressly defined in this specification.

[0018] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the drawings, the same components are given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted.

[0019] FIG. 1 is a diagram illustrating a search framework for searching for an optimal architecture of a neural network according to one embodiment.

[0020] Referring to FIG. 1, the search framework 100 is a framework for searching an optimal architecture (or neural network structure) for a basic neural network 120 through machine learning. The basic neural network 120 is a neural network before learning (or an unlearned neural network) in which the operations and parameters (e.g., connection weights) of each layer are not determined. The basic neural network 120 may include multiple neural network layers (or simply "layers"). The basic neural network 120 may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRRDNN), a deep Q-network, or a combination of two or more of these, but is not limited to the above examples. The basic neural network 120 may include a hardware structure and / or a software structure.

[0021] The search framework 100 performs machine learning on a basic neural network 120 using training data stored in a database 110. The machine learning can be performed by a supervised learning or a partially supervised learning method.

[0022] In one embodiment, the exploration framework 100 may train the base neural network 120 through supervised learning. The exploration framework 100 may perform the learning using a training algorithm and a loss function, such as stochastic gradient descent. The learning data used for learning may include input data input to the neural network and validation data corresponding to the input data. The base neural network 120 may process the input data included in the learning data and output result data. The exploration framework 100 may determine a neural network loss based on a comparison result between the result data output from the base neural network 120 and the validation data, and search for an optimal architecture that minimizes the neural network loss.

[0023] The search framework 100 may perform a multiple objective neural architecture search (NAS) method to search for an optimal architecture for the target neural network 130. The search framework 100 may search for an optimal architecture based on a method of setting various candidate operations for each layer of the basic neural network 120, rather than sampling the architecture of the basic neural network 120, and selecting a target operation, which is the most appropriate candidate operation among the candidate operations, for each layer. Through this search method, the search framework 100 may perform early optimization.

[0024] The search framework 100 can derive a target neural network 130 having an optimal architecture according to a purpose (e.g., object classification, object recognition, speech recognition, etc.) through a learning process. Searching for the optimal architecture includes determining the operations to be performed at each layer of the neural network and determining optimal values ​​of the neural network parameters. The search framework 100 can be executed by an apparatus for searching for an optimal architecture of a neural network described herein (e.g., the search apparatus 200 shown in FIG. 2).

[0025] The search framework 100 considers hardware resource constraints in searching for an optimal architecture for the target neural network 130. The search framework 100 can optimize the validation loss for a task performed by the neural network, as well as the hardware resources used when the neural network is executed. The search framework 100 can search for the target neural network 130 by considering the hardware resources required by the neural network when it is running. The hardware resources can be, for example, power consumption, memory requirements, number of operations (e.g., number of multiply-accumulate (MAC) operations), processing time, and GPU occupancy. The search framework 100 can consider one or more hardware resources, and can consider any hardware resources that can be observed numerically other than the hardware resources in the above examples, without any restrictions.

[0026] Once candidate architectures for the base neural network 120 are determined, the search framework 100 can determine neural network losses and hardware resource losses for the candidate architectures, and search for a target architecture that minimizes the neural network losses and hardware resource losses. The neural network losses and hardware resource losses constitute an optimization loss for determining the target architecture.

[0027] When determining the hardware resource loss, the search framework 100 may determine the hardware resource loss based on actual measurements of the hardware resources required by the neural network of the candidate architecture and a predicted value for the hardware resources derived using a hardware resource prediction module (e.g., the hardware resource prediction module 420 shown in FIG. 4). As described in more detail below, the hardware resource prediction module is a module that provides a predicted value of the hardware resources for the neural network of the candidate architecture that is the subject of current learning. The hardware resource prediction module may be realized by a neural network trained to output a predicted value for the hardware resources required by the neural network of the candidate architecture based on input parameters of the candidate architecture. The hardware resource prediction module has differentiable characteristics, and differentiability in the search process may be preserved through the hardware resource prediction module. Since the differentiability is preserved, end-to-end learning is possible. The search framework 100 reflects the hardware resources for the neural network architecture in the optimization loss through the hardware resource prediction module.

[0028] As described above, the exploration framework 100 can explore an optimal architecture for a neural network taking into account hardware resource limitations, and can perform optimization in a short time. The exploration framework 100 can also explore an optimal architecture taking into account actual measured hardware resource measurements.

[0029] FIG. 2 is a block diagram showing the configuration of a search device for searching for an optimal architecture of a neural network according to an embodiment.

[0030] Referring to Fig. 2, a search device 200 is an apparatus for searching an optimal architecture for a neural network, and can perform the search framework 100 described with reference to Fig. 1. The search device 200 may perform one or more operations described or illustrated herein in connection with an architecture search. The search device 200 includes a processor 210 and a memory 220. A storage device 230 can store data for the architecture search (e.g., training data) and store the neural network used for training.

[0031] The memory 220 can store various data used by the components (e.g., the processor 210) of the search device 200. The data may include, for example, input data or output data for software and instructions related thereto. The memory 220 may include one or more of volatile and non-volatile memory.

[0032] The processor 210 controls the overall operation of the search device 200 and executes instructions to perform the operations of the search device 200. The processor 210 can, for example, execute software to control at least one other component (e.g., hardware or software component) of the search device 200 connected to the processor 210 and perform various data processing or calculations.

[0033] According to one embodiment, as at least a part of the data processing or calculation, the processor 210 may store instructions or data in the memory 220, process the instructions or data stored in the memory 220, and store the result data in the memory 220. The processor 210 may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processing unit, a sensor hub processor, or a communication processor) that may operate independently or in conjunction with a main processor.

[0034] The processor 210 may use the training data to train a candidate architecture of a neural network (e.g., the basic neural network 120 shown in FIG. 1 ) and determine a neural network loss. The neural network before training may include a plurality of layers each including one or more artificial neurons, and each layer may have a candidate operation that can be performed predefined. The candidate operations may include, but are not limited to, a 3×3 kernel-based convolution operation, a 5×5 kernel-based convolution operation, a pooling operation, and the like. A candidate architecture may be determined by selecting one of the candidate operations for each layer of the neural network. The processor 210 may determine a neural network loss based on a parameter for the candidate architecture of the neural network. The processor 210 may determine a neural network loss based on a difference between the result data and the validation data derived by the neural network of the candidate architecture by processing the training data. The neural network loss may be determined by a predefined loss function.

[0035] The processor 210 can measure hardware resources required (or used) when the neural network of the candidate architecture operates. The measured hardware resources can include, for example, but are not limited to, one or more of power consumption, memory requirements, number of operations, and processing time when the neural network of the candidate architecture operates. The processor 210 can measure the hardware resources to determine hardware resource measurements.

[0036] The processor 210 can use a hardware resource prediction module (e.g., hardware resource prediction module 420 shown in FIG. 4) to predict hardware resources required when a neural network of a candidate architecture operates. The hardware resource prediction module may be a neural network that receives parameters of the candidate architecture (e.g., information on a selected candidate operation from among the candidate operations of each layer) as input, predicts hardware resources of the neural network of the candidate architecture based on the input parameters, and outputs a hardware resource prediction value.

[0037] The processor 210 can determine the hardware resource loss based on the measured hardware resource and the predicted hardware resource. The processor 210 can determine the hardware resource loss based on the difference between the measured hardware resource and the predicted hardware resource and a predefined loss function. For example, the processor 210 can determine the hardware resource loss by applying the difference between the actually measured processing time measurement value and the processing time prediction value output from the hardware resource prediction model to the loss function when the neural network of the candidate architecture is operated.

[0038] The processor 210 may determine a target architecture of the neural network based on the neural network loss and the hardware resource loss. The processor 210 may determine a target architecture and target parameters that reduce the neural network loss and the hardware resource loss. The processor 210 may update the parameters of the candidate architecture to minimize the hardware resource loss. The processor 210 may determine an optimization loss including the neural network loss and the hardware resource loss, and may determine a target architecture by selecting a target operation that minimizes the optimization loss from among candidate operations of each layer included in the neural network of the candidate architecture. The processor 210 may select a target operation of each layer that best minimizes the neural network loss and the hardware resource loss, and update the parameters of the neural network. The processor 210 may determine a weighted sum of the neural network loss according to the candidate architecture and the parameters of the candidate architecture, and the hardware resource loss according to the parameters of the candidate architecture, as an optimization loss, and may determine a target architecture that minimizes the optimization loss.

[0039] The operations performed by the searching device 200 described above can be variously applied to neural network-based algorithms that can operate in embedded systems as well as mobile devices such as wearable devices and smartphones.

[0040] FIG. 3 is a diagram illustrating a process of selecting an optimal target operation from among candidate operations of each layer according to an embodiment.

[0041] 3, in step S310, a neural network before being trained (e.g., the basic neural network 120 shown in FIG. 1) includes multiple layers 312, 314, 316, and 318, and each layer 312, 314, 316, and 318 has executable candidate operations defined. In the illustrated embodiment, the neural network 310 is defined to have three candidate operations for each layer 312, 314, 316, and 318. The candidate operations to be executed between layers may be different operation methods. For example, the candidate operations to be executed between layers 312 and 314 may be different operation methods.

[0042] In the learning process, the search device (for example, the search device 200 shown in FIG. 2) can select a target operation that is an optimal candidate operation from among the candidate operations. In step S320, the search device can select one of the candidate operations 322, 324, 326, 328, 329 for each of the layers 312, 314, 316, 318, and determine an optimization loss for a candidate architecture composed of a combination of the selected candidate operations 322, 324, 326, 328, 329. The search device can combine the candidate operations of each of the layers 312, 314, 316, 318 several times in the search space, calculate the optimization loss for each combination, and determine a combination of the candidate operations (in other words, the target operations) that minimizes the optimization loss. Once the learning process for various combinations is completed, a target architecture can be determined based on the target operations selected for each of the layers 312, 314, 316, 318. The target architecture includes the target operations of each layer.

[0043] FIG. 4 is a diagram illustrating a search process for determining a target architecture of a neural network according to an embodiment.

[0044] 4, in the process of searching for a target architecture of a neural network, if a candidate architecture a 412 of the neural network is given, a parameter w(a) 416 of the candidate architecture a 412 can be determined. The candidate architecture a 412 indicates a neural network structure including a set of candidate operations selected for each layer.

[0045] The parameters of the candidate architecture a412 may include parameters for a candidate operation selected from among the candidate operations of each layer of the neural network, and parameters indicating operation characteristics for each selected candidate operation. For example, assuming that the candidate operations of a layer included in the neural network include a 3×3 kernel-based convolution operation and a 5×5 kernel-based convolution operation, the parameters of the candidate architecture a412 may include a parameter indicating which of the two convolution operations has been selected, and a kernel parameter of the selected convolution operation. Here, the convolution operation may be realized as a convolution layer.

[0046] The searcher (e.g., the searcher 200 shown in FIG. 2) determines the neural network loss L for the specific task of the neural network of the candidate architecture a 412 based on the parameter w(a) 416. TASK (w(a), a) 418 can be determined. The neural network loss L TASK (w(a), a) 418 is the loss to minimize the loss of the task performed by the neural network.

[0047] The search device executes (432) the operation of the neural network having the candidate architecture a 412, and actually measures (434) the hardware resources of the entire neural network required in the process of executing the operation. The measured hardware resources may include, for example, power consumption, memory requirements, the number of operations, and processing time. Through the measurement of the hardware resources 434, a hardware resource measurement value LAT for the candidate architecture a 412 is obtained. M (a) 436 is determined. According to an embodiment, the hardware resources can be measured by performing a neural network operation on the architecture determined through the max operation. Since the process 430 including such a process is not differentiable, forward and backward operations cannot be defined for the process 430. To solve this, the hardware resource prediction module 420 may be used.

[0048] The search device can use the hardware resource prediction module 420 to predict the hardware resources of the entire neural network that the neural network having the candidate architecture a 412 is expected to require when performing this operation. P The hardware resource prediction module 420 receives parameters of the candidate architecture a 412 as input, predicts the hardware resources of the neural network of the candidate architecture a 412 based on the input parameters, and outputs a hardware resource prediction value LAT(a) 422. P It can output (a) 422. The operations in the hardware resource prediction module 420 consist of differentiable operations.

[0049] The hardware resource prediction module 420 may be a neural network trained through a learning process to output a predicted value of hardware resources that are predicted to be required or used by the architecture based on the parameters of the neural network architecture. However, the hardware resource prediction module 420 may be realized by other means other than a neural network that can predict the hardware resources of the neural network based on the candidate architecture A412.

[0050] The search device uses hardware resource measurement values ​​LAT M (a) 436 and hardware resource prediction value LAT P (a) Based on 422, the resource loss L for candidate architecture a412 HW (a) 442 can be determined. M (a) 436 and hardware resource prediction value LAT P (a) The resource loss L is calculated so that the difference between HW (a)442 is defined.

[0051] In one embodiment, the resource loss L HW (a) 442 is the hardware resource measurement LAT M (a) 436 and hardware resource prediction value LAT P (a) L showing the loss due to the difference between 422 HW1 (a) and the L of factors for optimizing hardware resources (e.g., factors for minimizing latency) HW2 (a) is the basis for the determination of L HW1 (a) and L HW2 (a) can be determined by the following equations (1) and (2), respectively.

number

number

number

[0052] The search device uses hardware resource measurement values ​​LAT M (a) 436 and hardware resource prediction value LAT P (a) Applying the difference between 422 and a predefined loss function to the hardware resource loss L HW (a)442 can be determined.

[0053] The search device calculates the neural network loss L TASK (w(a), a) 418 and hardware resource loss L HW The target architecture can be determined by determining an optimization loss including (a) 442 and selecting a target operation that minimizes the optimization loss from among the candidate operations of each layer included in the neural network of the candidate architecture a 412. For example, the target architecture can be determined by selecting an optimization loss L TASK (w(a), a) + λ L HW The optimization loss may be determined by searching for a candidate architecture a 412 and a parameter w(a) 416 of the candidate architecture a that minimizes (a). As an example of Equation (4), the optimization loss is calculated by subtracting the hardware resource loss L HW (a) The neural network loss L with weighting value λ applied to 442 TASK (w(a), a) 418 and hardware resource loss L HW (a) 442 may be determined as a weighted sum between

number

[0054] 5 is a flow chart illustrating the operation of a method for searching for an optimal architecture of a neural network according to an embodiment of the present invention. The operation of the method may be performed by the searching device 200 shown in FIG.

[0055] 5, in operation 510, the searcher selects a candidate architecture for the neural network (e.g., the basic neural network 120 shown in FIG. 1). The searcher may select the candidate architecture by selecting one of the candidate operations defined for each layer of the neural network.

[0056] In operation 520, the searcher determines a neural network loss based on the parameters for the candidate architecture of the neural network. The searcher uses the training data to train the candidate architecture of the neural network and determines the neural network loss. The searcher can determine the neural network loss based on the difference between the validation data and the result data derived by the neural network of the candidate architecture processing the training data. The greater the difference between the result data derived from the neural network of the candidate architecture and the intended validation data, the greater the neural network loss.

[0057] In operation 530, the searcher measures hardware resources required when the neural network of the candidate architecture is operated, including, but not limited to, one or more of power consumption, memory requirements, number of operations, and processing time when the neural network of the candidate architecture is operated.

[0058] In operation 540, the exploration apparatus predicts hardware resources required when the neural network of the candidate architecture operates using a hardware resource prediction module (e.g., hardware resource prediction module 420 shown in FIG. 4). Information about selected candidate operations that constitute the candidate architecture among candidate operations that can be executed in each layer of the neural network is input to the hardware resource prediction model, and the hardware resource prediction model can provide a predicted value of the hardware resources required by the corresponding neural network based on the input information.

[0059] In operation 550, the searcher determines a hardware resource loss based on the measured hardware resource and the predicted hardware resource. The searcher can determine the hardware resource loss based on a difference between the measured hardware resource and the predicted hardware resource and a predefined loss function.

[0060] In operation 560, the search device determines a target architecture and target parameters of the neural network based on the neural network loss and the hardware resource loss. The search device may determine a target architecture and target parameters that minimize an optimization loss including the neural network loss and the hardware resource loss. The search device may determine the target architecture by selecting a target operation that minimizes the optimization loss from among the candidate operations of each layer included in the neural network of the candidate architecture. The search device determines the optimization loss as a weighted sum of the neural network loss due to the candidate architecture and the parameters of the candidate architecture and the hardware resource loss due to the parameters of the candidate architecture, and determines a target architecture that minimizes the optimization loss.

[0061] FIG. 6 is a diagram showing a configuration of an electronic device according to an embodiment.

[0062] 6, the electronic device 600 may be an electronic device in various forms, for example, but is not limited to, a wearable device (e.g., an augmented reality providing device such as AR glasses, a head mounted display (HMD)), a smartphone, a tablet computer, a netbook, a laptop, a product inspection device, a personal computer, or a server.

[0063] The electronic device 600 includes a processor 610, a memory 620, a camera 630, a sensor 640, an output device 650, and a communication device 660. At least some of the components of the electronic device 600 can be connected to each other via a peripheral communication interface 670 (e.g., a bus, a general purpose input and output (GPIO) interface, a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) to exchange signals (e.g., commands or data) between them.

[0064] The processor 610 controls the overall operation of the electronic device 600 and executes functions and instructions to be executed within the electronic device 600. The processor 610 performs the operations of the searching device (e.g., the searching device 200 of FIG. 2) described above with reference to FIGS. 1 to 5.

[0065] The memory 620 stores instructions executable by the processor 610 and data to be input / output by the processor 610. The memory 620 may include volatile memory, such as RAM, DRAM, SRAM, and / or non-volatile memory, such as ROM, flash memory, and the like, as known in the art.

[0066] The camera 630 captures an image. The camera 630 may capture, for example, a color image, a black and white image, a gray image, an infrared image, or a depth image.

[0067] The sensor 640 can detect an operating state (e.g., power or temperature) of the electronic device 600 or an external environmental state (e.g., a user state) and generate an electrical signal or data value corresponding to the detected state. The sensor 640 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0068] Output device(s) 650 provide output of electronic device 600 to a user through visual, auditory, or tactile channels and may include, for example, a liquid crystal display, an LED / OLED display, a micro light emitting diode (microLED), a touch screen, a speaker, a vibration generator, or any other device capable of providing output to a user.

[0069] The communication device 660 supports the establishment of a direct (e.g., wired) or wireless communication channel between the electronic device 600 and an external device, and the execution of communication through the established communication channel. According to an embodiment, the communication device 660 can include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module (e.g., a local area network (LAN) communication module, or a power line communication module). The wireless communication module can communicate with an external device through a short-range communication network (e.g., Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a long-range communication network (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a WAN)).

[0070] In one embodiment, the electronic device 600 may be an augmented reality providing device (e.g., augmented reality glasses) using a neural network-based algorithm. The augmented reality providing device is worn on a user's face and provides the user with content related to augmented reality services and / or virtual reality services. The processor 610 performs processing operations using a neural network having a target architecture. The camera 630 captures an image for generating the augmented reality content, and the processor 610 can generate the augmented reality content by processing the image using a neural network having a target architecture. For example, the processor 610 can recognize a specific object from the image acquired via the camera 630 and generate the augmented reality content by overlapping and expressing virtual content in the recognized object area or the area surrounding the object.

[0071] The processor 610 may determine a target architecture of the neural network through the process described with reference to FIG. 2 and FIG. 5. For example, the processor 610 may determine a neural network loss based on parameters for a candidate architecture of the neural network (e.g., the basic neural network 120 shown in FIG. 1) and measure hardware resources required when the neural network of the candidate architecture operates. The processor 610 may predict hardware resources required when the neural network of the candidate architecture operates using a hardware resource prediction module (e.g., the hardware resource prediction module 420 shown in FIG. 4) and determine a hardware resource loss based on the measured hardware resources and the predicted hardware resources. The processor 610 may determine a target architecture based on the neural network loss and the hardware resource loss. The processor 610 may determine an optimization loss based on a weighted sum of the neural network loss according to the candidate architecture and the parameters of the candidate architecture and the hardware resource loss according to the parameters of the candidate architecture, and determine parameters of the target architecture and the candidate architecture that minimize the optimization loss.

[0072] The embodiments described above may be implemented with hardware components, software components, or a combination of hardware and software components. For example, the devices and components described herein may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or other devices that execute and respond to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the operating system. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For ease of understanding, the processing device may be described as being one in which only one processing device is used, but one skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0073] The software may include computer programs, codes, instructions, or a combination of one or more thereof, which may configure or independently or in combination instruct a processing device to operate as desired. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, either permanently or temporarily, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed over computer systems coupled to a network, and may be stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable storage media.

[0074] The method according to the present invention may be embodied in the form of program instructions to be executed by various computer means and recorded on a computer-readable recording medium. The recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The recording medium and the program instructions may be specially designed and constructed for the purposes of the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter, etc.

[0075] The hardware devices described above may be configured to operate as one or more software modules to perform the operations illustrated in the present invention, and vice versa.

[0076] Although the embodiments have been described above by way of limited examples and drawings, those skilled in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, and may be replaced or substituted by other components or equivalents to achieve suitable results.

[0077] Accordingly, other implementations, embodiments, and equivalents of the claims are intended to be within the scope of the following claims. [Explanation of symbols]

[0078] 100: Exploration Framework 110: Database 120: Basic Neural Networks 130: Target Neural Network 200: Search device 210: Processor 220: Memory 230: Storage device 600:Electronic equipment 610: Processor 620:Memory 630: Camera 640: Sensor 650: Output device 660:Communication equipment 670: Communication interface

Claims

1. An apparatus for searching for an optimal architecture of a neural network, comprising a processor, wherein the processor determines a neural network loss based on parameters for a candidate architecture of the neural network, measures hardware resources required when the neural network of the candidate architecture operates, predicts hardware resources required when the neural network of the candidate architecture operates using a hardware resource prediction module, determines a hardware resource loss based on the measured hardware resources and the predicted hardware resources, and determines a target architecture of the neural network based on the neural network loss and the hardware resource loss. Apparatus.

2. The apparatus according to claim 1, wherein the hardware resource prediction module is a neural network that takes parameters of the candidate architecture as an input, and predicts the hardware resources of the neural network of the candidate architecture based on the input parameters to output a hardware resource prediction value.

3. The processor determines the hardware resource loss based on a difference between the measured hardware resources and the predicted hardware resources, and updates parameters of the candidate architecture so that the hardware resource loss is minimized. The apparatus according to claim 1 or 2.

4. The apparatus according to claim 1, wherein the processor determines a weighted sum of the neural network loss caused by the candidate architecture and its parameters and the hardware resource loss caused by the parameters of the candidate architecture as an optimization loss, and determines the target architecture that minimizes the optimization loss.

5. The apparatus according to claim 1, wherein the processor determines the target architecture and target parameters that reduce the neural network loss and the hardware resource loss.

6. The apparatus according to claim 1, wherein for each layer of the neural network, the candidate architecture is determined by selecting any one of the candidate operations included in each layer.

7. The apparatus according to claim 1, wherein information regarding the selected candidate operation that constitutes the candidate architecture is input to the hardware resource prediction module among the candidate operations executable in each layer of the neural network.

8. The apparatus according to claim 1, wherein the hardware resource includes at least one of power consumption, memory requirement, number of operations, and processing time when the neural network of the candidate architecture operates.

9. The processor determines an optimization loss including the neural network loss and the hardware resource loss, and determines the target architecture by selecting a target operation that minimizes the optimization loss among the candidate operations of each layer included in the neural network of the candidate architecture. The apparatus according to claim 1.

10. The apparatus according to claim 1, wherein the processor determines the neural network loss based on the difference between the result data derived by the neural network of the candidate architecture processing the learning data and the verification data.

11. An augmented reality providing apparatus, comprising a processor that performs a processing operation using a neural network having a target architecture, wherein the processor determines a neural network loss based on parameters for the candidate architecture of the neural network, measures the hardware resources required when the neural network of the candidate architecture operates, predicts the hardware resources required when the neural network of the candidate architecture operates using a hardware resource prediction module, determines a hardware resource loss based on the measured hardware resources and the predicted hardware resources, and determines the target architecture based on the neural network loss and the hardware resource loss. An augmented reality providing apparatus.

12. The augmented reality providing apparatus further includes a camera that captures an image. The augmented reality providing apparatus according to claim 11, wherein the processor generates augmented reality content by processing the video using a neural network having the target architecture.

13. The augmented reality providing apparatus according to claim 11 or 12, wherein the hardware resource prediction module is a neural network that takes the parameters of the candidate architecture as input and predicts the hardware resources of the neural network of the candidate architecture based on the input parameters, and outputs a hardware resource prediction value.

14. The augmented reality providing apparatus according to claim 11, wherein the processor determines the target architecture that minimizes a weighted sum of the neural network loss based on the candidate architecture and the parameters of the candidate architecture, and the hardware resource loss based on the parameters of the candidate architecture.

15. A method for searching for an optimal architecture of a neural network, comprising: determining a neural network loss based on parameters for a candidate architecture of the neural network; measuring hardware resources required when the neural network of the candidate architecture operates; predicting, using a hardware resource prediction module, hardware resources required when the neural network of the candidate architecture operates; determining a hardware resource loss based on the measured hardware resources and the predicted hardware resources; determining a target architecture of the neural network based on the neural network loss and the hardware resource loss; and the method includes the above steps.

16. The method according to claim 15, wherein the hardware resource prediction module is a neural network that takes the parameters of the candidate architecture as input and predicts the hardware resources of the neural network of the candidate architecture based on the input parameters, and outputs a hardware resource prediction value.

17. The operation of determining the target architecture includes the operation of determining the target architecture that minimizes the weighted sum of the neural network loss based on the candidate architecture and the parameters of the candidate architecture and the hardware resource loss based on the parameters of the candidate architecture. The method according to claim 15.

18. The operation of determining the target architecture includes the operation of determining the target architecture by selecting a target operation from among candidate operations of each layer included in the neural network of the candidate architecture. The method according to claim 15.

19. The hardware resources include at least one of power consumption, memory requirement, number of operations, and processing time when the neural network of the candidate architecture operates. The method according to claim 15.

20. A computer program for causing a processor to execute the method according to any one of claims 15 to 19.

Images