Learning method and apparatus for neural network models to improve their performance

Abstract

To provide a method and apparatus for training a neural network model to increase performance of the neural network model.SOLUTION: A method for training a neural network model includes the steps of: receiving input data and target data; pooling, by a neural network model, a feature map extracted from the input data based on a probability for each of classes of the feature map; generating output data by inputting the input data to a neural network model; determining a loss based on comparing the output data and the target data and an auxiliary loss of the pooling; and training the neural network model based on the loss.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method and apparatus for training a neural network model to improve the performance of the neural network model.

Background Art

[0002] In recent years, neural network model technologies have been used in various fields such as autonomous driving, advanced driver assistance systems, virtual reality, and the Internet of Things. As an example, a neural network model can be used to recognize objects from images.

[0003] A Convolutional Neural Network, which is a neural network model commonly used in image recognition, may include a convolutional layer that performs a convolutional operation and a pooling layer that performs pooling. The pooling layer can sample the input data to generate resized output data.

[0004] Since the pooling method in the prior art uses a representative value or an average value, loss of the input data occurs during the pooling process, and there is a risk that the performance of the neural network model will deteriorate due to the loss. Therefore, there is a need for a technology that can prevent loss of the input data during the pooling process.

Summary of the Invention

Means for Solving the Problems

[0005] A learning method for a neural network model according to one embodiment includes the steps of: receiving input data and target data; pooling feature maps extracted from the input data by the neural network model based on the probability of each class of the feature maps; inputting the input data into the neural network model to generate output data; comparing the output data, the target data, and the pooling auxiliary loss to determine the loss; and training the neural network model based on the loss.

[0006] The auxiliary loss may include a loss determined based on the difference between the feature map pooled by the neural network model and the target data.

[0007] The neural network model can determine the probability for each class based on the ratio of elements belonging to any given class among the elements included in the unit region of the feature map, which is determined by the pooling scale factor.

[0008] The input data includes an image containing multiple objects, and the pixels contained in the image can be divided into classes corresponding to each of the multiple objects.

[0009] The target data includes a distribution of probabilities for each class of the input data, and the auxiliary loss may include a loss calculated based on the distance between the distribution of probabilities for each class of the input data included in the target data and the distribution of probabilities for each class of the feature map.

[0010] In a learning device for a neural network model according to one embodiment, the learning device includes a processor, the processor receives input data and target data, pools the feature maps extracted from the input data using the neural network model based on the probability of each class of the feature maps, inputs the input data to the neural network model to generate output data, compares the output data, the target data, and the pooling auxiliary loss to determine the loss, and trains the neural network model based on the loss.

[0011] The auxiliary loss may include a loss determined based on the difference between the feature map pooled by the neural network model and the target data.

[0012] The neural network model can determine the probability for each class based on the ratio of elements belonging to any given class among the elements included in the unit region of the feature map, which is determined by the pooling scale factor.

[0013] The input data includes an image containing multiple objects, and the pixels contained in the image can be classified into classes corresponding to each of the multiple objects.

[0014] The target data includes a distribution of probabilities for each class of the input data, and the auxiliary loss may include a loss calculated based on the distance between the distribution of probabilities for each class of the input data included in the target data and the distribution of probabilities for each class of the feature map.

[0015] A learning method for a neural network model according to one embodiment includes the steps of: receiving input data and target data; extracting feature maps from the input data; pooling the feature maps based on a scale factor and the probability of the class of objects present in the feature maps; inputting the input data into the neural network model to generate output data; comparing the output data, the target data, and the pooling auxiliary loss to determine the loss; and training the neural network model based on the loss.

[0016] The probability of an object belonging to a particular class in the feature map can determine the ratio of elements belonging to that class to the elements within a unit region of the feature map.

[0017] The target data includes the distribution of probabilities for each class present in each unit region of the input data, and the auxiliary loss may be based on the distance between the distribution of class probabilities in the feature map and the distribution of class probabilities in the corresponding regions of the input data in the target data.

[0018] The target data includes ground truth labels, and the step of determining the loss may include comparing the output data with the ground truth labels to determine the loss.

[0019] The method further includes the step of pooling the target data to make the size of the target data the same as the size of the feature map, wherein the auxiliary loss may be based on the distance between the distribution of class probabilities of the feature map and the distribution of class probabilities of the pooled target data. [Effects of the Invention]

[0020] According to the present invention, a method and apparatus for learning a neural network model to improve the performance of the neural network model can be provided.

Brief Description of Drawings

[0021] [Figure 1] It is a diagram for explaining a learning device according to an embodiment. [Figure 2] It is a diagram for explaining a pooling operation executed by a neural network model. [Figure 3A] It is a flowchart for explaining a pooling operation method according to an embodiment. [Figure 3B] It is a flowchart for explaining a pooling operation method according to an embodiment. [Figure 3C] It is a flowchart for explaining a pooling operation method according to an embodiment. [Figure 3D] It is a flowchart for explaining a pooling operation method according to an embodiment. [Figure 3E] It is a flowchart for explaining a pooling operation method according to an embodiment. [Figure 3F] It is a flowchart for explaining a pooling operation method according to an embodiment. [Figure 4] It is a flowchart showing a learning method according to an embodiment. [Figure 5A] It is a diagram showing the result of executing pooling according to an embodiment. [Figure 5B] It is a diagram showing the result of executing pooling according to an embodiment. [Figure 5C] It is a diagram showing the result of executing pooling according to an embodiment. [Figure 6A] It is a diagram for explaining the structure of a neural network model according to various embodiments. [Figure 6B] It is a diagram for explaining the structure of a neural network model according to various embodiments. [Figure 6C] It is a diagram for explaining the structure of a neural network model according to various embodiments. [Figure 6D] It is a diagram for explaining the structure of a neural network model according to various embodiments. [Modes for carrying out the invention]

[0022] Embodiments will be described in detail below with reference to the attached drawings. However, the specific structural or functional descriptions disclosed herein are illustrative only for the purpose of illustrating embodiments, and embodiments can be carried out in various different forms. The present invention is not limited to the embodiments described herein. All modifications, equivalents, or substitutions to embodiments should be understood to be included within the scope of the rights.

[0023] The terms used in the embodiments are for illustrative purposes only and should not be construed as intended to limit them. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “includes” or “having” indicate the presence of features, figures, steps, actions, components, parts, or combinations thereof described in the specification, and should not be understood as preemptively excluding the possibility of the presence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0024] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which this embodiment belongs. Commonly used, predefined terms should be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as ideal or overly formal unless expressly defined herein.

[0025] Furthermore, when explaining with reference to the attached drawings, the same reference numerals will be assigned to the same components regardless of the reference numerals in the drawings, and redundant explanations will be omitted. In the description of embodiments, if a specific explanation of related prior art is deemed to unnecessarily obscure the gist of the embodiment, such detailed explanation will be omitted.

[0026] Furthermore, terms such as 1st, 2nd, A, B, (a), (b), etc., may be used in the description of the components of the embodiments. Such terms are merely used to distinguish a component from other components, and do not limit the essence, order, or sequence of the component in question.

[0027] Components that have functions common to any one embodiment will be described using the same name in the other embodiments as well. Unless otherwise stated, the descriptions in any one embodiment will also apply to the other embodiments, and specific descriptions will be omitted to the extent of overlap.

[0028] Figure 1 is a diagram illustrating a learning device according to one embodiment.

[0029] The present invention enhances the performance of the neural network model 103 by minimizing losses incurred during the pooling process by using class-specific probabilities when performing pooling during the learning process of the neural network model 103.

[0030] Referring to Figure 1, the learning device 101 of the neural network model 103 includes a processor 102. The processor 102 may perform learning methods according to various embodiments. Referring to Figure 1, the learning device 101 includes a neural network model 103 that generates output data from input data. All operations processed by the neural network model 103 can be performed by the processor 102.

[0031] Processor 102 can read / use neural network data, such as text data, audio data, image data, feature map data, and kernel data, from memory (not shown), and execute the neural network model 103 using the read / written data. Once the neural network model 103 is executed, processor 120 can repeatedly perform convolution operations between the input feature map and the kernel to generate data for the output feature map. Here, the number of convolution operations can be determined by various factors such as the number of channels in the input feature map, the number of channels in the kernel, the size of the input feature map, the size of the kernel, and the precision of the values. The neural network model 103 is implemented in a complex architecture in which processor 102 performs convolution operations up to several hundred million to several billion times, and processor 102 accesses memory for convolution frequently, with the frequency of processor 120 accessing memory for convolution operations increasing rapidly.

[0032] The neural network model 103 included in the learning device 101 may include an input layer for receiving input data, a hidden layer for performing operations on the input data, and an output layer for generating output data. In one embodiment, the neural network model 103 may be a convolutional neural network model 103. However, the type or structure of the neural network model 103 is not limited to a specific example. Various learning methods according to various embodiments of the present invention may be applied to various types of neural network models 103 for pooling.

[0033] The input and output data may vary depending on the intended use of the neural network model 103. In one embodiment, the input data may be an image, and the output data may be objects contained within the image. The input data is not limited to the examples described, and the output data may be determined differently depending on the purpose of the neural network model 103.

[0034] Referring to Figure 1, the target data represents the correct labels (e.g., ground truth) in training the neural network model 103 using supervised learning. The neural network model 103 may be trained to generate output data identical to the target data from the input data. In one embodiment, the processor 102 can compare the target data with the output data to determine the loss and update the parameters of the neural network model 103 to minimize the loss.

[0035] According to one embodiment, the neural network model 103 may include a plurality of operational blocks that extract feature maps from input data and pool the feature maps. Pooling means sampling and resizing the input data. According to one embodiment, data loss can be minimized during the pooling process. In various embodiments of the present invention, the neural network model 103 is used for various inference tasks. Inference tasks may include, for example, pattern recognition (e.g., image recognition, object detection, object recognition, face recognition, etc.), sequence recognition (e.g., speech, gesture, and written text recognition, machine translation, machine interpretation, machine sound range, etc.), control (e.g., vehicle control, process control, etc.), segmentation, depth estimation, recommendation services, decision making, medical diagnosis, financial applications, data mining, etc.

[0036] Figure 2 is a diagram illustrating the pooling behavior performed in the neural network model.

[0037] Pooling is a process that reduces the size of data. For example, in a neural network model, the pooling process is a process to reduce the size of feature maps. Referring to Figure 2, the size of feature map 201 may be 224 × 224 × 64. Pooling can be performed on feature map 201 via 211 shown in Figure 2. The size of feature map 202 after pooling may be 112 × 112 × 64.

[0038] As shown in Figure 2, the image 203 corresponding to the feature map 201 is downsampled via 213. Referring to Figure 2, if the size of the image 203 corresponding to the feature map 201 is 224 × 224 and the pooling scale factor is 2, the size of the pooled image 204 may be 112 × 112.

[0039] Figures 3A to 3F are flowcharts illustrating a pooling operation method according to one embodiment.

[0040] Figure 3A shows input data, which is an image containing one or more objects. In one embodiment, the input data includes an image containing multiple objects, and the pixels contained in the image are divided into classes corresponding to each of the multiple objects. For example, there may be 20 classes for a city landscape dataset.

[0041] Figure 3B shows a feature map generated from a specific region in the input data. The feature map may contain multiple unit regions (for example, 16 in Figure 3B). In one embodiment, a unit region means a pooling unit region. The unit region is determined by the pooling scale factor. In one embodiment, if the pooling scale factor is 2, the horizontal and vertical sizes of the feature map may be sampled to 1 / 2 by pooling.

[0042] Referring to Figure 3B, a unit region consists of multiple elements. Each element is classified into one of several classes. The unit region 301 shown in Figure 3B may consist of elements of classes 1 and 13. For example, an element means one or more pixels. Each class means an object. For example, in Figure 3A, class (0) 311 may be a road, class (1) 312 a sidewalk, class (13) 313 a vehicle, and class (19) 314 ignore. Ignore means an area that is ignored during the image processing process.

[0043] The neural network model may perform pooling of unit regions of feature maps (e.g., downsampling). In one embodiment, the neural network model may perform feature map pooling based on the probability of each class of feature maps. In one embodiment, pooling is performed based on the following formula (1).

number

[0044] In one embodiment, the neural network model can determine the probability for each class based on the ratio of elements belonging to any given class among the elements contained in a unit region. Referring to Figure 3B, the unit region 301 may contain four elements. The four elements may consist of three elements of class (1)312 and one element of class (13)313. The ratio of elements of class (1)312 may be 0.75, and the ratio of elements of class (13)313 may be 0.25.

[0045] Figure 3C shows the feature maps pooled by class. In one embodiment, Figure 3C shows the case where the pooling scale factor is set to 2. In one embodiment, when there are a total of 20 classes, the feature map has 1 channel in Figure 3B, but the feature map has 20 channels in Figure 3C.

[0046] Referring to Figure 3C, since there are no elements of class (0)311 in unit region 301, the value corresponding to unit region 301 in the pooled feature map 302 for class (0)311 is determined to be 0. Referring to Figure 3A(c), since the ratio of elements of class (1)312 in unit region 301 is 0.75, the value corresponding to unit region 301 in the pooled feature map 303 for class (1)312 may be determined to be 0.75. Referring to Figure 3C, since the ratio of elements of class (13)313 in unit region 301 is 0.25, the value corresponding to unit region 301 in the pooled feature map 304 for class (13)313 may be determined to be 0.25.

[0047] The neural network model can perform pooling based on the proportion of elements of a specific class among the total elements contained in a unit region determined by a pooling scale factor. The channels in the pooled feature map correspond to the number of classes contained in the input data. Referring to Figures 3A to 3C, if the scale factor is 2, the size of the pooled unit region is half the size of the unit region before pooling. 2 It may be decided that way.

[0048] Figure 3E shows input data, which is an image containing one or more objects. According to one embodiment, the input data includes an image containing multiple objects, and the pixels contained in the image may be divided into classes corresponding to each of the multiple objects.

[0049] Figure 3E shows a feature map generated from a specific region in the input data. The feature map may contain multiple unit regions (for example, four in Figure 3E). Figure 3E is the case when the scale factor is 4. Referring to Figures 3D to 3F, the horizontal and vertical sizes of the feature map may be sampled to 1 / 4 each by pooling.

[0050] Referring to Figure 3E, a unit region is composed of multiple elements. The unit region 321 shown in Figure 3E may be composed of elements of classes 12, 8, and 3. For example, in Figure 3E, class 12 may be a person, class 8 a plant, class 3 a wall, and class 17 a motorcycle.

[0051] The neural network model may perform pooling (e.g., downsampling) of unit regions of feature maps. In one embodiment, the neural network model may perform feature map pooling based on the probability of each class of feature maps. In one embodiment, the neural network model may determine the probability of each class based on the ratio of elements belonging to any given class among the elements contained in a unit region.

[0052] Referring to Figure 3E, the unit region 321 may contain 16 elements. The 16 elements may consist of 9 elements of class 12, 8 elements of class 8, and 2 elements of class 3. The ratio of elements of class 12 may be 0.5625, and the ratio of elements of class 8 may be 0.3125.

[0053] Figure 3F shows the feature maps pooled by class. In one embodiment, Figure 3F shows the case where the pooling scale factor is set to 4. In one embodiment, if there are a total of 20 classes, the feature map has 1 channel in Figure 3E, but the feature map may have 20 channels in Figure 3F.

[0054] Referring to Figure 3F, since there are no elements of class (0)331 in unit region 321, the value corresponding to unit region 321 in the pooled feature map 302 for class (0)331 may be determined to be 0.

[0055] Referring to Figure 3F, the ratio of elements of class (12)334 to unit region 321 is 0.5625. Therefore, in the pooled feature map 325 for class (12)334, the value corresponding to unit region 321 may be determined to be 0.5625.

[0056] Referring to Figure 3F, since the ratio of elements of class 8 (333) to unit region 321 is 0.3125, the value corresponding to unit region 321 in the pooled feature map 324 for class 8 (333) may be determined to be 0.3125.

[0057] Referring to Figure 3F, the ratio of elements belonging to class (3) 332 to the unit region 321 is 0.125. Therefore, in the pooled feature map 323 for class (3) 332, the value corresponding to the unit region 321 may be determined to be 0.125. The channels of the pooled feature map correspond to the number of classes included in the input data. Referring to Figure 3B, when the scale factor is 4, the size of the pooled unit region is 1 / 4 of the size of the unit region before pooling. 2 It may be decided that way.

[0058] Figure 4 is a flowchart of a learning method according to one embodiment. The operations in Figure 4 are performed in the order and manner shown in Figure 4, but the order of some operations may be changed or some operations may be omitted without deviating from the idea and scope of the exemplary example described. Many of the operations shown in Figure 4 may be performed in parallel or simultaneously. One or more blocks and combinations of blocks shown in Figure 4 can be realized by a combination of computer instructions and special-purpose hardware-based computers or special-purpose hardware, such as a processor that performs a specific function. Furthermore, the description of Figures 1 to 3 may also apply in the following description of Figure 4, and Figures 1 to 3 are also applicable to Figure 1 and may be included by reference. Therefore, the above description is not repeated here.

[0059] In step S401, the learning device identifies the input data and the target data. In one embodiment, the input data may be an image. The input data may include an image containing multiple objects, and the pixels included in the image may be divided into classes corresponding to each of the multiple objects.

[0060] In step S402, the learning device inputs the input data into the neural network model to generate output data. In one embodiment, the neural network model may be a convolutional neural network model. In one embodiment, the neural network model may include multiple operation blocks that extract feature maps from the input data and perform pooling on the feature maps.

[0061] In step S403, the learning device compares the output data with the target data to determine the loss. The target data may include not only the correct labels but also the distribution of probabilities for each class of the input data. The distribution of probabilities for each class may exist for each unit region of the input data.

[0062] The loss includes not only the difference between the output data and the target data, but also the loss determined based on the difference between the feature map pooled by the neural network model and the target data. The loss determined based on the difference between the pooled feature map and the target data is defined as the auxiliary loss.

[0063] According to one embodiment, the learning device can pool target data. The size of the target data may be larger than the size of the pooled feature maps. In order to determine the auxiliary loss, the learning device can pool the target data such that the size of the target data is the same as the size of the feature maps output from each layer.

[0064] For example, the learning device may pool the target data according to a distribution of probabilities for each class of input data included in the target data. The learning device can determine the probability for each class by pooling the target data by class, similar to how feature maps are pooled.

[0065] The auxiliary loss is calculated based on the distance between the distribution of class-specific probabilities in the input data contained in the target data and the distribution of class-specific probabilities in the feature map. In one embodiment, Kullback-Leibler divergence, mean square error (MSE), etc., may be used to calculate the distance between the distribution of class-specific probabilities in the input data and the distribution of class-specific probabilities in the feature map. However, in one embodiment, the method for calculating the distance between the distribution of class-specific probabilities in the input data and the distribution of class-specific probabilities in the feature map is not limited to a specific example.

[0066] For example, for a specific location in the input data, the element values ​​of each class in the pooled feature map may be represented by a probability distribution for each class. The processor can determine an auxiliary loss by calculating the probability distribution and distance for each class contained in the target data for the same location.

[0067] In step S404, the learning device trains a neural network model based on the loss. The loss includes a loss determined based on the difference between the output data and the target data, and an auxiliary loss determined based on the difference between the output data and the target data. The learning device's processor can update the parameters of the neural network model to minimize the loss.

[0068] Figures 5A to 5C show the results of pooling performed according to one embodiment.

[0069] Figure 5A shows images pooled using the nearest neighbor pooling method for all classes for each scale factor. Figure 5B shows images pooled using the nearest neighbor pooling method for the human class for each scale factor. Figure 5C shows images pooled using the pooling method according to one embodiment of the present invention for the human class for each scale factor.

[0070] Referring to Figure 5B, the larger the scale factor, the greater the potential loss of information regarding the boundary between people and objects. However, referring to Figure 5C, which relates to one embodiment of the present invention, it can be seen that even with a large scale factor, some information regarding the boundary between objects and people is retained. Therefore, the pooling described above prevents information loss in determining representative values ​​when downsampling feature maps.

[0071] Figures 6A to 6D are diagrams illustrating the structure of neural network models according to various embodiments.

[0072] Figures 6A to 6D all represent neural network models with the same structure, but show cases where the number and location of pooling performed according to one embodiment differ from one another. The neural network models shown in Figures 6A to 6D may include groups (e.g., groups 1-4) composed of multiple convolutional hierarchies. A stem refers to an input hierarchy. Referring to Figures 6A to 6D, auxiliary heads 601-604 are used to determine auxiliary losses based on loss calculations for feature outputs from one or more intermediate neural network hierarchies. In Figures 6A to 6D, H, W, and C refer to the height, width, and channels of the feature map, respectively.

[0073] In Figure 6A, pooling according to one embodiment is performed on the feature map where the calculation for group 3 has been performed via the auxiliary head 601, and the auxiliary loss can be determined. In Figure 6B, pooling according to one embodiment is performed on each feature map where the calculation for groups 1 to 4 has been performed via the auxiliary head 602, and the auxiliary loss can be determined. In Figure 6C, pooling according to one embodiment is performed on the feature map where the calculation for the semantic head has been performed via the auxiliary head 603, and the auxiliary loss can be determined. In Figure 6D, pooling according to one embodiment is performed on each feature map where the calculation for groups 1 to 4 and the semantic head has been performed via the auxiliary head 604, and the auxiliary loss can be determined.

[0074] The training apparatus, other apparatus, devices, units, modules, and components described in relation to Figures 1 to 4 may be implemented as hardware components. Where appropriate, examples of hardware components used to perform the work described in this application may include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and other electronic components that perform the work described. As a different example, one or more hardware components that perform the operation described in this application may be implemented by computing hardware, for example, one or more processors or computers. The processor or computer may be implemented by logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other devices or combinations of devices configured to execute in response to instructions in a defined manner to achieve the desired results. As an example, the processor or computer may include, or be connected to, one or more memories that store instructions or software executed by the processor or computer. Hardware components embodied by a processor or computer can execute instructions or software, such as an operating system (OS) and one or more software applications running on the OS, in order to perform the described tasks. Hardware components may access, manipulate, process, generate, and store data in response to instruction or software execution. For simplicity, the terms “processor” or “computer” may be used to describe the examples described in this application, but in different examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or various types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller.One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or other processors and other controllers. One or more processors, or a processor and a controller, can implement a single hardware component or two or more hardware components. For example, a hardware component may include a single processor, an independent processor, a parallel processor, single instruction single data (SISD) multiplexing, single instruction multiple data (SIMD) multiplexing, MISD (multiple instruction single data) multiplexing, MIMD (multiple instruction multiple data) multiplexing, a controller and an ALU (Arithmetic Logic Unit), a DSP, a microcomputer, an ASIC (Application-Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a PLU (Programmable Logic Unit), a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural Processing Unit), or other devices that execute in response to instructions in a defined manner.

[0075] The methods illustrated in Figures 1 to 6 for performing the operations described herein can be performed by computing hardware, for example, one or more processors or a computer, and can be implemented by executing instructions or software as described above to perform the operations described herein performed by the methods. For example, a single operation or two or more operations may be performed by a single processor, two or more processors, or a processor and a controller. One or more operations may be performed by one or more processors, or a processor and a controller, and one or more other operations may be performed by one or more other processors, or a controller different from the other processors. One or more processors, or a processor and a controller, may perform a single task or two or more tasks.

[0076] Instructions or software that control a processor or computer to embody hardware components and perform the methods described above are made up of computer programs, code segments, instructions, or combinations thereof that individually or collectively instruct or configure the processor or computer to operate as a machine or special-purpose computer in order to perform the work performed by the hardware components. For example, the instructions or software include machine code that is executed directly by the same processor or computer as the machine code generated by the compiler. For other examples, the instructions or software include higher-level code that is executed by the processor or computer using an interpreter. For example, the instructions or software include at least one of applets, DLLs (Dynamic Link Libraries), middleware, firmware, device drivers, or application programs that store methods for training neural network models. Those skilled in the art can easily create instructions or software based on block diagrams and flowcharts illustrated in the drawings and corresponding descriptions of the specification that disclose algorithms for performing the work performed by the hardware components and methods described above.

[0077] Instructions or software for controlling a processor or computer to embody hardware components and perform the methods described above, and all associated data, data files, or data structures, may be recorded, stored, or fixed on a storage medium readable by one or more non-temporary computers.Examples of non-temporary computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), magnetic RAM (MRAM), spin-transfer torque (STT)-MRAM, static random-access memory (SRAM), thyristor RAM (T-RAM), zero-capacitor RAM (Z-RAM), twin-transistor RAM (TTRAM), conducted-bridge RAM (CBRAM), and ferroelectric RAM. (FeRAM), Phase Change RAM (PRAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nanofloating Gate Memory (NFGM), Holographic Memory, Molecular Electronic Memory Device, Insulator Resistivity Change Memory, Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash Memory, Non-Volatile Memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD(Registered Trademark)-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray (registered trademark), or optical disc storage, hard disk drives (HDDs), solid state drives (SSDs), flash memory, multimedia card micro or card (e.g., Security Digital (SD) or Extreme Digital (XD)), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid state disks, and other devices configured to store instructions or software and associated data, data files or data structures in a non-temporary manner, to provide instructions or software and associated data, data files, and to transmit data structures to a processor or computer so that the processor or computer can execute instructions.For example, instructions or software, and all associated data, data files, and data structures may be distributed through a network-connected computer system, and the instructions and software, and all associated data, data files, and data structures may be stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0078] The hardware device described above may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0079] As described above, embodiments have been illustrated with limited drawings, but a person with ordinary skill in the art can apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the described systems, structures, devices, circuits, and other components may be combined or assembled in a different manner than described, and may be replaced or substituted by other components or equivalents, while still achieving appropriate results.

[0080] Therefore, other embodiments, other embodiments, and those equivalent to the claims described below also fall within the scope of the claims. [Explanation of Symbols]

[0081] 101 Learning device 102 processors 103 Neural Network Model Feature maps 201 and 202 301 Unit Area Classes 311, 312, 313, and 314 601, 602, 603, 604 Auxiliary heads

Claims

1. A learning method for neural network models, which is performed by the processor of a learning device. The steps include receiving input data and target data, The steps include: pooling the feature maps extracted from the input data using the neural network model based on the probability of each class in the feature maps; The steps include inputting the aforementioned input data into the neural network model to generate output data, The steps include determining the loss based on a comparison of the output data and the target data, and determining the auxiliary loss based on the results of the pooling, A step of training the neural network model based on the loss and the auxiliary loss, Learning methods, including

2. The auxiliary loss is determined based on the difference between the target data and the pooling result. The learning method according to claim 1.

3. The neural network model determines the probability for each class based on the ratio of elements belonging to any given class among the elements included in the unit region of the feature map, which is determined by the pooling scale factor. The learning method according to claim 1.

4. The aforementioned input data includes an image containing multiple objects, The pixels contained in the aforementioned image are divided into classes corresponding to each of the multiple objects. The learning method according to claim 1.

5. The target data includes a distribution of probabilities for each class of the input data, The auxiliary loss includes a loss calculated based on the distance between the distribution of class-specific probabilities of the input data included in the target data and the distribution of class-specific probabilities of the feature map. The learning method according to claim 1.

6. A computer-readable recording medium for storing computer programs that include instructions, When the aforementioned instruction is executed, the computer will: To carry out the method described in any one of claims 1 to 5, A storage medium that can be read by a computer.

7. A learning device for neural network models, The learning device includes a processor, The aforementioned processor, Receive input data and target data, The feature maps extracted from the input data are pooled by the neural network model based on the probability of each class in the feature maps. The input data is input to the neural network model to generate output data. Based on a comparison of the output data and the target data, the loss is determined, and based on the results of the pooling, the auxiliary loss is determined. The neural network model is trained based on the aforementioned loss and the aforementioned auxiliary loss. Learning device.

8. The auxiliary loss is determined based on the difference between the target data and the pooling result. The learning device according to claim 7.

9. The neural network model determines the probability for each class based on the ratio of elements belonging to any given class among the elements included in the unit region of the feature map, which is determined by the pooling scale factor. The learning device according to claim 7.

10. The aforementioned input data includes an image containing multiple objects, The pixels contained in the aforementioned image are divided into classes corresponding to each of the multiple objects. The learning device according to claim 7.

11. The target data includes a distribution of probabilities for each class of the input data, The auxiliary loss includes a loss calculated based on the distance between the distribution of class-specific probabilities of the input data included in the target data and the distribution of class-specific probabilities of the feature map. The learning device according to claim 7.

12. A learning method for neural network models, which is performed by the processor of a learning device. The steps include receiving input data and target data, The steps include extracting a feature map from the aforementioned input data, The steps include pooling the feature map based on the scale factor and the probability of the object class present in the feature map, The steps include inputting the aforementioned input data into the neural network model to generate output data, The steps include determining the loss based on a comparison of the output data and the target data, and determining the auxiliary loss based on the results of the pooling, A step of training the neural network model based on the loss and the auxiliary loss, Learning methods, including

13. The probability of an object belonging to a particular class in the feature map determines the ratio of elements belonging to that class to elements within a unit region of the feature map. The learning method according to claim 12.

14. The target data includes the probability distribution of each class present in each unit region of the input data. The auxiliary loss is based on the distance between the distribution of class probabilities in the feature map and the distribution of class probabilities in the corresponding region of the input data in the target data. The learning method according to claim 12.

15. The aforementioned target data includes the ground truth label, The step of determining the loss includes the step of comparing the output data with the correct label to determine the loss. The learning method according to claim 14.

16. The method further includes the step of pooling the target data in order to make the size of the target data the same as the size of the feature map, The auxiliary loss is based on the distance between the distribution of class probabilities for the feature map and the distribution of class probabilities for the pooled target data. The learning method according to claim 14.

Images