Data processing device and method thereof

Abstract

To reduce overhead in a case of selectively achieving any recognition task in multi-task recognition.SOLUTION: Provided is a data processing device including: storage means for storing a plurality of types of parameter groups to be used for a plurality of types of recognition tasks; selection means for selecting two or more recognition tasks to be executed from among the plurality of types of recognition tasks; holding means for holding parameter groups; transfer means for successively transferring parameter groups to be used for the two or more recognition tasks from the storage means to the holding means; and execution means for successively executing the two or more recognition tasks using the parameter groups held in the parameter holding means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a data processing device and method for performing multitask recognition to detect multiple types of objects from image data. [Background technology]

[0002] Hierarchical computation methods, such as the Convolutional Neural Network (CNN), have been attracting attention. Using such methods, pattern recognition methods that are robust against changes in the recognition target based on deep learning technology have been researched. For example, Non-Patent Document 1 discloses various application and implementation examples.

[0003] Patent Document 1 discloses a method for efficiently learning multiple recognition tasks by utilizing common features of CNN, while Patent Document 2 discloses the configuration of various parameter groups (hereinafter referred to as operation parameters) for efficiently processing CNN. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-140546 A [Non-patent literature]

[0005] [Non-Patent Document 1] Yann LeCun, Koray Kavukvuoglu and Clement Farabet: Convolutional Networks and Applications in Vision, Proc. International Symposium on Circuits and Systems (ISCAS'10), IEEE, 2010, Summary of the Invention [Problem to be solved by the invention]

[0006] In a multitask recognition device, there are cases where it is desired to selectively execute only a specific recognition task from among multiple recognition tasks depending on the use case. The technique described in Patent Document 1 discloses a method for executing multiple predetermined recognition tasks by utilizing common features, but when selectively executing a recognition task, it is necessary to replace the software of the multitask processing CPU, and it is not possible to execute it efficiently. Patent Document 2 discloses a method for efficiently processing CNN in hardware by referring to the operation parameters required to execute the recognition task. However, when selectively executing multitask recognition, it is necessary to prepare multiple operation parameters for each combination of selection, and the total amount of operation parameters increases. [Means for solving the problem]

[0007] In order to solve the above problems, according to one aspect of the present invention, a data processing device includes: A common parameter group for extracting common feature data from an input image that is commonly used in the plurality of types of recognition tasks. A storage means for storing a plurality of types of parameter groups, a selection means for selecting two or more recognition tasks to be executed from the plurality of types of recognition tasks, a storage means for storing parameter groups, and a parameter group used for the two or more recognition tasks. The common parameter group is placed before the other multiple types of parameter groups. a transfer means for sequentially transferring data from the storage means to the holding means; The common parameter group is used to extract the common feature data from the input image, and the common feature data is and an execution means for sequentially executing the two or more recognition tasks. Effect of the Invention

[0008] According to the present invention, it is possible to provide a data processing device that efficiently executes a plurality of recognition tasks that are selected in any combination depending on the use case. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a multitask recognition processing unit according to an embodiment. [Diagram 2]FIG. 1 is a diagram illustrating an image processing device including a multitask recognition processing unit according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a configuration of a convolution calculation unit according to an embodiment. [Figure 4] Diagram explaining an example of CNN configuration [Diagram 5] A diagram explaining an example of the configuration of a multitask recognition CNN. [Figure 6] FIG. 1 is a diagram for explaining an example of the configuration of a recognition task designation unit according to the first embodiment. [Figure 7] FIG. 1 is a diagram for explaining an example of an operation parameter set according to the first embodiment. [Figure 8] FIG. 1 is a flowchart for explaining a control unit according to an embodiment. [Figure 9] FIG. 1 is a diagram for explaining an operation time chart of a multitask recognition processing unit according to the first embodiment. [Figure 10] FIG. 11 is a diagram for explaining an example of the configuration of a recognition task designation unit according to the second embodiment. [Figure 11] FIG. 13 is a diagram for explaining an example of an operation parameter set according to the second embodiment. [Figure 12] FIG. 13 is a diagram for explaining an example of an operation parameter set according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. Note that the configurations described in the following embodiments are representative examples, and the scope of the present invention is not necessarily limited to those specific configurations.

[0011] <Embodiment 1> 2 shows an example of the configuration of a data processing device that uses the multitask recognition processing according to this embodiment. This device has a function for recognizing the position of a specific object or a specific area in image data. In addition, a task here is defined as a unit of recognition processing for recognizing a specific target.

[0012] In Fig. 2, 201 is a multitask recognition processing unit. 202 is an image input unit, which includes an optical system such as a lens, and a photoelectric conversion device such as a CCD (Charge-Coupled Devices) or CMOS (Complimentary Metal Oxide Semiconductor) sensor. It also includes a driver circuit / AD converter for controlling the sensor. The multitask recognition processing unit 201 executes multiple types of recognition processing (recognition tasks) for image data acquired by the image input unit 202.

[0013] Reference numeral 203 denotes a CPU (Central Processing Unit) that controls the entire imaging apparatus. Reference numeral 204 denotes a ROM (Read Only Memory) that stores commands that define the operation of the CPU 203 and various parameter data. Reference numeral 205 denotes a RAM (Random Access Memory) that is used as a work memory required for the operation of the CPU 203. The RAM 205 is composed of a large-capacity DRAM (Dynamic Access Memory) or the like. Reference numeral 206 denotes a user interface unit that includes a display device that displays the recognition results and a GUI (Graphical User Interface) that specifies recognition tasks. Reference numeral 207 denotes a data bus, which is a data transfer path between each device.

[0014] The multitask recognition processing unit 201 executes two or more selected recognition tasks according to instructions from the CPU 203, and stores the results in the RAM 205. The CPU 203 uses the multitask recognition results to provide various applications. For example, the recognition results are fed back to the image input unit 202 and used for optical system focus control, sensor exposure control, white balance control, etc.

[0015] 1 is a diagram showing the functional configuration of a multitask recognition processing unit 201 according to this embodiment. In this embodiment, multitask recognition processing is executed by CNN. Reference numeral 101 denotes an external bus interface unit, which is an interface that allows a CPU 203, a DMAC 102, and a control unit 107 to access each other via a data bus 207. Reference numeral 102 denotes a DMAC (Direct Memory Access Controller), which controls various data transfers between the multitask recognition processing unit 201 and a RAM 205.

[0016] Reference numeral 103 denotes a convolution calculation processing unit, which executes a convolution calculation by referring to CNN coefficients (filter coefficients, described later) stored in a CNN coefficient buffer 104 and CNN feature data stored in a CNN feature buffer 105. The CNN feature buffer 105 holds the CNN calculation results of a layer preceding the layer to be processed as CNN feature data to be referenced, and further holds the calculation results of the layer to be processed as CNN feature data to be referenced in the next layer.

[0017] The CNN coefficient buffer 104 is a coefficient storage unit that supplies coefficients with low delay to the convolution calculation processing unit 103, and is made up of a high-speed SRAM (Static RAM) and registers. The CNN coefficient buffer 104 stores coefficients used in multiple hierarchical calculations.

[0018] The CNN feature buffer 105 is a storage unit that stores the calculation results of the convolution calculation processing unit 103 or the nonlinear transformation processing unit 106. Like the CNN coefficient buffer 104, the CNN feature buffer 105 is also configured with a high-speed SRAM or the like.

[0019] The nonlinear conversion processing unit 106 performs nonlinear conversion on the output of the convolution calculation result calculated by the convolution calculation processing unit 103. The nonlinear conversion process uses well-known methods such as ReLU (Rectified Linear Unit) and sigmoid function. In the case of ReLU, this can be achieved by threshold processing, and in the case of the sigmoid function, values ​​are converted using a lookup table or the like.

[0020] The control unit 107 controls the operation of the DMAC, which manages the CNN calculation by convolution calculation and the transfer of various data. The control unit 107 is composed of a hardware sequencer and a simple CPU that control access to the convolution calculation processing unit 103, the CNN coefficient buffer 104, and the CNN feature buffer 105. Reference numeral 108 denotes a recognition task designation unit, which is a register set for designating a recognition task to be executed by the CPU 203. Reference numeral 109 denotes a storage destination address designation unit, which is a register for designating an address on the RAM 205 where the CPU 203 stores an operation parameter set, which will be described later.

[0021] The convolution calculation processing unit 103 sets the size of the kernel (filter coefficient matrix: CNN coefficient) of the convolution calculation to columnSize×rowSize. When the number of feature planes (the definition of feature planes will be described later) in the previous layer is L, one CNN feature is calculated by the convolution calculation shown in the following formula (1).

[0022]

number

[0023] Where: input(x,y): Reference pixel value at two-dimensional coordinates (x,y) output(x,y): Calculation result in two-dimensional coordinates (x,y) weight(column, row): CNN coefficient at coordinate (x+column, y+row) L: Number of feature faces in the previous layer columnSize, rowSize: Horizontal and vertical size of the 2D convolution kernel It is.

[0024] Generally, in CNN calculation processing, multiple convolution calculation kernels are used to scan the input image or the feature plane of the previous layer on a pixel-by-pixel basis, and multiply-and-accumulate operations are repeated according to formula (1). The final product-and-accumulate result is then nonlinearly transformed (activation processing) to calculate the feature plane of the target layer. In other words, the nonlinear calculation result of multiple spatial filter calculations and their sum becomes the pixel data of a single feature plane. CNN coefficients are equivalent to spatial filter coefficients, and are also called coupling coefficients because they express the coupling relationship between feature planes. In reality, multiple feature planes are generated for each layer.

[0025] An example of the configuration of the convolution calculation processing unit 103 is shown in Figure 3. The convolution calculation processing unit 103 is made up of a multiplier 301 and an accumulator 302, and executes the convolution calculation processing of equation (1). The nonlinear transformation processing unit 106 executes nonlinear transformation processing on the result. Hereinafter, CNN features are referred to as feature planes because they can be expressed in multiple two-dimensional map formats. In a normal CNN, the above process is repeated for the number of feature planes to be generated. The calculated feature planes are stored in the CNN feature buffer 105.

[0026] Reference numeral 303 denotes a control unit for controlling the convolution calculation process, and cooperates with the control unit 107 of the multitask recognition processing unit 201 to realize CNN calculation. The control unit 303 has a storage unit for setting information related to the operation corresponding to the configuration of the CNN to be processed, and realizes hierarchical calculation processing according to the information. It also has a function for controlling the CNN coefficient buffer 104 and the CNN feature buffer 105, and processes all layers of the CNN based on the control of a predetermined buffer for each layer. Since the CNN coefficients are composed of a large amount of data, all the coefficients are stored in the RAM 205, and only the data required for the calculation is transferred to the CNN coefficient buffer 104 by the DMAC 102 for use.

[0027] The network for realizing recognition processing using CNN is explained using Figure 4. For the sake of explanation, we will explain the case where a small CNN (3 layers) is used. In reality, it is often configured with many feature planes and multiple layers.

[0028] 401 is an input image of the input layer, which corresponds to raster-scanned image data of a predetermined size when CNN arithmetic processing is performed on image data. 403a to 403c indicate feature planes of the first layer 408. Since the feature planes are the processing results of raster-scanned image data, the processing results are also composed of two-dimensional planes as described above. The feature planes 403a to 403c are calculated by convolution calculation and nonlinear processing on the input image 401. For example, 403a is calculated by a convolution calculation using a two-dimensional convolution kernel diagrammatically shown as 4021a and a nonlinear transformation of the calculation result. 405a is the result of performing a convolution calculation on the feature planes 403a to 403c using convolution kernels 4041a to 4043a, respectively, and performing a nonlinear transformation process on the sum of the results. In CNN, feature plane calculations are sequentially performed by hierarchical processing using the convolution kernels in this way.

[0029] When a predetermined learning has been performed on the CNN, for example, in a task whose main purpose is to detect the position of an object, the feature plane 407 of the final layer has a high value of data corresponding to the position of the detection target. Also, in a task whose main purpose is to determine the area of ​​the object, the data value of the area of ​​the object has a high value. Learning is an act of determining CNN coefficients, and is assumed to have been performed in advance by a device other than the image processing device. Various well-known methods can be applied to the learning. The control unit 303 in the convolution calculation processing unit 103 controls the storage control of the feature plane data required for processing the CNN in the CNN feature buffer 105 and the transfer of the CNN coefficients to the multiplier 301.

[0030] Next, a network configuration for performing multitask recognition will be described. FIG. 5 is a diagram for diagrammatically explaining an example of a multitask configuration. 501 is image data of an input image, and 502 is a common CNN for extracting features common to each recognition task, which is configured as a network consisting of multiple layers as described with reference to FIG. 4. 503 is common feature data extracted as features common to each recognition task, and corresponds to the output of the final layer of the common CNN 502. The common feature data 503 consists of multiple feature planes to hold various features. 504a to d are task CNNs 1 to 4 for performing a recognition processing task of a predetermined object, and each performs recognition processing of a different object. The recognition objects are different, for example, four types of a person's head, torso, hands, and feet, or four types of a person, a dog, a cat, and a bird. 505a to d are feature planes that become recognition results 1 to 4 of each recognition task.

[0031] Normally, when multiple recognition tasks are executed, image data needs to be transferred for each recognition task. However, in the multitask recognition of this embodiment, multiple recognition tasks can be executed with one image data transfer, reducing the overhead associated with image transfer. This allows recognition processing of multiple targets to be executed at high speed. In addition, by using common feature data for multiple recognition tasks, the configuration of multiple task CNNs can be made compact.

[0032] 6 is a diagram showing the configuration of the recognition task designation unit 108. The recognition task designation unit 108 is made up of registers accessible from the CPU 203 and the control unit 107. Reference numerals 601 to 604 are registers corresponding to recognition tasks 1 to 4, respectively, and each is made up of a 1-bit flip-flop or the like.

[0033] Prior to execution of recognition processing, the CPU 203 sets a value (=1) in one of the registers 601-604 corresponding to the recognition task to be executed via the external bus I / F unit 101. The control unit 107 controls the processing operation of tasks CNN1-4 by sequentially transferring parameters required for the operation of the specified recognition task according to the contents set in the registers 601-604. Similarly to the recognition task designation unit 108, the storage address designation unit 109 designates the storage destination of the operation parameter set, and is configured from a register accessible from the CPU 203 and the control unit 107 (a flip-flop with a number of bits capable of expressing a storage address).

[0034] Fig. 7 is a diagram showing the configuration of an operation parameter set for realizing multitask processing. The operation parameter set shown in Fig. 7 is an operation parameter consisting of a parameter group including the filter coefficients (CNN coefficients) of the convolution kernel based on the learning results of CNN and setting values ​​required to execute CNN. The operation parameter set is a collection of operation parameters corresponding to feature extraction and multiple tasks.

[0035] Reference numeral 702 denotes common feature extraction operation parameters 0, which are a common parameter group for the common CNN 502 that extracts common feature data from each task. Reference numerals 703 to 706 denote operation parameters corresponding to recognition tasks 1 to 4, respectively. Each operation parameter is made up of various parameters such as filter coefficients previously learned by a known CNN learning tool, and setting values ​​necessary for operating the hardware. The setting values ​​here include information that specifies the configuration of the CNN, such as the number of hierarchical layers and the number of feature planes to be processed, information regarding the storage destination of the feature planes in the CNN feature buffer 105, information regarding the storage destination of the coefficients to be stored in the CNN coefficient buffer, and the like.

[0036] The operation parameter set is stored in the RAM 205 by the CPU 203 before the start of the multitask recognition process. 707 is a register definition of the recognition task specification unit 108 corresponding to the operation parameter set. For example, when 1 is set in the register 1 (601), the operation parameter 1 is used to execute the recognition task 1.

[0037] Reference numeral 701 denotes offset information required for accessing the common feature extraction operation parameters and operation parameters 1 to 4. Of these, offset 0 is offset information relative to the top address of the storage destination of the common feature extraction operation parameters. Also, offsets 1 to 4 are offset information relative to the top addresses of the storage destinations of the operation parameter sets on RAM 205 for operation parameters 1 to 4, respectively. Also, it is assumed that the data size of each operation parameter is recorded in the offset information.

[0038] The storage destination of the operation parameters corresponding to the task to be recognized can be obtained by adding the address information stored in the storage address designation unit 109 and the address information described in the offset information. The control unit 107 grasps the recognition task to be executed according to the setting contents 707 of the recognition task designation unit 108, transfers the corresponding operation parameters from the operation parameter set stored in the RAM 205 to the multitask recognition processing unit, and executes the specified recognition task.

[0039] When a recognition task is executed, the operation parameters for extracting common features are always transferred and CNN processing is executed. In addition, the recognition tasks are executed in ascending or descending order of the bit positions specified for execution by the recognition task specification unit 108. The processing of the control unit 107 can be simplified by using a simple method for determining the execution order.

[0040] Fig. 8 is a flow chart for explaining the operation of the control unit 107. Fig. 9 is a time chart showing the operation of multitask recognition according to this embodiment. The operation of the present invention will be explained below with reference to Figs. 8 and 9.

[0041] Step S801 is an initialization process, which performs various initializations prior to execution of the multitask recognition processing unit 201 after startup. In step S802, when the CPU 203 issues an instruction to start processing, the control unit 107 checks the settings of the recognition task designation unit 108 to determine the recognition task to be executed. In step S803, the control unit 107 sets the DMAC for transferring operation parameters for the common feature CNN 502. The control unit 107 determines the address and size of the storage destination of the operation parameters from the offset information 701 of the operation parameter set, and sets the corresponding information in the DMAC. The convolution calculation processing unit 103 operates according to the contents of the data group transferred here.

[0042] In step S804, the DMAC for transferring the image data is set. In step S805, the DMAC 102 is instructed to start transfer and the convolution calculation processing unit 103 is instructed to start CNN calculation processing. The convolution calculation processing unit 103 executes CNN calculation processing on the image data according to the operating parameters transferred by the DMAC. Common features to be used in the recognition task are extracted by the CNN calculation processing here. The extracted common features are stored in the CNN feature buffer 105.

[0043] When the control unit 107 detects the completion of the CNN calculation, it then processes the setting of the DMAC for the transfer of the operation parameters corresponding to the recognition task grasped in step S802 in step S806. For example, when the recognition task specification unit 108 specifies the recognition tasks 1 and 4, it first performs the setting of the DMAC for the transfer of the operation parameters corresponding to the recognition task 1. Next, in step S807, it instructs the start of the calculation of the task CNN1 corresponding to the recognition task 1. Then, when it detects the completion of the execution of the task CNN1, it executes the post-processing corresponding to the task CNN1 in step S808. The post-processing is a process of picking up the coordinates and reliability of the detection target from the generated CNN calculation result and storing them in the calculation area of ​​the RAM 205 in a predetermined format, etc.

[0044] In step S809, it is determined whether or not all of the recognition tasks specified by the recognition task specifying unit 108 have been completed. If not, the process returns to the transfer setting of the operation parameters (step S806) and controls preparation and execution of the next task to be executed. When execution of all the recognition tasks has been completed, a processing completion notification is issued to the CPU 203. The completion notification is sent by a method such as enabling an interrupt signal of the CPU 203 (not shown).

[0045] Fig. 9 is a diagram for explaining a time chart showing the operation of the multitask recognition processing unit 201 based on the sequence of Fig. 8. Here, a case where recognition tasks 1 and 4 are specified by the recognition task specifying unit 108 will be explained.

[0046] When the CPU 203 issues a command to start processing, the control unit 107 actually transfers motion parameters, grasps the recognition task for executing CNN, and prepares to calculate common features (901). Next, data (motion parameters 0) for calculating common features is transferred and stored in the CNN coefficient buffer 104 or the like (902). Next, image data is transferred and stored in the CNN feature buffer 105 (903). Once storage is complete, the common feature CNN calculation is executed (904).

[0047] Upon completing the calculation of the common feature CNN, the control unit 107 instructs the transfer of motion parameters 1 for recognition task 1 (905), and the CNN for task 1 is executed using the transferred motion parameters 1 (906). Upon completing the execution of the CNN for task 1, the control unit 107 executes post-processing (907). Subsequently, the control unit 107 instructs the transfer of motion parameters 4 for recognition task 4 (908), and executes the CNN corresponding to recognition task 4 (909). Upon completing the execution of the CNN corresponding to recognition task 4, the control unit 107 executes the corresponding post-processing for recognition task 4 (910).

[0048] Through the above processing, common features are extracted and recognition tasks 1 and 4 are executed consecutively. After all processing is completed, a processing completion interrupt is enabled for the CPU 203. Fig. 9 is a time chart for executing recognition tasks 1 and 4, but the recognition task designation unit 108 can instruct the execution of recognition tasks in any combination.

[0049] As described above, according to this embodiment, by simply preparing one type of operation parameter set consisting of a plurality of operation parameters and storing it in advance in the RAM 205, it is possible to realize multitask recognition processing that selects and executes a task to be recognized according to a use case. In this case, since only the necessary operation parameters are transferred to the CNN coefficient buffer 104, it is possible to minimize the transfer bandwidth of the data bus 207. Furthermore, when a series of multitask recognition is performed, the CPU 203 only needs to select a recognition task to be performed and start processing, and even when multitask recognition is selectively performed, it is possible to minimize the intervention of the CPU 203 (the processing cost of the CPU 203).

[0050] <Embodiment 2> In this embodiment, only the differences from embodiment 1 will be described. Fig. 10 is a diagram showing an example of a recognition task designation unit 108 in embodiment 2. In addition to a register set 1001 for selecting four recognition tasks, a register set 1002 for selecting a subcategory or the like within the selected recognition task is added. 1003 is a register that stores information for selecting multiple types of coefficients within the recognition task designated by register 4.

[0051] 11 is a diagram showing an example of an action parameter set of embodiment 2. In addition to the action parameter set of embodiment 1, action parameter 4 is expanded to two types, 4-1 (1106) and 4-2 (1107). These are action parameters for subdividing detection targets within the recognition task of action parameter 4. For example, when action parameter 4 recognizes birds, action parameters are used to select action parameters for each type of bird.

[0052] The control unit 107 executes CNN execution control and post-processing according to the recognition task 4 specified in the register 4 (604). The operation parameters used at that time are transferred according to the information specified in the register 1003 (if 0, the operation parameter 4-1 is selected, if 1, the operation parameter 4-2 is selected). In this case, the control contents processed by the control unit 107 are the same as those of the recognition task 4, but the contents of the coefficients processed by the CNN are different. In other words, the control unit 107 processes tasks of different subcategories with the same processing. 1108 shows the register definition of this embodiment. The register 4' (1003) is used to determine the contents of the recognition task 4.

[0053] In this embodiment, a register set 1002 for specifying subcategories in response to detailed recognition task categories is provided in the recognition task specification unit 108. This makes it possible to realize multitask recognition suited to various use cases without complicating the processing configuration of the control unit 107.

[0054] <Embodiment 3> In this embodiment, only the differences from embodiment 1 will be described. Fig. 12 is a diagram showing examples of two types of operation parameter sets used in embodiment 3.

[0055] Reference numeral 1201 denotes an operation parameter set consisting of the same four operation parameters as in the first embodiment, and reference numeral 1202 denotes an operation parameter set consisting of three operation parameters. Each of these operation parameters is stored in the RAM 205 and is selected and used by the operation parameter set storage address designation unit 109. Reference numerals 1203 and 1204 denote register definitions of the recognition task designation unit 108 when the operation parameters 1201 and 1202, respectively, are used.

[0056] In the third embodiment, one operation parameter set is selected from a plurality of operation parameter sets, and the recognition task is executed according to the definition of the recognition task designation unit 108 corresponding to the selected operation parameter set. That is, the meaning of the register definition of the recognition task designation unit 108 differs depending on the operation parameter set. 1205 and 1206 are attribute information in which the type of operation parameter set is recorded. The control unit 107 grasps the type of operation parameter set according to the attribute information (1205, 1206), and controls the multitask recognition processing unit 201 according to the definition contents (1203, 1204) of the recognition task designation unit 108 corresponding to the type.

[0057] In this way, by changing the register definition of the recognition task designation unit 108 for each operation parameter set, various multi-task recognition processes can be selectively executed using a common recognition task designation unit 108 without adding any hardware, etc.

[0058] <Other embodiments> In the above-described embodiment, an example of recognition processing using CNN has been described, but the present invention is not limited to this and can be applied to various recognition algorithms. For example, the present invention can be applied to recognition algorithms using multi-layer perceptrons and transformers other than CNN. In addition, the algorithm for acquiring common features and the algorithm for processing a recognition task may be different methods. Furthermore, the recognition task may include different algorithms.

[0059] In the above embodiment, the CPU 203 designates the recognition task in the recognition task designation unit 108, but the present invention is not limited to this. A configuration in which the multitask recognition processing unit 201 autonomously selects a recognition task depending on the results of a specific recognition task or multiple recognition tasks may also be used.

[0060] Furthermore, in the above embodiment, the case where CNN is started after image transfer has been described, but CNN calculations may be performed while transferring images by executing CNN in line processing. In this case, the coefficient buffer 104 holds multiple hierarchical combination coefficients.

[0061] In the above embodiment, the convolution calculation according to the present invention is performed by hardware. However, the convolution calculation may be performed by a processor such as a CPU, a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processing Unit).

[0062] The present invention can also be realized by a process in which a program for realizing one or more functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for realizing one or more functions.

[0063] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0064] 101 External bus interface unit 102 DMAC 103 Convolution processing unit 104 CNN coefficient buffer 105 CNN feature buffers 106 Nonlinear conversion processing section 107 Control Unit 108 Recognition task specification part 109 Storage destination address specification section

Claims

1. A storage means for storing a plurality of types of parameter sets including a common parameter set for extracting common feature data commonly used in the plurality of types of recognition tasks from an input image; A selection means for selecting two or more recognition tasks to be executed from the plurality of types of recognition tasks; A storage means for storing a parameter group; a transfer means for sequentially transferring the common parameter set among the parameter sets used in the two or more recognition tasks from the storage means to the holding means prior to the other multiple types of parameter sets; an execution means for extracting the common feature data from the input image using the common parameter group stored in the storage means, and for sequentially executing the two or more recognition tasks using the common feature data.

2. a designation means for designating a storage destination of the plurality of types of parameter groups in the storage means, 2. The data processing apparatus according to claim 1, wherein said transfer means sequentially transfers the parameter groups used for said two or more recognition tasks based on said storage destination.

3. 3. The data processing apparatus according to claim 2, wherein the storage destination in the memory means stores offset information for each of the plurality of types of parameter groups, and the transfer means sequentially transfers the parameter groups used for the two or more recognition tasks based on the storage destination and the offset information.

4. 3. The data processing apparatus according to claim 2, wherein said storage means stores a plurality of sets of said plurality of types of parameter groups, and said designation means switches between said plurality of types of parameter groups depending on which of said plurality of sets is designated as a storage destination.

5. 5. The data processing apparatus according to claim 4, wherein the storage means stores the plurality of sets of the plurality of types of parameter groups together with attribute information for each of the sets, and further comprises a control means for controlling execution of the two or more recognition tasks by the execution means based on the attribute information.

6. 2. The data processing apparatus according to claim 1, wherein the recognition task further includes a plurality of subcategories of recognition tasks, and the selection means further selects a subcategory to be executed from among the plurality of subcategories of the selected recognition task.

7. 7. The data processing apparatus according to claim 6, wherein each of the recognition tasks for the plurality of subcategories is a recognition task having as its recognition target a respective one of a plurality of subcategories that are recognition targets of the recognition task having the plurality of subcategories.

8. 8. The data processing apparatus according to claim 1, wherein the execution means is a hierarchical calculation means, and the set of parameters are coupling coefficients of the hierarchical calculation means.

9. 9. The data processing apparatus according to claim 8, wherein said hierarchical calculation means is a neural network.

10. The selection means selects the two or more recognition tasks one by one; The transfer means transfers a set of parameters used for the selected one recognition task; 2. The data processing apparatus according to claim 1, wherein said execution means executes said selected one recognition task using said transferred parameter group.

11. A data processing device comprising: a storage means for storing a plurality of types of parameter groups including a common parameter group for extracting common feature data commonly used in the plurality of types of recognition tasks from an input image; and a holding means for holding the parameter groups, a selection step of selecting two or more recognition tasks to be executed from the plurality of types of recognition tasks; a transfer step of sequentially transferring the common parameter set from the storage means to the holding means prior to the other multiple types of parameter sets, among the parameter sets used for the two or more recognition tasks; an execution step of extracting common feature data from the input image using the common parameter group stored in the storage means, and sequentially executing the two or more recognition tasks using the common feature data.

Images