Regression processing device, method, and computer program for performing regression processing using machine learning models

Abstract

To provide a technique capable of accurately executing regression processing using a vector neural network.SOLUTION: A regression processing unit is configured to execute: processing (a) of obtaining a predicted output value with respect to input data using a machine learning model; processing (b) of reading out a known feature spectrum group from a memory; processing (c) of calculating a degree of similarity relating to the predicted output value between the known feature spectrum group and a feature spectrum obtained from an output of a specific layer when the input data is input to the machine learning model; and processing (d) of outputting the predicted output value using the degree of similarity.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to a regression processing device, method, and computer program for performing regression processing using a machine learning model. [Background technology]

[0002] Patent Documents 1 and 2 disclose a vector neural network type machine learning model that uses vector neurons, called a capsule network. A vector neuron is a neuron whose input and output are vectors. A capsule network is a machine learning model that uses vector neurons called capsules as network nodes. A vector neural network type machine learning model such as a capsule network can be used for classification processing of input data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 5,210,798 [Patent Document 2] International Publication No. 2019 / 083553 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the application of vector neural networks to regression processing has not been sufficiently studied in the past, and there has been a demand for technology that can perform regression processing with high accuracy using vector neural networks. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a regression processing device that performs a regression process to determine a predicted output value for input data using a machine learning model including a vector neural network with multiple vector neuron layers. The regression processing device includes a regression processing unit that performs the regression process and a memory that stores a group of known feature spectra obtained from the output of a specific layer of the machine learning model when multiple pieces of training data are input to the machine learning model. The regression processing unit is configured to perform the following processes: (a) determine the predicted output value for the input data using the machine learning model, (b) read the group of known feature spectra from the memory, (c) calculate a similarity between the group of known feature spectra and a feature spectrum obtained from the output of the specific layer when the input data is input to the machine learning model, and (d) output the predicted output value using the similarity.

[0006] According to a second aspect of the present disclosure, there is provided a method for performing a regression process to determine a predicted output value for input data using a machine learning model including a vector neural network having multiple vector neuron layers, the method including: (a) determining the predicted output value for the input data using the machine learning model; (b) reading from a memory a group of known feature spectra obtained from an output of a specific layer of the machine learning model when multiple pieces of training data are input to the machine learning model; (c) calculating a similarity between the group of known feature spectra and a feature spectrum obtained from an output of the specific layer when the input data is input to the machine learning model; and (d) outputting the predicted output value using the similarity. The way computers do it .

[0007] According to a third aspect of the present disclosure, there is provided a computer program causing a processor to execute a regression process for determining a predicted output value for input data using a machine learning model including a vector neural network having multiple vector neuron layers. The computer program causes the processor to execute the following processes: (a) determining the predicted output value for the input data using the machine learning model, (b) reading from a memory a group of known feature spectra obtained from an output of a specific layer of the machine learning model when multiple pieces of training data are input to the machine learning model, (c) calculating a similarity between the group of known feature spectra and a feature spectrum obtained from an output of the specific layer when the input data is input to the machine learning model, and (d) outputting the predicted output value using the similarity. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a regression processing system according to an embodiment. [Figure 2] FIG. 1 is an explanatory diagram showing an example of the configuration of a machine learning model. [Figure 3] 10 is a flowchart showing the processing procedure of a preparation step. [Figure 4] FIG. 10 is an explanatory diagram showing how training data is created from sample data. [Figure 5] FIG. [Figure 6] FIG. 2 is an explanatory diagram showing the configuration of a group of known characteristic spectra. [Figure 7] 10 is a flowchart showing the procedure of a regression processing step. [Figure 8] FIG. 10 is an explanatory diagram showing an example of an output of a regression processing result. [Figure 9] FIG. 10 is an explanatory diagram showing another example of output of the regression processing result. [Figure 10] FIG. 10 is an explanatory diagram showing yet another example of output of the regression processing result. [Figure 11] FIG. 10 is an explanatory diagram showing experimental results of regression processing using a trained machine learning model. [Figure 12]FIG. 4 is an explanatory diagram showing a first calculation method of similarity. [Figure 13] FIG. 10 is an explanatory diagram showing a second similarity calculation method. [Figure 14] FIG. 10 is an explanatory diagram showing a third similarity calculation method. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. Implementation: FIG. 1 is a block diagram showing a regression processing system according to an embodiment. The regression processing system includes an information processing device 100 and a camera 400. The camera 400 is used to capture images as input data for the regression processing. The camera 400 may be a camera that captures color images, or a camera that captures monochrome images or spectral images. In this embodiment, images captured by the camera 400 are used as training data or input data, but data other than images may also be used as training data or input data. In this case, an input data acquisition device appropriate for the type of data is used instead of the camera 400.

[0010] The information processing device 100 has a processor 110, a memory 120, an interface circuit 130, and an input device 140 and a display device 150 connected to the interface circuit 130. A camera 400 is also connected to the interface circuit 130. For example, but not limited to, the processor 110 not only has the function of executing the processes described in detail below, but also has the function of displaying on the display device 150 data obtained by the processes and data generated in the course of the processes.

[0011] The processor 110 functions as a learning execution unit 112 that executes learning of a machine learning model, and a regression processing unit 114 that executes regression processing of input data. The regression processing unit 114 includes a similarity calculation unit 310 and an output execution unit 320. The learning execution unit 112 and the regression processing unit 114 are each implemented by the processor 110 executing a computer program stored in the memory 120. However, the learning execution unit 112 and the regression processing unit 114 may also be implemented as hardware circuits. The term "processor" in the present disclosure also includes such hardware circuits. Furthermore, one or more processors that execute the learning processing or regression processing may be processors included in one or more remote computers connected via a network.

[0012] The memory 120 stores a machine learning model 200, a training data group TD, and a known feature spectrum group GKSp. The machine learning model 200 is used for processing by the regression processing unit 114. An example configuration and operation of the machine learning model 200 will be described later. The training data group TD is a collection of labeled data used for training the machine learning model 200. In this embodiment, the training data group TD is a collection of image data. The known feature spectrum group GKSp is a collection of feature spectra obtained when training data is input again into the trained machine learning model 200. The feature spectra will be described later.

[0013] FIG. 2 is an explanatory diagram showing the configuration of a machine learning model 200. This machine learning model 200 has an input layer 210, a hidden layer 280, and an output layer 260. The hidden layer 280 includes a convolutional layer 220, a primary vector neuron layer 230, a first convolutional vector neuron layer 240, and a second convolutional vector neuron layer 250. The output layer 260 is also referred to as a "regression vector neuron layer 260." Of these layers, the input layer 210 is the lowest layer, and the output layer 260 is the highest layer. In the following description, each layer of the hidden layer 280 is referred to as a "Conv layer 220," a "PrimeVN layer 230," a "ConvVN1 layer 240," and a "ConvVN2 layer 250," respectively, and the output layer 260 is referred to as a "RegressVN layer 260."

[0014] 2, two convolution vector neuron layers 240 and 250 are used, but the number of convolution vector neuron layers is arbitrary, and a convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.

[0015] An image of 28×28 pixels is input to the input layer 210. The configuration of each layer other than the input layer 210 can be described as follows. ·Conv layer 220: Conv[32,5,2] ·PrimeVN layer 230: PrimeVN[16,1,1] ·ConvVN1 layer 240:ConvVN1[12,3,2] ·ConvVN2 layer 250:ConvVN2[6,3,1] ·RegressVN layer 260:RegressVN[M,3,1] Vector dimension VD: VD=16 In the description of each layer, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, the kernel surface size, and the stride. For example, the layer name of the Conv layer 220 is "Conv," the number of channels is 32, the kernel surface size is 5x5, and the stride is 2. In Figure 2, these descriptions are shown below each layer. The hatched rectangles drawn in each layer represent the kernel surface size used when calculating the output vector of the adjacent higher layer. In this embodiment, since the input data is image data, the kernel surface size is also two-dimensional. Note that the parameter values ​​used in the description of each layer are merely examples and can be changed as desired.

[0016] The input layer 210 and the Conv layer 220 are layers composed of scalar neurons. The other layers 230 to 260 are layers composed of vector neurons. A vector neuron is a neuron that uses vectors as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.

[0017] FIG. 2 shows the first axis x and second axis y that define the planar coordinates of the node array for the Conv layer 220, and the third axis z that represents depth. It also shows that the sizes of the Conv layer 220 in the x, y, and z directions are 12, 12, and 32. The sizes in the x and y directions are called "resolution." The size in the z direction is the number of channels. These three axes x, y, and z are also used in other layers as coordinate axes that indicate the position of each node. However, in FIG. 2, these axes x, y, and z are omitted from the illustration for layers other than the Conv layer 220.

[0018] As is well known, the resolution W1 after convolution is given by the following equation: W1=Ceil{(W0-Wk+1) / S} (A1) Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function that rounds up the decimal point of X. The resolution of each layer shown in FIG. 2 is an example in which the resolution of the input data is 28, and the actual resolution of each layer is changed appropriately depending on the size of the input data.

[0019] The RegressVN layer 260 has M channels. M is the number of predicted output values ​​output from the machine learning model 200. In this embodiment, M is 1, and one predicted output value θpr is output. The predicted output value θpr is not a discrete value but a continuous value. The number M of predicted output values ​​may be 2 or more. For example, when an image of a three-dimensional object is used as input data, the machine learning model 200 can be configured to obtain three rotation angles around three axes as predicted output values.

[0020] As the activation function of the RegressVN layer 260, a linear function shown in the following equation (A2) can be used.

number

[0021] As the activation function for the RegressVN layer 260, various functions other than the linear function in equation (A2) above can be used, but the softmax function is inappropriate. Any activation function can be used in layers other than the RegressVN layer 260.

[0022] FIG. 2 also illustrates subregions Rn in each layer 220, 230, 240, 250, and 260. The subscript "n" in subregion Rn refers to the reference number of the layer. For example, subregion R220 indicates a subregion in the Conv layer 220. A "subregion Rn" is a region in each layer that is identified by a planar position (x, y) defined by the position of the first axis x and the position of the second axis y, and that includes multiple channels along the third axis z. The subregion Rn has dimensions of "Width" × "Height" × "Depth," corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one "subregion Rn" is "1 × 1 × number of depths," i.e., "1 × 1 × number of channels."

[0023] As shown in Fig. 2, a feature spectrum Sp, which will be described later, is calculated from the output of the ConvVN2 layer 250 and input to the similarity calculation unit 310. The similarity calculation unit 310 calculates a similarity, which will be described later, using this feature spectrum Sp and a group of known feature spectra GKSp that has been created in advance. In this embodiment, this similarity is used to output a predicted output value θpr. The method of outputting the predicted output value θpr will be described further below.

[0024] In this disclosure, the vector neuron layer used to calculate the similarity is also referred to as the "specific layer." As the specific layer, a vector neuron layer other than the ConvVN2 layer 250 may be used, and any number of vector neuron layers greater than or equal to one may be used. The configuration of the feature spectrum and the method of calculating the similarity using the feature spectrum will be described later.

[0025] 3 is a flowchart showing the processing steps for preparing a machine learning model. In step S110, the learning execution unit 112 creates labeled training data.

[0026] FIG. 4 is an explanatory diagram showing how labeled training data is created. Here, multiple handwritten character images of the numbers 0 to 9 are captured by a camera 400 as sample images SD. This sample image SD contains 49 handwritten character images. Each character image is 28 × 28 pixels in size. The training data TD is created by randomly rotating each character image included in the sample image SD within the range of -45°<θ<45°. In this embodiment, 5,000 pieces of such training data TD are prepared. Each character image is labeled with the value of the rotation angle θ. More specifically, the rotation angle θ is normalized by dividing it by 180 and then added with 5.0 to create a learning label. In this case, a rotation angle θ between -45° and +45° is converted to a label between 4.75 and 5.25. If the machine learning model 200 is trained using such labeled training data, it is possible to obtain a predicted output value θpr for any input data, even for angles outside the range of -45° to +45°.

[0027] In step S120, the learning execution unit 112 uses the training data group TD to learn the machine learning model 200. Any loss function can be used during learning, but in this embodiment, MSE (Mean Square Error) is used. When learning is completed, the learned machine learning model 200 is saved in the memory 120.

[0028] In step S130, the learning execution unit 112 generates a group of known characteristic spectra GKSp by again inputting a plurality of pieces of training data into the trained machine learning model 200. The group of known characteristic spectra GKSp is a collection of characteristic spectra, which will be described below.

[0029] FIG. 5 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the trained machine learning model 200. As shown in FIG. 2, in this embodiment, the feature spectrum Sp is created from the output of the ConvVN2 layer 250. The horizontal axis in FIG. 5 represents the position of a vector element in the output vector of multiple nodes included in one subregion R250 of the ConvVN2 layer 250. The position of this vector element is represented by a combination of the element number ND of the output vector at each node and the channel number NC. In this embodiment, since the vector dimension is 16 (the number of elements of the output vector output by each node), the element number ND of the output vector is 16, ranging from 0 to 15. Furthermore, since the ConvVN2 layer 250 has six channels, the channel number NC is 6, ranging from 0 to 5. In other words, this feature spectrum Sp is obtained by arranging multiple element values ​​of the output vector of each vector neuron included in one subregion R250 across multiple channels along the third axis z.

[0030] The vertical axis of Fig. 5 represents the feature value C at each spectral position. V In this example, the feature value C V is the value of each element of the output vector V ND Feature value C V For the feature value C, statistical processing such as centering to the average value 0 may be performed. V As the value of each element of the output vector V ND Alternatively, the normalization coefficient may be used as is. In the latter case, the feature value C included in the feature spectrum Sp may be used as is. V The number of is equal to the number of channels, which is 6. The normalization coefficient is a value corresponding to the vector length of the output vector of the node.

[0031] The number of feature spectra Sp obtained from the output of the ConvVN2 layer 250 for one piece of input data is nine, since it is equal to the number of planar positions (x, y) of the ConvVN2 layer 250, that is, the number of subregions R250.

[0032] The learning execution unit 112 inputs the training data again into the trained machine learning model 200, calculates the feature spectrum Sp shown in FIG. 5, and registers it in the memory 120 as a group of known feature spectra GKSp.

[0033] 6 is an explanatory diagram showing the configuration of the known feature spectrum group GKSp. In this example, the known feature spectrum group GKSp obtained from the output of the ConvVN2 layer 250 is shown. Note that it is sufficient that the known feature spectrum group GKSp is obtained from the output of at least one vector neuron layer, and it is also possible to register known feature spectrum groups obtained from the output of the ConvVN1 layer 240 or the RegressVN layer 260.

[0034] Each record in the group of known feature spectra GKSp includes a parameter k indicating the order of the subregion Rn within the layer, a parameter q indicating the data number, and a known feature spectrum KSp. The known feature spectrum KSp is the same as the feature spectrum Sp in Figure 5.

[0035] The parameter k of the subregion Rn takes a value indicating which of the multiple subregions Rn included in a specific layer it is, i.e., which planar position (x, y) it is. For the ConvVN2 layer 250, there are nine subregions R250, so k = 1 to 9. The parameter q of the data number indicates the consecutive number of the training data and takes a value from 1 to max. For example, max = 5000.

[0036] The plurality of training data used in step S130 does not need to be the same as the plurality of training data used in step S120. However, if some or all of the plurality of training data used in step S120 is used in step S130, there is an advantage in that there is no need to prepare new training data.

[0037] FIG. 7 is a flowchart showing the processing steps of a regression processing step using a trained machine learning model 200. In step S210, the regression processing unit 114 generates input data. In this embodiment, a character image of 28 × 28 pixels is created as input data by photographing handwritten characters using the camera 400. In step S220, the regression processing unit 114 performs preprocessing on the input data as necessary. Preprocessing may include resolution adjustment, data normalization (min-max normalization), and the like. Preprocessing is optional. In step S230, the regression processing unit 114 reads the trained machine learning model 200 and the known feature spectrum group GKSp from the memory 120.

[0038] In step S240, the regression processing unit 114 inputs input data to the machine learning model 200 and calculates a predicted output value θpr. In this embodiment, the predicted output value θpr is the rotation angle of a handwritten character included in the input data. In step S250, the regression processing unit 114 calculates the feature spectrum Sp shown in FIG. 5 using the output of the ConvVN2 layer 250, which is a specific layer. In step S260, the similarity calculation unit 310 calculates the similarity using the feature spectrum Sp obtained in step S250 and the known feature spectrum group GKSp shown in FIG. 6. The similarity is an index that indicates the degree to which the input data resembles the features of the training data. A method for calculating the similarity will be described later.

[0039] In step S270, the output execution unit 320 executes output of the predicted output value θpr using the similarity.

[0040] FIG. 8 is an explanatory diagram showing an example of the output of the regression processing result. The regression processing result display window WD1 displays an image of the input data GF, the predicted output value θpr, and the similarity Sm. In this example, the input data GF is an image of a rotated handwritten number "3." The predicted output value θpr is "23°," and the similarity Sm is "0.96." This predicted output value θpr can be obtained by subtracting 5.0 from the output of the Regress VN layer 260 and multiplying the result by 180. The user can determine whether the predicted output value θpr is reliable from the value of the similarity Sm. The possible range of the similarity Sm is -1 to +1. In the example of FIG. 8, the similarity Sm is close to 1, so it can be determined that the predicted output value θpr is reliable.

[0041] FIG. 9 is an explanatory diagram showing another example of output of the regression processing results. The regression processing result display window WD2 also displays an image of the input data GF, the predicted output value θpr, and the similarity Sm. The only difference from FIG. 8 is that the similarity Sm is named "reliability." In this example, too, the user can determine whether the predicted output value θpr is reliable from the reliability value.

[0042] FIG. 10 is an explanatory diagram showing yet another example output of the regression processing results. This example shows a display example in the same result display window WD2 as in FIG. 9 when the reliability value is low. Here, the value of the similarity Sm as reliability is quite low at 0.55, so the display format of the predicted output value θpr differs from that in FIG. 9 in that it indicates low reliability. Specifically, the value of the predicted output value θpr is shaded, making it difficult to see. For example, the output execution unit 320 may determine the predicted output value θpr to be valid and output as shown in FIG. 9 when the similarity Sm is equal to or greater than a predetermined threshold, and may determine the predicted output value θpr to be invalid and output as shown in FIG. 10 when the similarity Sm is less than the threshold. Alternatively, instead of displaying the predicted output value θpr in different display formats when the similarity Sm is less than the threshold and when the similarity Sm is equal to or greater than the threshold, the output of the predicted output value θpr may be stopped when the similarity Sm is less than the threshold. In either case, if the similarity Sm is less than the threshold value, the reliability of the predicted output value θpr is low, and therefore the predicted output value θpr obtained by the machine learning model 200 can be determined to be invalid.

[0043] FIG. 11 is an explanatory diagram showing experimental results of a regression process using the trained machine learning model 200. Here, the results are shown in which a large number of handwritten character images are used as input data, and the rotation angles of the input data are calculated as predicted output values ​​θpr using the machine learning model 200. The horizontal axis represents the true rotation angle θ, and the vertical axis represents the predicted output value θpr. Open circles represent results where the similarity Sm is equal to or greater than the threshold Th, and filled circles represent results where the similarity Sm is less than the threshold Th. In this example, the threshold Th is set to 0.95. As shown in FIG. 3, the training data had a rotation angle θ ranging from −45° to +45°. However, the results shown in FIG. 11 show that good predicted output values ​​θpr are obtained in the range from −50° to +50°. Furthermore, outside the range from −50° to +50°, the similarity Sm of the predicted output values ​​θpr tends to decrease significantly. As described with reference to the examples of FIGS. 8 to 10, using the similarity Sm to output the predicted output value θpr has the advantage of providing a high similarity Sm and a reliable predicted output value θpr.

[0044] As described above, in the above embodiment, the regression process can be performed with high accuracy using the machine learning model 200 including the vector neural network. Furthermore, since the predicted output value θpr is output using the similarity Sm, it is possible to obtain a reliable predicted output value θpr.

[0045] B. Similarity calculation method: As a method for calculating the similarity, for example, any of the following methods can be adopted. (1) A first calculation method M1 for calculating the similarity between the feature spectrum Sp and the subregion Rn in the known feature spectrum group GKSp without considering the correspondence between the feature spectrum Sp and the subregion Rn in the known feature spectrum group GKSp (2) A second calculation method M2 for calculating the similarity between the feature spectrum Sp and the corresponding subregion Rn of the known feature spectrum group GKSp. (3) A third calculation method M3 for calculating the similarity without considering the partial region Rn at all Below, methods for calculating similarity from the output of the ConvVN2 layer 250 according to these calculation methods M1, M2, and M3 will be described in order.

[0046] Fig. 12 is an explanatory diagram showing a first similarity calculation method M1. In the first calculation method M1, first, a local similarity S(k) indicating the similarity for each subregion k is calculated from the output of the ConvVN2 layer 250, which is a specific layer, according to a formula described below. In the machine learning model 200 of Fig. 2, the number of subregions R250 in the ConvVN2 layer 250 is 9, so the subregion parameter k takes a value from 1 to 9. From these local similarities S(k), one of three types of similarities Sm shown on the right side of Fig. 12 is calculated.

[0047] In the first calculation method M1, the local similarity S(k) is calculated using the following formula: S(k)=max[G{Sp(k), KSp(k=all,q=all)}] (B1) where: k is a parameter indicating the subregion Rn, q is a parameter indicating the data number, G{a,b} is a function that calculates the similarity between a and b. Sp(k) is the feature spectrum obtained from the output of a specific subregion k of a specific layer according to the input data. KSp (k=all, q=all) is the known feature spectrum of all data numbers q in all partial regions k of a specific layer among the known feature spectrum group GKSp shown in FIG. 6. max[X] is a logical operation that takes the maximum value of X. As the function G{a, b} for calculating the similarity, for example, an equation for calculating cosine similarity or an equation for calculating similarity according to distance can be used.

[0048] The three types of similarity Sm shown on the right side of FIG. 12 are obtained by taking the maximum, average, or minimum value of the local similarity S(k) for multiple partial regions k. Which of the maximum, average, or minimum value calculations to use depends on the purpose of using the similarity Sm. Which of these three types of calculations to use is set in advance by the user experimentally or empirically. In the above-described embodiment, the similarity Sm was determined by taking the minimum value of the local similarity S(k).

[0049] As described above, in the first similarity calculation method M1, (1) According to the input data, a local similarity S(k) is calculated, which is the similarity between a feature spectrum Sp obtained from the output of a specific subregion k of a specific layer and all known feature spectra Ksp associated with that specific layer; (2) The similarity Sm is calculated by taking the maximum, average, or minimum value of the local similarities S(k) for a plurality of partial regions k. According to this first calculation method M1, the similarity Sm can be calculated using relatively simple calculations and procedures.

[0050] 13 is an explanatory diagram showing a second similarity calculation method M2. In the second calculation method M2, the local similarity S(k) is calculated using the following equation instead of the above-mentioned equation (B1). S(k)=max[G{Sp(k), KSp(k,q=all)}] (B2) where: KSp(k, q=all) is the known feature spectrum of all data numbers q in a specific partial region k of a specific layer, among the known feature spectrum group GKSp shown in FIG.

[0051] While the first calculation method M1 described above uses known feature spectra KSp(k=all, q=all) in all partial regions k of a specific layer, the second calculation method M2 uses only known feature spectra KSp(k, q=all) for the partial region k that is the same as the partial region k of the feature spectrum Sp(k). The other aspects of the second calculation method M2 are the same as those of the first calculation method M1.

[0052] In the second similarity calculation method M2, (1) According to the input data, a local similarity S(k) is calculated, which is the similarity between a feature spectrum Sp obtained from the output of a specific subregion k of a specific layer and all known feature spectra Ksp associated with the specific subregion k of the specific layer; (2) The similarity Sm is calculated by taking the maximum, average, or minimum value of the local similarities S(k) for a plurality of partial regions k. This second calculation method M2 also makes it possible to find the similarity Sm through relatively simple calculations and procedures.

[0053] 14 is an explanatory diagram showing a third similarity calculation method M3. In the third calculation method M3, the similarity Sm is calculated from the output of the ConvVN2 layer 250, which is a specific layer, without calculating the local similarity S(k).

[0054] The similarity Sm obtained by the third calculation method M3 is calculated using the following formula. Sm=max[G{Sp(k=all), KSp(k=all,q=all)}] (B3) where: Sp(k=all) is the feature spectrum obtained from the output of all subregions k of a specific layer according to the input data.

[0055] As described above, in the third similarity calculation method M3, (1) The similarity Sm between all feature spectra Sp obtained from the output of a specific layer according to the input data and all known feature spectra KSp associated with that specific layer is calculated. According to the third calculation method M3, the similarity Sm can be calculated using even simpler calculations and procedures.

[0056] The above-mentioned three calculation methods M1 to M3 are all methods for calculating similarity using the output of one specific layer. However, the similarity can be calculated using one or more of the vector neuron layers 240, 250, and 260 shown in Fig. 2 as specific layers. When multiple specific layers are used, it is preferable to use, for example, the minimum or average value of the multiple similarities obtained from the multiple specific layers as the final similarity.

[0057] C. How to calculate the output vector of each layer of the machine learning model: The calculation method for the output of each layer shown in Figure 2 is as follows.

[0058] Each node in the PrimeVN layer 230 regards the scalar output of the 1x1x32 nodes in the Conv layer 220 as a 32-dimensional vector and obtains the vector output of that node by multiplying this vector by a transformation matrix. This transformation matrix is ​​an element of a kernel with a surface size of 1x1, and is updated by learning the machine learning model 200. Note that the processing of the Conv layer 220 and the PrimeVN layer 230 can also be integrated into one primary vector neuron layer.

[0059] When the PrimeVN layer 230 is referred to as the "lower layer L" and the ConvVN1 layer 240 adjacent to it on the upper side is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following equation.

number

[0060] As the normalization function F(X), for example, the following formula (E3a) or (E3b) can be used.

number

[0061] In the above equation (E3a), the sum vector u j Norm of |u j The activation value a is normalized by the softmax function | j On the other hand, in equation (E3b), the sum vector u j Norm of |u j | is the norm |u j Activation value a by dividing by the sum of | j It should be noted that a function other than equation (E3a) or (E3b) may be used as the normalization function F(X).

[0062] The ordinal number i in the above equation (E2) is the output vector M of the jth node in the upper layer L+1. L+1 j The integer n is assigned for convenience to the nodes in the lower layer L used to determine the output vector M L+1 j is the number of nodes in the lower layer L used to determine . Thus, the integer n is given by n = Nk × Nc (E5) Here, Nk is the surface size of the kernel, and Nc is the number of channels in the lower layer, the PrimeVN layer 230. In the example of FIG. 2, Nk=9 and Nc=16, so n=144.

[0063] One kernel used to calculate the output vector of the ConvVN1 layer 240 has a surface size of 3 × 3 and a depth of 16 channels in the lower layer, with 3 × 3 × 16 = 144 elements. Each of these elements is a prediction matrix W L ij In addition, 12 sets of this kernel are required to generate output vectors for 12 channels of the ConvVN1 layer 240. Therefore, the prediction matrix W of the kernel used to obtain the output vector of the ConvVN1 layer 240 is L ijThe number of prediction matrices W is 144 × 12 = 1728. L ij is updated by learning of the machine learning model 200.

[0064] As can be seen from the above equations (E1) to (E4), the output vector M of each node in the upper layer L+1 L+1 j is calculated by the following calculation: (a) Output vector M of each node in the lower layer L L i The prediction matrix W L ij Multiplying by the predicted vector v ij Seeking (b) Prediction vector v obtained from each node in the lower layer L ij The sum vector u is a linear combination of j Seeking (c) Sum vector u j Norm of |u j The activation value a is normalized by normalizing | j Seeking (d) Sum vector u j norm |u j Divide by | and then use the activation value a j Multiply by.

[0065] In addition, the activation value a j is the norm |u j is the normalization factor obtained by normalizing |. Therefore, the activation value a j can be considered as an index showing the relative output strength of each node among all nodes in the upper layer L+1. The norm used in equations (E3) and (4) is typically the L2 norm, which represents the vector length. In this case, the activation value a j is the output vector M L+1 j The activation value a corresponds to the vector length of jis only used in the above equations (E3) and (E4), and does not need to be output from the node. However, the activation value a j It is also possible to configure the upper layer L+1 so that it outputs

[0066] The configuration of a vector neural network is almost the same as that of a capsule network, and the vector neurons of a vector neural network correspond to the capsules of a capsule network. However, the calculations according to the above formulas (E1) to (E4) used in a vector neural network are different from the calculations used in a capsule network. The biggest difference between the two is that in a capsule network, the predicted vector v on the right side of the above formula (E2) ij are multiplied by weights, and the weights are searched by repeating dynamic routing multiple times. On the other hand, in the vector neural network of this embodiment, the output vector M is calculated by calculating the above-mentioned equations (E1) to (E4) once in order. L+1 j Therefore, there is no need to repeat dynamic routing, which has the advantage of allowing faster calculations. In addition, the vector neural network of this embodiment has the advantage that it requires less memory for calculations than a capsule network, and according to experiments by the inventors of this disclosure, it only requires about 1 / 2 to 1 / 3 of the memory required.

[0067] Vector neural networks are similar to capsule networks in that they use nodes that use vectors as input and output. Therefore, they share the advantages of using vector neurons with capsule networks. Furthermore, the multiple layers 220-260 are similar to conventional convolutional neural networks in that the higher layers represent features of larger areas and the lower layers represent features of smaller areas. Here, "feature" refers to a characteristic part contained in the input data to the neural network. Vector neural networks and capsule networks are superior to conventional convolutional neural networks in that the output vector of a node contains spatial information representing the spatial information of the feature represented by that node. That is, the vector length of a node's output vector represents the probability of the feature represented by that node, and the vector direction represents spatial information such as the direction and scale of the feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the relative positions of the respective features. Alternatively, the vector direction of the output vectors of the two nodes can be said to represent the variation of the feature. For example, for a node corresponding to the "eye" feature, the direction of the output vector can represent variations such as the narrowness of the eyes or the way they are lifted. In conventional convolutional neural networks, it is said that spatial information of features is lost due to the pooling process. As a result, vector neural networks and capsule networks have the advantage of being superior to conventional convolutional neural networks in terms of the performance of identifying input data.

[0068] The advantages of vector neural networks can also be considered as follows. In other words, the advantage of vector neural networks is that the output vectors of nodes represent the features of input data as coordinates in continuous space. Therefore, output vectors can be evaluated such that the closer the vector directions, the more similar the features. Another advantage is that even if the features contained in the input data are not covered by the training data, they can be determined by interpolation. On the other hand, conventional convolutional neural networks have the disadvantage that the features of input data cannot be represented as coordinates in continuous space due to the chaotic compression caused by the pooling process.

[0069] The outputs of each node in the ConvVN2 layer 250 and the RegressVN layer 260 are similarly determined using the above-mentioned equations (E1) to (E4), and therefore detailed explanations are omitted. The resolution of the topmost layer, the RegressVN layer 260, is 1x1, and the number of channels is M.

[0070] In the RegressVN layer 260, instead of the above equation (E3), a linear function such as the above equation (A2) can be used as the activation function. That is, the output vector of the RegressVN layer 260 is converted into a predicted output value θpre by the above linear function of the above equation (A2). Alternatively, the above-mentioned sigmoid function can be used as the activation function.

[0071] In the above-described embodiment, a vector neural network that determines an output vector by calculating the above equations (E1) to (E4) was used as the machine learning model 200, but instead, a capsule network disclosed in U.S. Pat. No. 5,210,798 or WO 2009 / 083553 may be used.

[0072] Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following aspects. The technical features in the above embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0073] (1) According to a first aspect of the present disclosure, there is provided a regression processing device that performs a regression process to determine a predicted output value for input data using a machine learning model including a vector neural network with multiple vector neuron layers. The regression processing device includes a regression processing unit that performs the regression process and a memory that stores a group of known feature spectra obtained from the output of a specific layer of the machine learning model when multiple pieces of training data are input to the machine learning model. The regression processing unit is configured to perform the following processes: (a) determine the predicted output value for the input data using the machine learning model, (b) read the group of known feature spectra from the memory, (c) calculate a similarity between the group of known feature spectra and a feature spectrum obtained from the output of the specific layer when the input data is input to the machine learning model, and (d) output the predicted output value using the similarity. This device can perform regression processing with high accuracy using a machine learning model including a vector neural network. In addition, since it uses similarity to output predicted output values, it can obtain highly reliable predicted output values ​​with high similarity.

[0074] (2) In the regression processing device, the process (d) may include a process of outputting the similarity together with the predicted output value. This device allows the user to determine whether the predicted output value is reliable or not based on the similarity.

[0075] (3) In the regression processing device, the process (d) may include a process of outputting, together with the predicted output value, a reliability of the predicted output value according to the degree of similarity. This device allows the user to easily understand the reliability of the predicted output value.

[0076] (4) In the regression processing device, the process (d) may include a process of determining that the predicted output value is valid when the similarity is equal to or greater than a predetermined threshold, and determining that the predicted output value is invalid when the similarity is less than the threshold. According to this device, if the similarity is less than a threshold, the reliability of the predicted output value is low, and therefore the predicted output value obtained by the machine learning model can be determined to be invalid.

[0077] (5) In the regression processing device, the specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis that is oriented in a direction different from the first and second axes. The feature spectrum may be any one of (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of a vector neuron at one planar position in the specific layer are arranged across the plurality of channels along the third axis, (ii) a second type of feature spectrum obtained by multiplying each element value of the first type of feature spectrum by an activation value corresponding to the vector length of the output vector, and (iii) a third type of feature spectrum in which the activation values ​​at one planar position in the specific layer are arranged across the plurality of channels along the third axis. This device makes it possible to easily obtain characteristic spectra.

[0078] (6) According to a second aspect of the present disclosure, there is provided a method for performing a regression process to determine a predicted output value for input data using a machine learning model including a vector neural network having multiple vector neuron layers, the method including: (a) determining the predicted output value for the input data using the machine learning model; (b) reading from a memory a set of known feature spectra obtained from an output of a specific layer of the machine learning model when multiple pieces of training data are input to the machine learning model; (c) calculating a similarity between the set of known feature spectra and a feature spectrum obtained from an output of the specific layer when the input data is input to the machine learning model; and (d) outputting the predicted output value using the similarity.

[0079] (7) According to a third aspect of the present disclosure, there is provided a computer program that causes a processor to execute a regression process for determining a predicted output value for input data using a machine learning model including a vector neural network having multiple vector neuron layers. The computer program causes the processor to execute the following processes: (a) determining the predicted output value for the input data using the machine learning model, (b) reading from a memory a group of known feature spectra obtained from an output of a specific layer of the machine learning model when multiple pieces of training data are input to the machine learning model, (c) calculating a similarity between the group of known feature spectra and a feature spectrum obtained from an output of the specific layer when the input data is input to the machine learning model, and (d) outputting the predicted output value using the similarity.

[0080] The present disclosure can be realized in various forms other than those described above, such as a computer program for realizing the functions of a regression processing device, a non-transitory storage medium on which the computer program is recorded, and the like. [Explanation of symbols]

[0081] 100...information processing device, 110...processor, 112...learning execution unit, 114...regression processing unit, 120...memory, 130...interface circuit, 140...input device, 150...display device, 200...machine learning model, 210...input layer, 220...convolution layer, 230...primary vector neuron layer, 240...first convolution vector neuron layer, 250...second convolution vector neuron layer, 260...regression vector neuron layer, 280...hidden layer, 310...similarity calculation unit, 320...output execution unit, 400...camera

Claims

1. A regression processing device that executes regression processing to obtain a predicted output value for input data using a machine learning model including a vector neural network having a plurality of vector neuron layers, a regression processing unit that executes the regression processing; a memory that stores a group of known feature spectra obtained from an output of a specific layer of the machine learning model when a plurality of pieces of training data are input to the machine learning model; Equipped with The regression processing unit (a) a process of calculating the predicted output value for the input data using the machine learning model; (b) reading the group of known characteristic spectra from the memory; (c) calculating a similarity between the feature spectrum obtained from the output of the specific layer when the input data is input to the machine learning model and the group of known feature spectra, regarding the predicted output value; (d) performing an output of the predicted output value using the similarity; a regression processing unit configured to execute

2. 2. The regression processing device according to claim 1, The regression processing device, wherein the process (d) includes a process of outputting the similarity together with the predicted output value.

3. 2. The regression processing device according to claim 1, The regression processing device, wherein the process (d) includes a process of outputting the predicted output value together with a reliability of the predicted output value according to the degree of similarity.

4. 2. The regression processing device according to claim 1, The process (d) includes determining that the predicted output value is valid when the similarity is equal to or greater than a predetermined threshold, and determining that the predicted output value is invalid when the similarity is less than the threshold.

5. The regression processing device according to any one of claims 1 to 4, The specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, The characteristic spectrum is (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of a vector neuron at one plane position in the specific layer are arranged across the plurality of channels along the third axis; (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by an activation value corresponding to the vector length of the output vector; (iii) a third type of feature spectrum in which the activation values ​​at one planar position of the specific layer are arranged across the plurality of channels along the third axis; a regression processing device,

6. A method for performing a regression process to obtain a predicted output value for input data using a machine learning model including a vector neural network having a plurality of vector neuron layers, comprising: (a) determining the predicted output value for the input data using the machine learning model; (b) reading from a memory a group of known characteristic spectra obtained from the output of a specific layer of the machine learning model when a plurality of pieces of training data are input to the machine learning model; (c) calculating a similarity between the feature spectrum obtained from the output of the specific layer when the input data is input to the machine learning model and the group of known feature spectra, regarding the predicted output value; (d) using the similarity to perform output of the predicted output value; A computer-implemented method, including:

7. A computer program that causes a processor to execute a regression process to obtain a predicted output value for input data using a machine learning model including a vector neural network having a plurality of vector neuron layers, The computer program comprises: (a) a process of calculating the predicted output value for the input data using the machine learning model; (b) reading from a memory a group of known characteristic spectra obtained from the output of a specific layer of the machine learning model when a plurality of pieces of training data are input to the machine learning model; (c) calculating a similarity between the feature spectrum obtained from the output of the specific layer when the input data is input to the machine learning model and the group of known feature spectra, regarding the predicted output value; (d) performing an output of the predicted output value using the similarity; A computer program that causes the processor to execute the following.

Images