Method and system for performing print media classification process using machine learning model
The method and system use a machine learning model to classify print media by spectral reflectance, addressing the differentiation challenge of similar optical characteristics and optimizing print settings for enhanced accuracy and quality.
Patent Information
- Application Number
- JP2020213538
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-23
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-12-23
AI Technical Summary
Existing print media discrimination techniques using optical media detection sensors struggle to differentiate print media with similar optical characteristics.
A method and system utilizing a machine learning model, comprising multiple machine learning models, to classify print media by spectral reflectance, including steps for training, classification, medium addition, and exclusion, with clustering and relearning processes to enhance accuracy.
Accurately discriminates print media with similar optical characteristics and optimizes print settings for improved print quality by leveraging machine learning models with enhanced classification accuracy and efficiency.
Smart Images

Figure 0007589540000003 
Figure 0007589540000004 
Figure 0007589540000005
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a method and system for performing print media classification using machine learning models. [Background technology]
[0002] Patent Literature 1 discloses a technique for detecting a print medium using a media detection sensor and selecting print settings for the media that are linked to attribute information that can be acquired by the media detection sensor. The media detection sensor is configured with an optical sensor. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2019-55554 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, the above-mentioned technology has a problem in that it is not possible to distinguish between print media having similar optical characteristics because an optical media detection sensor is used. [Means for solving the problem]
[0005] According to a first aspect of the present disclosure, there is provided a method for performing a discrimination process for a print medium using a machine learning model, the method including: (a) preparing N machine learning models, where N is an integer equal to or greater than 1, each of the N machine learning models configured to discriminate the type of the print medium by classifying input spectral data, which is the spectral reflectance of the print medium, into one of a plurality of classes; (b) acquiring target spectral data, which is the spectral reflectance of a target print medium; and (c) discriminating the type of the target print medium by performing a classifying process for the target spectral data using the N machine learning models. In a first configuration of the first aspect, N is an integer greater than or equal to 2, and each of the N machine learning models is configured to have at least one class that is different from other machine learning models among the N machine learning models. In a second configuration of the first embodiment, each teacher data group has a representative point that represents the center of a spectral data group that constitutes the teacher data group, an upper limit is set for the number of classes that can be classified by any one machine learning model, and the multiple types of print media that are the subject of the classification process by the N machine learning models are classified into either required print media that cannot be excluded from the subject of the classification process without a user's exclusion instruction, or optional print media that can be excluded from the subject of the classification process even without a user's exclusion instruction. The step (a) includes a medium adding step of making a new additional print medium that has not been the subject of the classification process by the N machine learning models the subject of the classification process. The medium adding process includes the steps of: (a1) acquiring the spectral reflectance of the additional printing medium as additional spectral data; (a2) selecting a teacher data group from the N teacher data groups that has a representative point closest to the additional spectral data as a proximal teacher data group, and selecting a specific machine learning model that has been trained using the proximal teacher data group; and (a3) if the number of classes corresponding to the required printing medium in the specific machine learning model is less than the upper limit value, adding the additional spectral data to the proximal teacher data group to update the proximal teacher data group, and re-training the specific machine learning model using the updated proximal teacher data group. In a third configuration of the first aspect, N is an integer equal to or greater than 2, and the method further includes a medium excluding step of excluding one excluded print medium from the targets of the class classification process by one target machine learning model selected from the N machine learning models. The medium excluding step includes a step of (i) updating the teacher data group by deleting spectral data related to the excluded print medium from a teacher data group used in training the target machine learning model, and (ii) executing re-training of the target machine learning model using the updated teacher data group.
[0006] According to a second aspect of the present disclosure, there is provided a system for executing a discrimination process for a print medium using a machine learning model. The system includes a memory for storing N machine learning models, where N is an integer equal to or greater than 1, and a processor for executing the discrimination process using the N machine learning models. Each of the N machine learning models is configured to discriminate the type of the print medium by classifying input spectral data, which is the spectral reflectance of the print medium, into one of a plurality of classes. The processor is configured to execute a first process for acquiring target spectral data of a target print medium, and a second process for discriminating the type of the target print medium by executing a classifying process for the target spectral data using the N machine learning models. In a first configuration of the second aspect, N is an integer greater than or equal to 2, and each of the N machine learning models is configured to have at least one class that is different from other machine learning models among the N machine learning models. In a second configuration of the second embodiment, each teacher data group has a representative point that represents the center of a spectral data group that constitutes the teacher data group, an upper limit is set for the number of classes that can be classified by any one machine learning model, and the multiple types of print media that are the subject of the classification process by the N machine learning models are classified into either required print media that cannot be excluded from the subject of the classification process without a user's exclusion instruction, or optional print media that can be excluded from the subject of the classification process even without a user's exclusion instruction. The processor is further configured to execute a medium addition process that makes a new additional print medium that has not been the subject of the classification process by the N machine learning models the subject of the classification process. The medium addition process includes: (a1) a process of acquiring the spectral reflectance of the additional printing medium as additional spectral data; (a2) a process of selecting, from the N teacher data groups, a teacher data group having a representative point closest to the additional spectral data as a proximal teacher data group, and a process of selecting a specific machine learning model that has been trained using the proximal teacher data group; and (a3) a process of updating the proximal teacher data group by adding the additional spectral data to the proximal teacher data group if the number of classes corresponding to the required printing medium in the specific machine learning model is less than the upper limit value, and re-training the specific machine learning model using the updated proximal teacher data group. In a third configuration of the second aspect, N is an integer equal to or greater than 2, and the processor is further configured to execute a medium exclusion process to exclude one excluded print medium from the targets of the class classification process by one target machine learning model selected from the N machine learning models. The medium exclusion process includes: (i) updating the teacher data group by deleting spectral data related to the excluded print medium from a teacher data group used for training the target machine learning model; and (ii) re-training the target machine learning model using the updated teacher data group. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a block diagram of a printing system according to an embodiment. [Diagram 2] FIG. [Diagram 3] An explanatory diagram showing the configuration of a first machine learning model. [Figure 4] An explanatory diagram showing the configuration of a second machine learning model. [Diagram 5] 11 is a flowchart showing a processing procedure of a preparation process. [Figure 6] FIG. 13 is an explanatory diagram showing a medium identifier list. [Figure 7] FIG. 4 is an explanatory diagram showing a print setting table. [Figure 8] FIG. 11 is an explanatory diagram showing spectroscopic data that has been subjected to clustering processing. [Figure 9] FIG. 4 is an explanatory diagram showing a group management table. [Figure 10] FIG. [Figure 11] FIG. 4 is an explanatory diagram showing the configuration of a group of known characteristic spectra. [Figure 12] 11 is a flowchart showing a processing procedure of a medium discrimination / printing process. [Figure 13] 11 is a flowchart showing the procedure of a medium addition process. [Figure 14] FIG. 4 is an explanatory diagram showing a management state of a group of spectral data. [Figure 15] FIG. 11 is an explanatory diagram showing a medium identifier list updated in response to the addition of a print medium. [Figure 16] FIG. 11 is an explanatory diagram showing a group management table updated in response to the addition of a print medium. [Figure 17] FIG. 13 is an explanatory diagram showing a group management table updated in response to the addition of a machine learning model. [Figure 18] 13 is a flowchart showing the processing procedure of a medium removal step. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] A. Embodiment: FIG. 1 is a block diagram showing a printing system according to an embodiment. The printing system includes a printer 10, an information processing device 20, and a spectrometer 30. The spectrometer 30 can perform spectroscopic measurement on the printing medium PM used in the printer 10 in an unprinted state to obtain the spectral reflectance. In this disclosure, the spectral reflectance is also referred to as "spectroscopic data." The spectrometer 30 includes, for example, a variable wavelength interference spectrometer filter and a monochrome image sensor. The spectral data obtained by the spectrometer 30 is used as input data for a machine learning model to be described later. As will be described later, the information processing device 20 executes a class classification process of the spectral data using the machine learning model to classify the printing medium PM into which of a plurality of classes it belongs. The "class of the printing medium PM" refers to the type of the printing medium PM. The information processing device 20 controls the printer 10 to perform printing under appropriate printing conditions according to the type of the printing medium PM.
[0009] 2 is a block diagram showing the functions of the information processing device 20. The information processing device 20 has a processor 110, a memory 120, an interface circuit 130, and an input device 140 and a display unit 150 connected to the interface circuit 130. The spectrometer 30 and the printer 10 are also connected to the interface circuit 130. For example, but not limited to, the processor 110 not only has a function of executing the processes described in detail below, but also has a function of displaying on the display unit 150 data obtained by the processes and data generated in the course of the processes.
[0010] The processor 110 functions as a print processing unit 112 that executes print processing using the printer 10, as a class classification processing unit 114 that executes class classification processing of the spectral data of the print medium PM, and as a print setting creation unit 116 that creates print settings suitable for the print medium PM. Each of these units 112, 114, and 116 is realized by the processor 110 executing a computer program stored in the memory 120. However, each of these units 112, 114, and 116 may be realized by a hardware circuit. The term processor in this specification includes such a hardware circuit. Also, the processor that executes the class classification processing may be a processor included in a remote computer connected to the information processing device 20 via a network.
[0011] The memory 120 stores a plurality of machine learning models 201, 202, a plurality of spectral data groups SD1, SD2, a medium identifier list IDL, a plurality of group management tables GT1, GT2, a plurality of known characteristic spectrum groups KS1, KS2, and a print setting table PST. The machine learning models 201, 202 are used for calculations by the class classification processing unit 114. An example of the configuration and operation of the machine learning models 201, 202 will be described later. The spectral data groups SD1, SD2 are a collection of labeled spectral data used for learning the machine learning models 201, 202. The medium identifier list IDL is a list in which a medium identifier and spectral data are registered for each print medium. The plurality of group management tables GT1, GT2 are tables indicating the management state of the spectral data groups SD1, SD2. The known characteristic spectrum groups KS1, KS2 are a collection of characteristic spectra obtained when teacher data is input again to the learned machine learning models 201, 202. The characteristic spectra will be described later. The print setting table PST is a table in which print settings suitable for each print medium are registered.
[0012] 3 is an explanatory diagram showing the configuration of the first machine learning model 201. The machine learning model 201 includes, in order from the input data IM side, a convolution layer 211, a primary vector neuron layer 221, a first convolution vector neuron layer 231, a second convolution vector neuron layer 241, and a classification vector neuron layer 251. Of these five layers 211 to 251, the convolution layer 211 is the lowest layer, and the classification vector neuron layer 251 is the highest layer. In the following description, the layers 211 to 251 are also referred to as the "Conv layer 211", the "PrimeVN layer 221", the "ConvVN1 layer 231", the "ConvVN2 layer 241", and the "ClassVN layer 251", respectively.
[0013] In this embodiment, the input data IM is spectral data, and is therefore one-dimensionally arranged data. For example, the input data IM is data in which 36 representative values are extracted at intervals of 10 nm from spectral data in the range of 380 nm to 730 nm.
[0014] 3, two convolution vector neuron layers 231 and 241 are used, but the number of convolution vector neuron layers is arbitrary, and the convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.
[0015] 3 further includes a similarity calculation unit 261 that generates similarities. The similarity calculation unit 261 can calculate similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN, which will be described later, from the outputs of the ConvVN1 layer 231, the ConvVN2 layer 241, and the ClassVN layer 251. However, the similarity calculation unit 261 may be omitted.
[0016] The configuration of each of the layers 211 to 251 can be described as follows. <Description of the configuration of the first machine learning model 201> ·Conv layer 211: Conv[32,6,2] ·PrimeVN layer 221: PrimeVN[26,1,1] ·ConvVN1 layer 231:ConvVN1[20,5,2] ·ConvVN2 layer 241:ConvVN2[16,4,1] ·ClassVN layer 251:ClassVN[n1+1,3,1] Vector dimension VD: VD=16 In the description of each of these layers 211 to 251, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, the kernel size, and the stride. For example, the layer name of the Conv layer 211 is "Conv", the number of channels is 32, the kernel size is 1x6, and the stride is 2. In FIG. 3, these descriptions are shown under each layer. The hatched rectangles drawn in each layer represent the kernels used when calculating the output vector of the adjacent upper layer. In this embodiment, since the input data IM is data of a one-dimensional array, the kernel also has a one-dimensional array. Note that the parameter values used in the description of each of the layers 211 to 251 are merely examples and can be changed arbitrarily.
[0017] The Conv layer 211 is a layer composed of scalar neurons. The other four layers 221 to 251 are layers composed of vector neurons. A vector neuron is a neuron that takes a vector as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.
[0018] In FIG. 3, for the Conv layer 211, a first axis x and a second axis y that define the planar coordinates of the node array, and a third axis z that represents the depth are shown. Also, it is shown that the sizes of the Conv layer 211 in the x, y, and z directions are 1, 16, and 32. The sizes in the x and y directions are called "resolutions." In this embodiment, the resolution in the x direction is always 1. The size in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes indicating the position of each node in other layers. However, in FIG. 3, the illustration of these axes x, y, and z is omitted in layers other than the Conv layer 211.
[0019] As is well known, the resolution W1 in the y direction after convolution is given by the following equation: W1 = Ceil{(W0 - Wk + 1) / S} (1) Here, W0 is the resolution before convolution, Wk is the kernel size, S is the stride, and Ceil{X} is a function that rounds up X. The resolution of each layer shown in FIG. 3 is an example in which the resolution of the input data IM in the y direction is 36, and the actual resolution of each layer is changed appropriately depending on the size of the input data IM.
[0020] The ClassVN layer 251 has (n1+1) channels. In the example of FIG. 3, (n1+1)=11. From these channels, judgment values Class1-1 to Class1-10 for a plurality of known classes and a judgment value Class1-UN indicating an unknown class are output. The class having the largest value among these judgment values Class1-1 to Class1-10 and Class1-UN corresponds to the class to which the input data IM belongs. In general, n1 is an integer of 2 or more, and is the number of known classes that can be classified using the first machine learning model 201. In any one machine learning model, it is preferable to set an upper limit nmax and a lower limit nmin in advance for the number of known classes that can be classified.
[0021] The judgment value Class1-UN indicating an unknown class may be omitted. In this case, if the largest value among the judgment values Class1-1 to Class1-10 for the known classes is less than a predetermined threshold, the class of the input data IM is judged to be unknown.
[0022] 4 is an explanatory diagram showing the configuration of the second machine learning model 202. Like the first machine learning model 201, this machine learning model 202 has a Conv layer 212, a PrimeVN layer 222, a ConvVN1 layer 232, a ConvVN2 layer 242, a ClassVN layer 252, and a similarity calculation unit 262.
[0023] The configuration of each of the layers 212 to 252 can be described as follows. <Description of the configuration of the second machine learning model 202> ·Conv layer 212: Conv[32,6,2] ·PrimeVN layer 222: PrimeVN[26,1,1] ·ConvVN1 layer 232:ConvVN1[20,5,2] ·ConvVN2 layer 242:ConvVN2[16,4,1] ·ClassVN layer 252:ClassVN[n2+1,3,1] Vector dimension VD: VD=16
[0024] As can be understood by comparing FIG. 3 and FIG. 4, among the layers 212 to 252 of the second machine learning model 202, the lower four layers 212 to 242 have the same configuration as the layers 211 to 241 of the first machine learning model 201. On the other hand, the top layer 252 of the second machine learning model 202 differs from the top layer 251 of the first machine learning model 201 only in the number of channels. In the example of FIG. 4, the ClassVN layer 252 has (n2+1) channels, where (n2+1)=7. From these channels, judgment values Class2-1 to Class2-6 for a plurality of known classes and a judgment value Class2-UN indicating an unknown class are output. In the second machine learning model 202, it is preferable to set the same upper limit value nmax and lower limit value nmin as those of the first machine learning model 201 for the number of known classes.
[0025] The second machine learning model 202 is configured to have at least one known class different from the first machine learning model 201. In addition, since the first machine learning model 201 and the second machine learning model 202 have different classifiable classes, the values of the kernel elements are also different from each other. In the present disclosure, when N is an integer of 2 or more, any one of the N machine learning models is configured to have at least one known class different from the other machine learning models. Note that, in the present embodiment, the number N of machine learning models is 2 or more, but the present disclosure is also applicable to the case where only one machine learning model is used.
[0026] 5 is a flowchart showing the procedure of a preparation step for a machine learning model. This preparation step is a step that is executed by, for example, the manufacturer of the printer 10.
[0027] In step S10, spectral data of a plurality of initial print media is generated as initial spectral data. In this embodiment, the initial print media used for learning the machine learning model in the preparation process are all optional print media. In this disclosure, an "optional print medium" is a print medium that can be a target of class classification processing by the machine learning model, and can be excluded from the target of class classification processing even without an exclusion instruction from the user. On the other hand, the print media added in the medium addition process described below are required print media that cannot be excluded from the target of class classification processing without an exclusion instruction from the user. However, some or all of the initial print media may be required print media.
[0028] In step S10, initial spectral data is generated by performing spectral measurement on a plurality of initial print media in an unprinted state using the spectrometer 30. At this time, it is preferable to perform data extension in consideration of variations in spectral reflectance. In general, the spectral reflectance varies depending on the date of color measurement and the measurement device. Data extension is a process for generating a plurality of spectral data by adding random variations to the measured spectral data in order to simulate such variations. Note that the initial spectral data may be generated virtually without performing spectral measurement on the actual print media. In this case, the initial print media is also virtual.
[0029] In step S20, a medium identifier list IDL is created for a plurality of initial print media. FIG. 6 is an explanatory diagram showing the medium identifier list IDL. In the medium identifier list IDL, a medium identifier, a medium name, a data sub-number, and spectral data assigned to each print medium are registered. In this example, medium identifiers "A-1" to "A-16" are assigned to 16 print media. The medium name is the name of the print medium displayed in a window for the user to set print conditions. The data sub-number is a number for distinguishing between a plurality of spectral data related to the same print medium. In this example, three pieces of spectral data are registered for each print medium. However, the number of pieces of spectral data for each print medium may be different. Although it is sufficient that one or more pieces of spectral data are registered for each print medium, it is preferable that a plurality of pieces of spectral data are registered.
[0030] In step S30 in FIG. 5, print settings are created for each of a plurality of initial print media and registered in the print setting table PST. FIG. 7 is an explanatory diagram showing the print setting table PST. In each record of the print setting table PST, a medium identifier and print settings are registered for each print medium. In this example, printer profiles PR1 to PR16, medium feed speeds FS1 to FS16, and drying times DT1 to DT16 are registered as print settings. The printer profiles PR1 to PR16 are output color profiles of the printer 10 and are created for each print medium. Specifically, a printer profile can be created by printing a test chart on a print medium without color correction using the printer 10, subjecting the test chart to spectroscopic measurement by the spectrometer 30, and processing the spectroscopic measurement results by the print setting creation unit 116. The medium feed speeds FS1 to FS16 and drying times DT1 to DT16 can also be determined experimentally. The "drying time" refers to the time it takes to dry the print medium after printing in a dryer (not shown) in the printer 10. For printers that dry the print media by blowing air on it after printing, the "drying time" is the time it takes to blow air. For printers that do not have a dryer, the "drying time" is the waiting time for the media to dry naturally. Note that other initial items may be set as print settings, but it is preferable to create print settings that include at least a printer profile.
[0031] In step S40 of FIG. 5, grouping is performed by clustering a plurality of initial spectral data for a plurality of initial printing media. FIG. 8 is an explanatory diagram showing spectral data grouped by clustering. In this example, a plurality of spectral data are grouped into a first spectral data group SD1 and a second spectral data group SD2. For example, k-means can be used as the clustering process. The spectral data groups SD1 and SD2 have representative points G1 and G2 that represent the centers of the respective spectral data groups SD1 and SD2. These representative points G1 and G2 are, for example, centers of gravity. When the spectral data is composed of reflectances at m wavelengths, it is possible to calculate the distance between the spectral data and the center of gravity of the plurality of spectral data by regarding one piece of spectral data as data representing one point in an m-dimensional space. In FIG. 8, for convenience of illustration, a plurality of points of spectral data are drawn in a two-dimensional space, but in reality, the spectral data can be expressed as points in an m-dimensional space. As described below, when a new print medium is added as a target for classification processing, these representative points G1 and G2 are used to determine which of the multiple spectral data groups SD1 and SD2 the spectral data of the added print medium is closest to. Note that points other than the center of gravity may be used as the representative points G1 and G2. For example, the average values of the maximum and minimum reflectance values at each wavelength may be calculated for multiple spectral data belonging to one group, and the spectral data consisting of these average values may be used as the representative point.
[0032] In this embodiment, the multiple spectral data are grouped into two spectral data groups SD1 and SD2, but only one spectral data group may be used, or three or more spectral data groups may be created. In addition, multiple spectral data groups may be created by a method other than the clustering process. However, if multiple spectral data are grouped by the clustering process, it is possible to group spectral data that are similar to each other into the same group. If multiple machine learning models are trained using each of such multiple spectral data groups, the accuracy of the class classification process by the machine learning model can be improved compared to the case where the clustering process is not performed. Note that even if spectral data of a new printing medium is added after grouping by the clustering process, it is possible to maintain a state equivalent to the state grouped by the clustering process.
[0033] In step S50 in FIG. 5, group management tables GT1 and GT2 are created. FIG. 9 is an explanatory diagram showing group management tables GT1 and GT2. In each record of group management tables GT1 and GT2, a group number, a medium identifier, a data sub number, a distance from a representative point, a model number, a class label, a current area, and coordinates of a representative point are registered for one spectral data. The group number is a number for distinguishing a plurality of group management tables GT1 and GT2. The medium identifier and the data sub number are used to distinguish each spectral data, similarly to the medium identifier list IDL described in FIG. 6. The model number is a number for identifying a machine learning model that performs learning using a group of spectral data of the group. Here, the symbols "201" and "202" of the two machine learning models 201 and 202 shown in FIG. 3 and FIG. 4 are used as model numbers. The "class label" is a value corresponding to the result of class classification processing by the machine learning model, and is also used as a label when the spectral data is used as training data. The model number and the class label are set for each medium identifier. "Current area" indicates whether the spectroscopic data belongs to the teacher area or the waiting area. "Teacher area" means that the spectroscopic data is actually being used for training the machine learning model. "Waiting area" means that the spectroscopic data is not being used for training the machine learning model and is in a waiting state from the teacher area. In the preparation process, all the spectroscopic data belongs to the teacher area because it is used for training the machine learning model.
[0034] In step S60 in FIG. 5, the user creates a machine learning model to be used in the class classification process and sets its parameters. In this embodiment, two machine learning models 201, 202 shown in FIG. 3 and FIG. 4 are created and their parameters are set. However, in step S60, only one machine learning model may be created, or three or more machine learning models may be created. In step S70, the class classification processing unit 114 executes learning of the machine learning models 201, 202 using the spectroscopic data groups SD1, SD2. When learning is completed, the learned machine learning models 201, 202 are stored in the memory 120.
[0035] In step S80, the class classification processing unit 114 inputs the spectral data groups SD1 and SD2 again into the trained machine learning models 201 and 202 to generate known feature spectrum groups KS1 and KS2. The known feature spectrum groups KS1 and KS2 are collections of feature spectra, which will be described below. Below, a method for generating the known feature spectrum group KS1 associated with the machine learning model 201 will be mainly described.
[0036] FIG. 10 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the trained machine learning model 201. Here, the feature spectrum Sp obtained from the output of the ConvVN1 layer 231 will be described. The horizontal axis of FIG. 10 is a spectral position represented by a combination of an element number ND of an output vector of a node at one planar position (x, y) of the ConvVN1 layer 231 and a channel number NC. In this embodiment, since the vector dimension of the node is 16, the element number ND of the output vector is 16, ranging from 0 to 15. In addition, since the number of channels of the ConvVN1 layer 231 is 20, the channel number NC is 20, ranging from 0 to 19.
[0037] The vertical axis of Fig. 10 is the feature value C at each spectral position. V In this example, the feature value C V is the value of each element of the output vector V ND Note that the feature value C VThe value of each element of the output vector V ND Alternatively, the activation value may be used as it is. In the latter case, the feature value C included in the feature spectrum Sp may be used as it is. V The number of nodes is equal to the number of channels, which is 20. The activation value is the value corresponding to the vector length of the output vector of the node.
[0038] The number of feature spectra Sp obtained from the output of the ConvVN1 layer 231 for one piece of input data is equal to the number of planar positions (x, y) of the ConvVN1 layer 231, that is, 1×6=6. Similarly, for one piece of input data, three feature spectra Sp are obtained from the output of the ConvVN2 layer 241, and one feature spectrum Sp is obtained from the output of the ClassVN layer 251.
[0039] When training data is input again to the trained machine learning model 201, the similarity calculation unit 261 calculates the feature spectrum Sp shown in FIG. 10 and registers it in the known feature spectrum group KS1.
[0040] 11 is an explanatory diagram showing the configuration of the known feature spectrum group KS1. In this example, the known feature spectrum group KS1 includes a known feature spectrum group KS1_ConvVN1 obtained from the output of the ConvVN1 layer 231, a known feature spectrum group KS1_ConvVN2 obtained from the output of the ConvVN2 layer 241, and a known feature spectrum group KS1_ConvVN1 obtained from the output of the ClassVN layer 251.
[0041] Each record of the known feature spectrum group KS1_ConvVN1 includes a record number, a layer name, a label Lb, and a known feature spectrum KSp. The known feature spectrum KSp is the same as the feature spectrum Sp in FIG. 10 obtained in response to the input of the teacher data. In the example of FIG. 11, the spectroscopic data group SD1 is input to the trained machine learning model 201, and the known feature spectrum KSp associated with each value of the label Lb is generated and registered from the output of the ConvVN1 layer 231. For example, N1_1max known feature spectrum KSp is registered in association with the label Lb=1, N1_2max known feature spectrum KSp is registered in association with the label Lb=2, and N1_n1max known feature spectrum KSp is registered in association with the label Lb=n1. N1_1max, N1_2max, and N1_n1max are each an integer equal to or greater than 2. As described above, each label Lb corresponds to a known class different from each other. Therefore, it can be understood that each known feature spectrum KSp in the known feature spectrum group KS1_ConvVN1 is registered in association with one of a plurality of known classes. The same is true for the other known feature spectrum groups KS1_ConvVN2, KS1_ConvVN1.
[0042] The spectral data group used in step S80 does not need to be the same as the spectral data groups SD1 and SD2 used in step S70. However, if part or all of the spectral data groups SD1 and SD2 used in step S70 are used in step S80, there is an advantage in that there is no need to prepare new training data. Step S80 may be omitted.
[0043] 12 is a flowchart showing the processing procedure of a medium discrimination / printing process using a trained machine learning model. This medium discrimination / printing process is executed by, for example, a user of the printer 10.
[0044] In step S210, it is determined whether or not a discrimination process is required for the target print medium, which is the print medium to be processed. If the discrimination process is not required, that is, if the type of the target print medium is known, proceed to step S280 to select print settings suitable for the target print medium, and then execute printing using the target print medium in step S290. On the other hand, if the type of the target print medium is unknown and a discrimination process is required, proceed to step S220.
[0045] In step S220, the class classification processing unit 114 acquires target spectral data by causing the spectrometer 30 to perform spectroscopic measurement of the target print medium. This target spectral data is subjected to class classification processing using a machine learning model.
[0046] In step S230, the classification processing unit 114 inputs the target spectral data into the existing trained machine learning models 201, 202 to perform classification processing of the target spectral data. In this case, either a first processing method in which the multiple machine learning models 201, 202 are used one by one in sequence, or a second processing method in which the multiple machine learning models 201, 202 are used simultaneously, can be used. In the first processing method, the classification processing is first performed using one machine learning model 201, and when it is determined that the target spectral data belongs to an unknown class as a result, the classification processing is performed using the other machine learning model 202. In the second processing method, the two machine learning models 201, 202 are used simultaneously to perform classification processing on the same target spectral data in parallel, and the classification processing unit 114 integrates the processing results. According to an experiment by the inventor of the present disclosure, the second processing method is more preferable because it requires a shorter processing time than the first processing method.
[0047] In step S240, the classification processing unit 114 determines whether the target spectral data belongs to an unknown class or a known class based on the result of the classification processing in step S230. If the target spectral data belongs to an unknown class, the target print medium is a new print medium that does not correspond to any of the multiple initial print media used in the preparation process or the print medium added in the medium addition processing described below, so the process proceeds to step S300 described below, where the medium addition processing is executed. On the other hand, if the target spectral data belongs to a known class, the process proceeds to step S250.
[0048] In step S250, the similarity to the known feature spectrum group is calculated using one machine learning model that has been determined to belong to a known class among the multiple machine learning models 201 and 202. For example, when the processing of the first machine learning model 201 determines that the target spectral data belongs to a known class, the similarity calculation unit 261 calculates the similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN to the known feature spectrum group KS1 from the outputs of the ConvVN1 layer 231, the ConvVN2 layer 241, and the ClassVN layer 251. On the other hand, when the processing of the second machine learning model 202 determines that the target spectral data belongs to a known class, the similarity calculation unit 262 calculates the similarities S2_ConvVN1, S2_ConvVN2, and S2_ClassVN to the known feature spectrum group KS2. A method for calculating the similarity S1_ConvVN1 from the output of the ConvVN1 layer 231 of the first machine learning model 201 will be described below.
[0049] The similarity S1_ConvVN1 can be calculated, for example, using the following formula. S1_ConvVN1(Class)=max[G{Sp(i,j),KSp(Class,k)}] Here, "Class" is an ordinal number for multiple classes, G{a,b} is a function for calculating the similarity between a and b, Sp(i,j) is a feature spectrum at all planar positions (i,j) obtained according to the target spectral data, KSp(Class,k) is all known feature spectra associated with the ConvVN1 layer 231 and a specific "Class", and max[X] is a logical operation for taking the maximum value of X. That is, the similarity S1_ConvVN1 is the maximum value of the similarities calculated between each of the feature spectra Sp(i,j) at all planar positions (i,j) of the ConvVN1 layer 231 and each of all known feature spectra KSp(k) corresponding to a specific class. Such a similarity S1_ConvVN1 is calculated for each of multiple classes corresponding to multiple labels Lb. The similarity S1_ConvVN1 indicates the degree to which the target spectral data is similar to the features of each class.
[0050] Similarities S1_ConvVN2 and S1_ClassVN related to the outputs of the ConvVN2 layer 241 and the ClassVN layer 251 are also generated in the same manner as the similarity S1_ConvVN1. It is not necessary to generate all three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN, but it is preferable to generate one or more of them. In the present disclosure, the layer used to generate the similarity is also referred to as a "specific layer."
[0051] In step S260, the classification processing unit 114 presents the similarity obtained in step S250 to the user, and the user checks whether the similarity is consistent with the result of the classification process. The similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN represent the degree to which the target spectral data is similar to the characteristics of each class, so that the result of the classification process can be checked from at least one of these similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN. For example, when at least one of the three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN does not match the result of the classification process, it can be determined that the two are not consistent. In another embodiment, when all of the three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN do not match the result of the classification process, it may be determined that the two are not consistent. In general, when a predetermined number of similarities among the multiple similarities generated from the outputs of multiple specific layers do not match the result of the classification process, it may be determined that the two are inconsistent. Note that the determination in step S260 may be performed by the class classification processing unit 114. Also, steps S250 and S260 may be omitted.
[0052] If the similarity is consistent with the result of the classification process, the process proceeds to step S270, where the classification processing unit 114 determines the medium identifier of the target print medium according to the result of the classification process. This process is performed, for example, by referring to the group management tables GT1 and GT2 shown in FIG. 9. In step S280, the print processing unit 112 selects print settings according to the medium identifier. This process is performed by referring to the print setting table PST shown in FIG. 7. In step S290, the print processing unit 112 executes printing according to the print settings.
[0053] If it is determined in step S260 that the similarity is not consistent with the result of the classification process, the target print medium is a new print medium that does not correspond to any of the multiple initial print media used in the preparation process or the print medium added in the medium addition process described below, so the process proceeds to step S300 described below. In step S300, a medium addition process is executed to make the new print medium a target for the classification process. In the medium addition process, the machine learning model is updated and added, so the medium addition process can be considered as part of the process of preparing the machine learning model.
[0054] Fig. 13 is a flowchart showing the procedure of the medium addition process, and Fig. 14 is an explanatory diagram showing the management state of the spectral data group in the medium addition process. In the following description, a new print medium added as a target for the class classification process is called an "added print medium" or an "added medium".
[0055] In step S310, the class classification processing unit 114 searches for a machine learning model that is closest to the spectral data of the additional print medium from among the existing machine learning models 201, 202. The "machine learning model that is closest to the spectral data of the additional print medium" refers to the machine learning model in which the distance between the representative points G1, G2 of the teacher data group used in learning the machine learning models 201, 202 and the spectral data of the additional print medium is the smallest. The distance between each of the representative points G1, G2 and the spectral data of the additional print medium can be calculated, for example, as the Euclidean distance. The teacher data group that is the smallest distance from the spectral data of the additional print medium is also called the "proximal teacher data group."
[0056] In step S320, the class classification processing unit 114 determines whether the number of classes corresponding to the required print media has reached the upper limit for the machine learning model searched for in step S310. As described above, in this embodiment, all the initial print media used in the preparation process are optional print media, and all print media added after the preparation process are required print media. If the number of classes corresponding to the required print media has not reached the upper limit, the process proceeds to step S330, and the machine learning model is trained using the teacher data to which the spectral data of the additional print media has been added. The state S1 in FIG. 14 shows the state of the spectral data group SD2 used in the training of the machine learning model 202 in the above-mentioned preparation process, and the state S2 shows the state in which the spectral data of the additional print media has been added as the spectral data of the required print media in step S330. In FIG. 14, "optional medium" means the spectral data of the optional print media used in the preparation process, and "required medium" means the spectral data of the required print media added by the medium addition process in FIG. 13. The "teacher area" means that the spectral data is teacher data that is actually used in training the machine learning model. The "shelter area" means that the spectroscopic data is not used in learning the machine learning model and is in a state of being evacuated from the teacher area. Furthermore, a state in which the teacher area is empty means that the number of classes in the machine learning model 202 has not reached the upper limit. In state S1, since the number of classes corresponding to the required print medium in the machine learning model 202 has not reached the upper limit, the spectroscopic data of the additional print medium is added to the teacher area, resulting in state S2, and the machine learning model 202 is re-learned using the spectroscopic data belonging to the teacher area as teacher data. In this re-learning, only the added spectroscopic data may be used as teacher data.
[0057] FIG. 15 shows the medium identifier list IDL in state S2 of FIG. 14, and FIG. 16 shows the group management table GT2 for the second spectral data group SD2 in state S2. In the medium identifier list IDL, "B-1" is assigned as the medium identifier of the added print medium, and the medium name and spectral data are registered. It is preferable that multiple pieces of spectral data are generated for the spectral data of the added print medium by performing data extension that adds random variation to the measured spectral data. In the group management table GT2, multiple pieces of spectral data are also registered for the added print medium with the medium identifier B-1. The representative point G2 for the teacher data group in the second spectral data group SD2 is recalculated including the added spectral data.
[0058] When more printing media are added from state S2 in FIG. 14, the state transitions to state S3, state S4, and state S5. In states S2 to S4, as in state S1, the number of classes corresponding to the required printing media in the machine learning model 202 has not reached the upper limit, so step S330 is executed, and the spectral data of the additional printing media is added to the teacher area, and re-learning of the machine learning model 202 is executed. Note that in state S3, the sum of the number of classes corresponding to the required printing media and the number of classes corresponding to the optional printing media in the machine learning model 202 has reached the upper limit, and there is no space in the teacher area. Therefore, when transitioning from state S3 to state S4, in step S330, the spectral data of the additional printing medium, which is the required printing medium, is added to the teacher area, and the spectral data of the optional printing medium is deleted from the teacher area. The deleted spectral data is saved to a save area. The reason for saving the spectral data to the save area is to make it possible to reuse the spectral data. It is preferable that the spectral data of the arbitrary printing medium to be evacuated from the teacher area to the evacuation area is selected from the data that is the farthest from the representative point of the teacher data group. This reduces the distance between the teacher data, thereby improving the accuracy of the class classification process.
[0059] In state S5 of FIG. 14, the number of classes corresponding to the required print medium in the machine learning model 202 has reached the upper limit. In this case, the process proceeds from step S320 to step S340. In step S340, the class classification processing unit 114 searches for a machine learning model that belongs to the same group as the machine learning model searched for in step S310 and for which the number of classes corresponding to the required print medium has not reached the upper limit. If such a machine learning model exists, the process proceeds from step S350 to step S360, and the machine learning model is trained using the teacher data to which the spectral data of the additional print medium has been added. This process is the same as the process in step S330 described above.
[0060] If the search in step S340 does not find a machine learning model, the process proceeds from step S350 to step S370, where a new machine learning model is created and the new machine learning model is trained using training data including the spectral data of the additional print medium. This process corresponds to the process of changing from state S5 to state S6 in FIG. 14. In state S5, the number of classes corresponding to the required print medium in the machine learning model 202 reaches the upper limit value, and there are no other machine learning models belonging to the same group. Therefore, by the process of step S370, as shown in state S6, a new machine learning model 203 is created and the new machine learning model is trained using training data including the spectral data of the additional print medium, which is the new required print medium. At this time, since the spectral data of the additional print medium alone is insufficient as training data, the spectral data of one or more optional print media that have been sheltered in the shelter area are also used as training data. In this way, the accuracy of the class classification process by the new machine learning model 203 can be improved.
[0061] Note that the above-mentioned steps S340 to S360 may be omitted, and if the number of classes of the essential print medium is equal to the upper limit in step S320, the process may proceed directly to step S370.
[0062] FIG. 17 shows the group management table GT2 for the second group in state S6. The spectral data of the print media with the medium identifiers A-1 to A-6 is the spectral data of the optional print media used in the preparation process. The spectral data of the print media with the medium identifiers B-1 to B-11 is the spectral data of the mandatory print media added after the preparation process. In this group management table GT2, the states of the spectral data for the two machine learning models 202 and 203 belonging to the same group are registered. For the machine learning model 202, the spectral data for the added 10 mandatory print media is stored in the teacher area, and the spectral data for the six optional print media is stored in the evacuation area. For the machine learning model 203, the spectral data for the one mandatory print medium and the spectral data for the six optional print media are stored in the teacher area, and the evacuation area is empty. The representative points G2a and G2b of the teacher data group of each machine learning model 202 and 203 are calculated using the spectral data stored in the respective teacher areas.
[0063] The medium addition process shown in FIG. 13 can also be executed when the number of existing machine learning models is one. When the number of existing machine learning models is one, for example, the second machine learning model 202 shown in FIG. 4 is not prepared, and the process of FIG. 12 is executed using only the first machine learning model 201 shown in FIG. 3. In this case, the process of step S370 in FIG. 13 is a process of adding the second machine learning model 202 as a new machine learning model. In this way, in the classification process performed using only the first machine learning model 201, the process of adding the second machine learning model 202 as a new machine learning model when it is determined that the input data belongs to an unknown class can also be understood as an example of a process of preparing two machine learning models 201 and 202.
[0064] When a machine learning model is updated or added in any of steps S330, S360, and S370, in step S380, the class classification processing unit 114 re-inputs the training data into the updated or added machine learning model to generate a group of known feature spectra. This process is the same as the process in step S230 in Fig. 12, and therefore a description thereof will be omitted. In step S390, the print setting creation unit 116 creates print settings for the added target print medium. This process is the same as the process in step S30 in Fig. 5, and therefore a description thereof will be omitted.
[0065] When the process of Fig. 13 ends in this way, the process of Fig. 12 also ends. After this, the process of Fig. 12 is executed again at any timing.
[0066] In the process of Fig. 13 described above, the process of step S310 corresponds to a process of selecting a proximity teacher data group having a representative point closest to the spectral data of the additional print medium from among the N teacher data groups used to train the N machine learning models, and selecting a specific machine learning model that was trained using the proximity teacher data group. By performing such a process, even if the spectral data of the additional print medium is added to the proximity teacher data group, the teacher data group after the addition can be maintained in a state equivalent to the state grouped by the clustering process. As a result, the accuracy of the class classification process by the machine learning model can be improved.
[0067] 13, it is possible to add a new print medium to the class classification process targets. On the other hand, it is also possible to exclude a print medium from the class classification process targets in response to a user instruction.
[0068] FIG. 18 is a flowchart showing the processing procedure of a medium excluding step of excluding a print medium from the target of the classification process. In step S410, the classification processing unit 114 receives an instruction to exclude a registered print medium from a user. In step S420, the spectral data of the excluded print medium is deleted from the teacher area of the machine learning model that discriminates the excluded print medium, and the spectral data of the optional print medium is supplemented to the teacher area as necessary. "As necessary" means, for example, a case where the number of classes of the machine learning model falls below a lower limit. For example, when an instruction to exclude a required print medium is received from the state S5 of FIG. 14, the spectral data of the required print medium is deleted from the teacher area of the machine learning model 202. If a large number of required print media are excluded, the number of classes of the machine learning model 202 falls below a lower limit. In that case, the spectral data of the optional print medium is moved from the waiting area to the teacher area to supplement the spectral data as teacher data. As a result, the optional print medium corresponding to the supplemented spectral data is added as a class of the machine learning model 202. This prevents the accuracy of the classification process from excessively decreasing due to the number of classes in the machine learning model 202 falling below the lower limit. It is preferable to select the spectral data of the arbitrary printing medium to be replenished from the waiting area to the teacher area that is the closest distance from the representative point of the teacher data group. This reduces the distance between the teacher data, thereby improving the accuracy of the classification process. The medium identifier list IDL, print setting table PST, and group management tables GT1 and GT2 are updated as appropriate in response to the deletion or movement of such spectral data.
[0069] In step S430, the class classification processing unit 114 executes re-learning of the machine learning model using the training data updated in step S420. In step S440, the class classification processing unit 114 inputs the training data again into the re-learned machine learning model to generate a group of known feature spectra. This process is the same as the process in step S230 in Fig. 12, and therefore a description thereof will be omitted. By executing the above-described medium exclusion process, essential print media can be excluded from the target of the class classification process of the machine learning model.
[0070] As described above, in this embodiment, since the classification process is performed using N machine learning models, where N is an integer equal to or greater than 1, it is possible to accurately distinguish between print media with similar optical characteristics. Furthermore, if the classification process is performed using two or more machine learning models, the process can be performed faster than when classification into multiple classes is performed using a single machine learning model.
[0071] In the above embodiment, a vector neural network type machine learning model using vector neurons is used, but instead, a machine learning model using scalar neurons such as a normal convolutional neural network may be used. However, the vector neural network type machine learning model is preferable in that it has higher accuracy of class classification processing than the machine learning model using scalar neurons.
[0072] B. How to calculate the output vector of each layer of the machine learning model: The method of calculating the output of each layer in the first machine learning model 201 shown in Fig. 3 is as follows. The second machine learning model 202 is similar.
[0073] Each node of the PrimeVN layer 221 regards the scalar output of the 1×1×32 nodes of the Conv layer 211 as a 32-dimensional vector, and obtains the vector output of the node by multiplying this vector by a transformation matrix. This transformation matrix is an element of a 1×1 kernel, and is updated by learning of the machine learning model 201. Note that it is also possible to integrate the processing of the Conv layer 211 and the PrimeVN layer 221 and configure them as one primary vector neuron layer.
[0074] When the PrimeVN layer 221 is referred to as the "lower layer L" and the ConvVN1 layer 231 adjacent to the upper side is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following formula.
number
[0075] As the normalization function F(X), for example, the following formula (4a) or (4b) can be used.
number
[0076] In the above formula (4a), the sum vector u j Norm of |u j The activation value a is normalized by the softmax function j On the other hand, in equation (4b), the sum vector u j Norm of |u j| is the norm |u j Activation value a by dividing by the sum of | j It should be noted that the normalization function F(X) may be a function other than equation (4a) or (4b).
[0077] The ordinal number i in the above equation (3) is the output vector M L+1 j The integer n is a number that is assigned for convenience to the nodes in the lower layer L used to determine the output vector M L+1 j is the number of nodes in the lower layer L used to determine . Thus, the integer n is given by n = Nk × Nc (6) Here, Nk is the number of elements of the kernel, and Nc is the number of channels in the lower layer, the PrimeVN layer 221. In the example of FIG.
[0078] A kernel used to calculate the output vector of the ConvVN1 layer 231 has 1×3×26=78 elements, with the kernel size 1×3 as the surface size and the number of channels in the lower layer being 26 as the depth. Each of these elements is a prediction matrix W L ij In addition, 20 sets of this kernel are required to generate output vectors for 20 channels of the ConvVN1 layer 231. Therefore, the prediction matrix W L ij The number of prediction matrices W is 78 × 20 = 1560. L ij is updated by learning of the machine learning model 201.
[0079] As can be seen from the above equations (2) to (5), the output vector M L+1 j is calculated by the following calculation: (a) The output vector M of each node in the lower layer L L i The prediction matrix W L ij Multiplying by the predicted vector v ij Seeking (b) Prediction vector v obtained from each node in the lower layer L ij The sum vector u is the sum, i.e., linear combination, of j Seeking (c) Sum vector u j Norm of |u j The activation value a, which is a normalization factor by normalizing | j Seeking (d) Sum vector u j The norm |u j Divide by | and then the activation value a j Multiply by.
[0080] In addition, the activation value a j is the norm |u j is the normalization factor obtained by normalizing |. Therefore, the activation value a j can be considered as an index showing the relative output strength of each node among all nodes in the upper layer L+1. The norm used in (4), (4a), (4b), and (5) is typically the L2 norm, which represents the vector length. In this case, the activation value a j is the output vector M L+1 j The activation value a corresponds to the vector length of j Since it is only used in the above equations (4) and (5), it does not need to be output from the node. However, the activation value a j It is also possible to configure the upper layer L+1 to output to the outside.
[0081] The configuration of a vector neural network is almost the same as that of a capsule network, and the vector neurons of a vector neural network correspond to the capsules of a capsule network. However, the calculations according to the above formulas (2) to (5) used in a vector neural network are different from those used in a capsule network. The biggest difference between the two is that in a capsule network, the predicted vector v ij Each of the vector neural networks in this embodiment is multiplied by a weight, and the weight is searched for by repeating dynamic routing multiple times. On the other hand, in the vector neural network of this embodiment, the output vector M L+1 j Since the above is obtained, there is no need to repeat dynamic routing, and the calculation speed is faster. In addition, the vector neural network of this embodiment has the advantage that the amount of memory required for calculation is smaller than that of a capsule network, and according to an experiment by the inventor of the present disclosure, the amount of memory required is about 1 / 2 to 1 / 3.
[0082] A vector neural network is the same as a capsule network in that it uses nodes that use vectors as input and output. Therefore, the advantage of using vector neurons is also common to capsule networks. In addition, the multiple layers 211 to 251 are the same as normal convolutional neural networks in that the higher the layer, the larger the area of the feature represented, and the lower the layer, the smaller the area of the feature represented. Here, "feature" means a characteristic part included in the input data to the neural network. A vector neural network or capsule network is superior to a normal convolutional neural network in that the output vector of a certain node contains spatial information that represents the spatial information of the feature represented by the node. That is, the vector length of the output vector of a certain node represents the probability of the existence of the feature represented by the node, and the vector direction represents spatial information such as the direction and scale of the feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the positional relationship of each feature. Alternatively, it can be said that the vector direction of the output vectors of the two nodes represents the variation of the feature. For example, if a node corresponds to the feature of "eyes", the direction of the output vector can represent variations such as the thinness of the eyes and the way they are lifted. In a conventional convolutional neural network, it is said that spatial information of features is lost due to pooling processing. As a result, vector neural networks and capsule networks have the advantage of being superior in the performance of identifying input data compared to conventional convolutional neural networks.
[0083] The advantages of vector neural networks can also be considered as follows. That is, the advantage of vector neural networks is that the output vectors of nodes express the features of the input data as coordinates in a continuous space. Therefore, output vectors can be evaluated such that if the vector directions are close, the features are similar. Another advantage is that even if the features contained in the input data are not covered by the training data, the features can be determined by interpolation. On the other hand, normal convolutional neural networks have the disadvantage that the features of the input data cannot be expressed as coordinates in a continuous space because of the chaotic compression caused by the pooling process.
[0084] The output of each node of the ConvVN2 layer 241 and the ClassVN layer 251 is determined similarly using the above-mentioned formulas (2) to (5), so detailed explanations are omitted. The resolution of the top layer, the ClassVN layer 251, is 1×1, and the number of channels is (n1+1).
[0085] The output of the ClassVN layer 251 is converted into a plurality of judgment values Class1-1 to Class1-2 for known classes, and a judgment value Class1-UN indicating an unknown class. These judgment values are usually normalized by a softmax function. Specifically, for example, a judgment value for each class can be obtained by performing an operation in which the vector length of the output vector of each node of the ClassVN layer 251 is calculated from the output vector of that output vector, and the vector length of each node is further normalized by a softmax function. As described above, the activation value a obtained by the above formula (4) is j is the output vector M L+1 j The activation value a at each node in the ClassVN layer 251 is a normalized value that corresponds to the vector length of j may be output and used as the judgment value for each class.
[0086] In the above-described embodiment, a vector neural network that obtains an output vector by the calculation of the above formulas (2) to (5) is used as the machine learning models 201 and 202. However, instead of this, a capsule network disclosed in U.S. Patent No. 5,210,798 or International Publication WO 2019 / 083553 may be used. Also, a neural network that uses only scalar neurons may be used.
[0087] The method of generating the known feature spectrum groups KS1 and KS2 and the method of generating output data of intermediate layers such as the ConvVN1 layer are not limited to the above embodiment, and for example, these data may be generated using the Kmeans method. Also, these data may be generated using transformations such as PCA, ICA, Fisher, etc. Also, the transformation methods of the known feature spectrum group KSG and the output data of the intermediate layers may be different.
[0088] Other embodiments: The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various forms without departing from the spirit of the present disclosure. For example, the present disclosure can be realized in the following aspects. The technical features in the above-mentioned embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined in order to solve some or all of the problems of the present disclosure, or to achieve some or all of the effects of the present disclosure. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted.
[0089] (1) According to a first aspect of the present disclosure, there is provided a method for performing a discrimination process for a print medium using a machine learning model, the method including: (a) preparing N machine learning models, where N is an integer equal to or greater than 1, each of the N machine learning models configured to discriminate the type of the print medium by classifying input spectral data, which is the spectral reflectance of a print medium, into one of a plurality of classes; (b) acquiring target spectral data, which is the spectral reflectance of a target print medium; and (c) discriminating the type of the target print medium by performing a classifying process for the target spectral data using the N machine learning models. According to this method, a machine learning model is used to perform class classification processing, so that printing media with similar optical characteristics can be accurately distinguished.
[0090] (2) In the above method, step (c) may include a step of determining a medium identifier indicating the type of the target print medium based on the result of class classification processing of the target spectral data, and the method may further include a step of selecting print settings for performing printing using the target print medium based on the medium identifier, and a step of performing printing using the target print medium in accordance with the print settings. According to this method, print settings are selected based on the results of the process of determining the target print medium, so that a beautiful printout can be produced using the target print medium.
[0091] (3) In the above method, N may be an integer greater than or equal to 2, and each of the N machine learning models may be configured to have at least one class that is different from other machine learning models among the N machine learning models. According to this method, class classification processing is performed using two or more machine learning models, so the processing can be performed faster than when classification processing into multiple classes is performed using a single machine learning model.
[0092] (4) In the above method, the N machine learning models may be trained using corresponding N sets of training data, and the N sets of spectroscopic data constituting the N sets of training data may be in a state equivalent to being grouped into N sets by a clustering process. According to this method, the spectroscopic data used for training each machine learning model is grouped by clustering processing, thereby improving the accuracy of class classification processing by the machine learning model.
[0093] (5) In the above method, each training data group has a representative point that represents the center of the spectroscopic data group that constitutes each training data group, and an upper limit is set for the number of classes that can be classified by any one machine learning model, and the multiple types of printing media that are the subject of the class classification process by the N machine learning models may be classified into either required printing media that cannot be excluded from the class classification process without a user's instruction to exclude them, or optional printing media that can be excluded from the class classification process even without a user's instruction to exclude them. The step (a) may include a medium adding step of making a new additional printing medium that has not been the subject of the class classification process by the N machine learning models the subject of the class classification process, and the medium adding step may include: (a1) a step of acquiring the spectral reflectance of the additional printing medium as additional spectral data; (a2) a step of selecting a teacher data group having a representative point closest to the additional spectral data from the N teacher data groups as a proximal teacher data group, and selecting a specific machine learning model that has been trained using the proximal teacher data group; and (a3) a step of updating the proximal teacher data group by adding the additional spectral data to the proximal teacher data group when the number of classes corresponding to the required printing medium in the specific machine learning model is less than the upper limit value, and re-training the specific machine learning model using the updated proximal teacher data group. According to this method, it is possible to perform class classification corresponding to the additional print medium. In addition, since the additional spectroscopic data is added to the proximate teacher data group having the center of gravity closest to the additional spectroscopic data of the additional print medium and then the machine learning model is re-learned, it is possible to improve the accuracy of the class classification process by the machine learning model.
[0094] (6) In the above method, step (a3) may include a step of deleting any spectral data relating to the optional printing medium from the group of nearby teacher data if the sum of the number of classes corresponding to the required printing medium and the number of classes corresponding to the optional printing medium in the specific machine learning model at a time before step (a3) is executed is equal to the upper limit value. According to this method, any spectroscopic data of any medium is deleted from the group of nearby teacher data, so that the accuracy of the class classification process can be improved without increasing the number of classes in the machine learning model.
[0095] (7) In the above method, the medium adding step may further include (a4) a step of creating a new machine learning model when the number of classes corresponding to the required printing media in the specific machine learning model is equal to the upper limit value, and performing learning of the new machine learning model using a new group of training data including the additional spectral data and arbitrary spectral data relating to one or more optional printing media. According to this method, it is possible to perform classification corresponding to the additional print medium. In addition, since the machine learning model is trained using a new teacher data group including the additional spectroscopic data and the arbitrary spectroscopic data, the accuracy of the classification process can be improved.
[0096] (8) The above method may further include a medium exclusion step of excluding one excluded print medium from the target of the class classification process by one target machine learning model selected from the N machine learning models, and the medium exclusion step may include: (i) a step of updating the teacher data group by deleting spectral data relating to the excluded print medium from the teacher data group used for training the target machine learning model; and (ii) a step of re-training the target machine learning model using the updated teacher data group. This method allows print media to be excluded from the classification process of the machine learning model.
[0097] (9) In the above method, step (i) may involve, when the number of classes held by the target machine learning model becomes less than a predetermined lower limit value by excluding the excluded target printing medium from the training data group used to train the target machine learning model, deleting spectral data relating to the excluded target printing medium from the training data group used to train the target machine learning model, and updating the training data group by adding arbitrary spectral data relating to one or more arbitrary printing media. According to this method, the number of classes in the machine learning model can be set to be equal to or greater than the lower limit, thereby preventing an excessive decrease in the accuracy of the class classification process.
[0098] (10) In the above method, one group of teacher data used for training each machine learning model, the spectroscopic data removed from the teacher data group, and the spectroscopic data added to the teacher data group may be managed as constituting the same spectroscopic data group, the spectroscopic data removed from the teacher data group may be stored in a shelter area of the spectroscopic data group, and the spectroscopic data added to the teacher data group may be selected from the spectroscopic data that was stored in the shelter area of the spectroscopic data group. According to this method, the spectroscopic data used as training data is managed as spectroscopic group data, so that the N spectroscopic data groups constituting the N training data groups can be maintained in a state equivalent to the state where they are grouped by clustering processing.
[0099] (11) According to a second aspect of the present disclosure, there is provided a system for executing a discrimination process for a print medium using a machine learning model. The system includes a memory for storing N machine learning models, where N is an integer equal to or greater than 1, and a processor for executing the discrimination process using the N machine learning models. Each of the N machine learning models is configured to discriminate the type of the print medium by classifying input spectral data, which is the spectral reflectance of the print medium, into one of a plurality of classes. The processor is configured to execute a first process for acquiring target spectral data of a target print medium, and a second process for discriminating the type of the target print medium by executing a classifying process for the target spectral data using the N machine learning models. According to this system, a machine learning model is used to perform class classification processing, making it possible to accurately distinguish between print media with similar optical characteristics.
[0100] The present disclosure may be realized in various forms other than those described above, such as a computer program for implementing the functions of a classification device, a non-transitory storage medium on which the computer program is recorded, and the like. [Explanation of symbols]
[0101] 10... printer, 20... information processing device, 30... spectrometer, 110... processor, 112... print processing unit, 114... class classification processing unit, 116... print setting creation unit, 120... memory, 130... interface circuit, 150... display unit, 201, 202... machine learning model, 211, 212... convolution layer, 221, 222... primary vector neuron layer, 231, 232... first convolution vector neuron layer, 241, 242... second convolution vector neuron layer, 251, 252... classification vector neuron layer, 261, 262... similarity calculation unit
Claims
1. 1. A method for performing a print medium classification process using a machine learning model, comprising: (a) preparing N machine learning models, where N is an integer equal to or greater than 1, each of the N machine learning models being configured to classify input spectral data, which is the spectral reflectance of a print medium, into one of a plurality of classes to identify the type of the print medium; (b) acquiring target spectral data, which is the spectral reflectance of a target print medium; (c) using the N machine learning models to perform a class classification process on the target spectral data to identify the type of the target print medium; Including, The N is an integer of 2 or more, 13. The method of claim 12, wherein each of the N machine learning models is configured to have at least one class that is different from other ones of the N machine learning models.
2. 2. The method of claim 1 , The step (c) determining a medium identifier indicating a type of the target print medium according to a result of classifying the target spectral data; Including, The method further comprises: selecting print settings for performing printing using the target print medium in response to the medium identifier; performing printing using the target print medium in accordance with the print settings; A method comprising:
3. 3. The method according to claim 1 or 2, The N machine learning models are trained using corresponding N sets of teacher data, The N spectroscopic data groups constituting the N teacher data groups are in a state equivalent to a state where the N spectroscopic data groups are grouped into N sets by a clustering process.
4. 1. A method for performing a print medium classification process using a machine learning model, comprising: (a) preparing N machine learning models trained using N sets of training data, where N is an integer equal to or greater than 1, each of the N machine learning models being configured to classify input spectral data, which is the spectral reflectance of a printing medium, into one of a plurality of classes to identify the type of the printing medium; (b) acquiring target spectral data, which is the spectral reflectance of a target print medium; (c) using the N machine learning models to perform a class classification process on the target spectral data to identify the type of the target print medium; Including, Each teacher data group has a representative point that represents the center of the spectroscopic data group that constitutes each teacher data group, There is an upper limit on the number of classes that can be classified by any one machine learning model. The multiple types of print media that are the targets of the class classification process by the N machine learning models are classified into either required print media that cannot be excluded from the target of the class classification process without a user's exclusion instruction, or optional print media that can be excluded from the target of the class classification process without a user's exclusion instruction, The step (a) includes a medium adding step of subjecting a newly added print medium that has not been subject to the class classification process by the N machine learning models to the class classification process, The medium adding step includes: (a1) acquiring a spectral reflectance of the additional printing medium as additional spectral data; (a2) selecting, from the N teacher data groups, a teacher data group having a representative point closest to the additional spectroscopic data as a proximate teacher data group, and selecting a specific machine learning model that has been trained using the proximate teacher data group; (a3) if the number of classes corresponding to the required printing medium in the specific machine learning model is less than the upper limit value, adding the additional spectral data to the proximal teacher data group to update the proximal teacher data group, and re-learning the specific machine learning model using the updated proximal teacher data group; A method comprising:
5. 5. The method of claim 4, The step (a3) a step of deleting the arbitrary spectral data relating to the arbitrary printing medium from the group of nearby teacher data when the sum of the number of classes corresponding to the required printing medium and the number of classes corresponding to the arbitrary printing medium in the specific machine learning model is equal to the upper limit value before the execution of the step (a3); A method comprising:
6. 6. The method according to claim 4 or 5, The medium adding step further includes: (a4) creating a new machine learning model when the number of classes corresponding to the required printing media in the specific machine learning model is equal to the upper limit value, and executing learning of the new machine learning model using a new training data group including the additional spectral data and arbitrary spectral data relating to one or more arbitrary printing media; A method comprising:
7. 1. A method for performing a print medium classification process using a machine learning model, comprising: (a) preparing N machine learning models, where N is an integer equal to or greater than 1, each of the N machine learning models being configured to classify input spectral data, which is the spectral reflectance of a print medium, into one of a plurality of classes to identify the type of the print medium; (b) acquiring target spectral data, which is the spectral reflectance of a target print medium; (c) using the N machine learning models to perform a class classification process on the target spectral data to identify the type of the target print medium; Including, The N is an integer of 2 or more, and the method further comprises: A medium excluding step of excluding one excluded target print medium from the target of the class classification process by one target machine learning model selected from the N machine learning models, The media removal step includes: (i) updating the training data group by deleting the spectral data related to the excluded printing medium from the training data group used to train the target machine learning model; (ii) performing re-learning of the target machine learning model using the updated teacher data group; A method comprising:
8. 8. The method of claim 7, The method includes, in step (i), if the number of classes held by the target machine learning model becomes less than a predetermined lower limit value due to the exclusion of the excluded target printing medium from the training data group used to train the target machine learning model, deleting spectral data relating to the excluded target printing medium from the training data group used to train the target machine learning model, and updating the training data group by adding arbitrary spectral data relating to one or more arbitrary printing media.
9. 9. The method according to claim 7 or 8, A group of teacher data used for learning each machine learning model, the spectroscopic data removed from the group of teacher data, and the spectroscopic data added to the group of teacher data are managed as constituting the same spectroscopic data group; The spectroscopic data removed from the teacher data group is stored in a storage area of the spectroscopic data group, The spectroscopic data to be added to the teacher data group is selected from the spectroscopic data that has been sheltered in the shelter area of the spectroscopic data group.
10. A system for performing a print medium discrimination process using a machine learning model, A memory that stores N machine learning models, where N is an integer equal to or greater than 1; A processor that executes the discrimination process using the N machine learning models; Equipped with Each of the N machine learning models is configured to classify input spectral data, which is a spectral reflectance of a print medium, into one of a plurality of classes to identify the type of the print medium; The processor, A first process for acquiring target spectral data of a target print medium; a second process of determining a type of the target print medium by performing a class classification process on the target spectral data using the N machine learning models; is configured to run The N is an integer of 2 or more, wherein each of the N machine learning models is configured to have at least one class that is different from other ones of the N machine learning models.
11. A system for performing a print medium discrimination process using a machine learning model, A memory that stores N machine learning models trained using N sets of teacher data, where N is an integer equal to or greater than 1; A processor that executes the discrimination process using the N machine learning models; Equipped with Each of the N machine learning models is configured to classify input spectral data, which is a spectral reflectance of a print medium, into one of a plurality of classes to identify the type of the print medium; The processor, A first process for acquiring target spectral data of a target print medium; a second process of determining a type of the target print medium by performing a class classification process on the target spectral data using the N machine learning models; is configured to run Each teacher data group has a representative point that represents the center of the spectroscopic data group that constitutes each teacher data group, There is an upper limit on the number of classes that can be classified by any one machine learning model. The multiple types of print media that are the targets of the class classification process by the N machine learning models are classified into either required print media that cannot be excluded from the target of the class classification process without a user's exclusion instruction, or optional print media that can be excluded from the target of the class classification process without a user's exclusion instruction, The processor is further configured to execute a medium addition process for subjecting a newly added print medium that has not been subject to the class classification process by the N machine learning models to the class classification process; The medium addition process includes: (a1) acquiring a spectral reflectance of the additional printing medium as additional spectral data; (a2) selecting, from the N teacher data groups, a teacher data group having a representative point closest to the additional spectroscopic data as a proximate teacher data group, and selecting a specific machine learning model that has been trained using the proximate teacher data group; (a3) if the number of classes corresponding to the required printing medium in the specific machine learning model is less than the upper limit value, a process of adding the additional spectral data to the proximal teacher data group to update the proximal teacher data group, and re-learning the specific machine learning model using the updated proximal teacher data group; Including, the system.
12. A system for performing a print medium discrimination process using a machine learning model, A memory that stores N machine learning models, where N is an integer equal to or greater than 1; A processor that executes the discrimination process using the N machine learning models; Equipped with Each of the N machine learning models is configured to classify input spectral data, which is a spectral reflectance of a print medium, into one of a plurality of classes to identify the type of the print medium; The processor, A first process for acquiring target spectral data of a target print medium; a second process of determining a type of the target print medium by performing a class classification process on the target spectral data using the N machine learning models; is configured to run The N is an integer equal to or greater than 2, and the processor further The method is configured to be capable of executing a medium exclusion process of excluding one exclusion target print medium from the targets of the class classification process by one target machine learning model selected from the N machine learning models, The media exclusion process includes: (i) updating the teacher data group by deleting the spectral data related to the excluded printing medium from the teacher data group used to train the target machine learning model; (ii) a process of re-learning the target machine learning model using the updated teacher data group; Including, the system.
Citation Information
Patent Citations
Image formation apparatus
JP2019055554A
Printer, machine learning device, machine learning method and printing control program
JP2020121503A
