Information processing device, method for controlling the information processing device, and program

Abstract

To estimate the type of recording medium with high accuracy. [Solution] The information processing device includes an acquisition means for acquiring multiple characteristic values ​​of a recording medium by measuring the recording medium being fed; a registration means for registering the types of recording media to be estimated; an estimation means for estimating the type of recording medium being fed using an estimation unit from among multiple types of recording media registered in the registration means, based on a first characteristic value which is selected from among the acquired multiple characteristic values ​​based on selection information that defines the characteristic value to be used; a modification means which can change the selection information; and an output means which outputs an instruction to update the estimation unit using the first characteristic value indicated by the selection information.

Description

[Technical Field]

[0001] This disclosure relates to a technology for estimating the type of recording medium. [Background technology]

[0002] In the commercial and industrial printing market, the uses of output materials are diverse, including CAD line drawings, posters, artwork, and signage. Therefore, a wide variety of recording media with different characteristics are used to suit these applications. As the types of recording media increase, the process of selecting the appropriate recording medium for the recording device becomes more cumbersome. Recent recording devices are increasingly incorporating features that improve usability by automatically estimating the type of recording medium fed into the device. However, because this automatic estimation function relies on predetermined recording medium information, it cannot estimate unknown printing media.

[0003] Patent Document 1 discloses a technique for estimating the type of recording medium using a pre-trained model that has been trained with spectral information of the unprinted area of ​​the printing medium and an identifier indicating the type of printing medium. In Patent Document 1, if an unknown recording medium is estimated, the pre-trained model is updated by retraining it with spectral information of the unknown printing medium and an identifier indicating the type of printing medium, thereby expanding the types of printing media that can be estimated. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-078426 [Overview of the project] [Problems that the invention aims to solve]

[0005] Patent Document 1 describes a configuration that uses specific types of features of the recording medium for learning. Therefore, it is possible to distinguish between recording mediums that have different values ​​for those specific types of features. However, when the recording medium to be distinguished has similar values ​​for those specific types of features, even if retraining is performed using those specific types of features, the detection accuracy cannot be sufficiently improved.

[0006] This disclosure aims to estimate the type of recording medium with high accuracy. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present disclosure is characterized by comprising: an acquisition means for acquiring a plurality of characteristic values ​​of a recording medium by measuring the recording medium being fed; a registration means for registering the types of recording media to be estimated; an estimation means for estimating the type of recording medium being fed using an estimation unit from among a plurality of recording media types registered in the registration means, based on a first characteristic value which is a characteristic value selected from the plurality of acquired characteristic values ​​based on selection information which defines the characteristic value to be used; a modification means which can change the selection information; and an output means which outputs an instruction to update the estimation unit using the first characteristic value indicated by the selection information. [Effects of the Invention]

[0008] According to this disclosure, the type of recording medium can be estimated with high accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] This is a perspective view showing an example of a recording device. [Figure 2] This is a cross-sectional view showing an example of the main parts of a recording device. [Figure 3] This block diagram shows an example of the configuration of a control system for a recording device. [Figure 4] This is a cross-sectional view showing an example of the area around a media sensor. [Figure 5]It is a flowchart showing an example of the estimation process of the sheet type. [Figure 6] It is a diagram showing an example of the characteristics of the sheet. [Figure 7] It is a block diagram showing a method for selecting feature quantities to be input to the learned model. [Figure 8] It is a diagram showing an example of flag information for selecting feature quantities. [Figure 9] It is a diagram showing an example of the estimation table of the sheet type. [Figure 10] It is a diagram showing an example of the estimation result screen. [Figure 11] It is a schematic diagram of the DNN. [Figure 12] It is a flowchart of FilterMethod. [Figure 13] It is a diagram showing an example of the correlation coefficient and the result of weighting. [Figure 14] It is a flowchart of the variable reduction method of WrapprerMethod. [Figure 15] It is a diagram showing the result of calculating the coefficient of determination. [Figure 16] It is a diagram showing an example of the update process of the learned model. [Figure 17] It is a diagram showing an example of the machine learning device.

Mode for Carrying Out the Invention

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the matters disclosed, and not all combinations of the features described in the following embodiments are essential for the solution means of the present disclosure. The same components are denoted by the same reference numerals. Also, the relative arrangement, shape, etc. of the components described in the embodiments are merely examples, and are not intended to limit the scope of this disclosure only to them.

[0011] In the following description of the embodiments, "recording" includes not only cases where meaningful information such as characters and figures is formed, but also cases where images, patterns, and other designs are formed on the sheet. Furthermore, although a roll sheet is assumed as the sheet in this embodiment, it may be cut paper, cloth, or plastic film, etc. In addition, "ink" should be interpreted broadly and refers to a liquid that can be applied to the sheet to form images, patterns, or other designs, or to process the sheet, or to treat the ink.

[0012] <<First Embodiment>> <Inkjet recording device> Figure 1 is a perspective view showing an inkjet recording device, which is an example of a recording device 101 for performing industrial and commercial printing in this embodiment. Figure 2 is a cross-sectional view showing an example of the main parts of the recording device 101. Figure 3 is a block diagram showing an example of the control configuration of the recording device 101. The configuration of the recording device 101 will be described below using Figures 1 to 3.

[0013] In this embodiment, processing is performed using a pre-trained model, as will be described later. In this embodiment, it is assumed that processing using the pre-trained model is performed in the recording device 101. That is, the recording device 101 is an information processing device that uses a pre-trained model. However, the information processing device that uses a pre-trained model is not limited to the recording device 101. A server (for example, a cloud server) that can exchange various types of data with the recording device 101 may be used as the information processing device that uses a pre-trained model.

[0014] Hereinafter, the transport direction in which the sheet S is transported in the recording device 101 will be defined as the +Y direction. The direction in which the recording head 204 ejects ink onto the sheet S will be defined as the -Z direction. The direction in which the recording head 204 moves from its standby position will be defined as the +X direction.

[0015] The recording device 101 rotatably holds a roll sheet R in which a sheet S is wound into a roll. By rotating the roll sheet R with a roll drive motor 308, the sheet S is supplied from the roll sheet R to the transport roller 203. The transport roller 203 can rotate while gripping the sheet S. By rotating the transport roller 203 with a transport roller drive motor 309, the sheet S is transported to a position where the recording head 204 can record on the sheet S. The recording head 204 is mounted on a carriage (not shown) and is configured to reciprocate in the X direction. While moving the recording head 204 in the X direction, an image is recorded on the sheet S by ejecting ink from the recording head 204 onto the sheet S that has been transported from the recording head 204. The sheet S on which the image has been recorded is discharged from a discharge section located downstream of the recording head 204 in the transport direction and loaded into a basket 103.

[0016] The control panel 102 is an interface module that accepts various operations from the user. The user can make various settings for the recording device 101 using the various switches or touch panel provided on the control panel 102. These settings for the recording device 101 include, for example, the size and type of sheet S. The control panel 102 also displays estimation results, which will be described later.

[0017] In the transport direction, a sheet detection sensor 202 is positioned upstream of the transport roller 203. When the sheet detection sensor 202 detects that a sheet S has been supplied from the roll sheet R by the user, the transport operation of the sheet S begins. The transport operation of the sheet S is performed by synchronously driving the roll drive motor 308 and the transport roller drive motor 309. At this time, the recording device 101 can estimate the type of sheet S by estimating the sheet type, which will be described later. Details will be described later.

[0018] In the transport direction, a media sensor 206 and an ultrasonic transmitter 207 are positioned upstream of the sheet detection sensor 202. The media sensor 206 is positioned above the sheet S in the direction of gravity (Z direction), and the ultrasonic transmitter 207 is positioned below the sheet S in the direction of gravity. The media sensor 206 and ultrasonic transmitter 207 are used for estimating the sheet type, which will be described later.

[0019] The recording of an image onto the sheet S is performed as follows: First, the recording device 101 performs a transport operation to transport the sheet S to a position opposite the recording head 204. Next, while ejecting ink, the recording head 204 is scanned in a direction that intersects (orthogonal to) the transport direction of the sheet S, thereby recording an image of the area of ​​the sheet S corresponding to the recording head 204. Next, after transporting a predetermined amount of the sheet S, the recording head 204 is scanned in the intersecting direction while ejecting ink. In this way, by alternately performing the transport operation of the sheet S and the image recording operation, the desired image is recorded on the sheet S. The sheet S on which the image has been recorded is sequentially transported downstream of the recording head 204 in the transport direction. The transported sheet S is cut by a cutter 205 provided in the discharge section. The cut sheet S is loaded into the basket 103.

[0020] Figure 3 is a block diagram showing an example of the control system configuration in the recording device 101. The recording device 101 includes an operation panel 102, a recording head 204, a CPU 301, a sensor control unit 302, and an input / output interface (IF) 303. The recording device 101 also includes a USB port 304, a memory 305, a motor control unit 306, and RAM 320. Furthermore, the recording device 101 includes a sheet detection sensor 202, a media sensor 206, an ultrasonic transmitter 207, and a carriage encoder 307. Additionally, the recording device 101 includes a roll drive motor 308, a transport roller drive motor 309, a carriage drive motor 310, a lift drive motor 311, a cutter drive motor 312, and a media sensor lifting motor 313. The memory 305 contains a program 351 and a trained model 352. The trained model 352 includes a trained model for coarse classification, a trained model for first detailed classification, and a trained model for second detailed classification, which will be described later. In this embodiment, when predetermined data is input to the trained model 352, a predetermined estimation result is output from the trained model 352. That is, the trained model 352 is an estimation unit that performs estimation.

[0021] The motor control unit 306 controls each drive motor according to the program 351 stored in the memory 305. The roll drive motor 308 rotates the spool 201 to transport the sheet S from the roll sheet R in the transport direction. The transport roller drive motor 309 rotates the transport roller 203 to transport the sheet S to a position facing the recording head 204. The transport roller drive motor 309 is equipped with an encoder that detects the amount of rotation to detect the amount of sheet S transported. By measuring the amount of rotation of the encoder, the amount of sheet S transported can be detected. The carriage drive motor 310 rotates a carriage belt (not shown) to move a carriage (not shown) and the recording head 204 mounted on the carriage. The lift drive motor 311 moves the carriage and the recording head 204 up and down. The cutter drive motor 312 drives the cutter. The media sensor lifting motor 313 lifts and lowers the media sensor 206.

[0022] The CPU 301 receives various setting information via the input / output IF 303 through user operations from the control panel 102 or a PC connected to the USB port 304 or a LAN port (not shown). The input information is stored in the memory 305. The CPU 301 can read the information stored in the memory 305 as needed and perform various processes on the read information. In other words, the CPU 301 includes a processing unit that executes various processes.

[0023] The CPU 301 controls the carriage encoder 307, sheet detection sensor 202, media sensor 206, and ultrasonic transmitter 207 via the sensor control unit 302, and obtains output data from each unit. The CPU 301 performs various controls based on the inputs from the carriage encoder 307, sheet detection sensor 202, and media sensor 206. The CPU 301 controls the carriage encoder 307, sheet detection sensor 202, media sensor 206, and ultrasonic transmitter 207 via the sensor control unit 302, and obtains information. The CPU 301 also performs various controls based on the inputs from the carriage encoder 307, sheet detection sensor 202, and media sensor 206. The RAM 320 is used as a temporary work area.

[0024] This embodiment describes a process for estimating the type of sheet using a pre-trained machine learning model.

[0025] <Estimation of sheet type> The operation for estimating the type of sheet S in this embodiment will be explained using Figures 4 to 9. Figure 4 is a cross-sectional view showing an example of the area around the media sensor 206. Figure 5 is a flowchart showing an example of the process for estimating the type of sheet S from the paper feed. Figure 6 is a diagram showing an example of data in which the characteristics of sheet S have been detected. Figure 6 is an example of data in which the characteristics of sheet S have been detected using the media sensor 206 and the ultrasonic transmitter 207, respectively. Figure 7 is a block diagram showing a method for selecting features to be input to the trained model. Figure 8 is a diagram showing an example of flag information for selecting features. Figure 9 is a diagram showing an example of the estimation table for the type of sheet S in this embodiment. The estimation table in Figure 9 is a table in which the types of sheet S corresponding to the output value y of the trained model, which will be described later, are registered. In this embodiment, the types of sheet S to be estimated in the current state are assumed to be nine types, from recording medium 1 to recording medium 9. That is, the trained model 352 that estimates the type of sheet S is configured to estimate the recording medium that is most suitable for the input features from among recording mediums 1 to recording medium 9 as the type of sheet S. The trained model 352 is described as being stored in memory 305, but it may also be located outside the recording device 101, and the CPU 301 of the recording device 101 may use the trained model located outside of it. As will be described later, this embodiment will also describe the process of updating the trained model when a new type of sheet S is added, but first, we will describe the process of estimating the sheet type using the currently available trained model.

[0026] The process shown in the flowchart in Figure 5 is realized when the CPU 301 of the recording device 101 reads the program 351 stored in the memory 305, etc., into the RAM 320 and executes it. Note that some or all of the functions of the steps in Figure 5 may be realized by hardware such as an ASIC or electronic circuit. The symbol "S" in the description of each process means that it is a step in the flowchart (the same applies hereafter in the flowcharts of this specification). The process shown in Figure 5 is started, for example, when the user sets the roll sheet S in the recording device 101. Alternatively, it may be started when the system detects that a predetermined operation has been entered by the user on the operation panel 102 after the user has set the roll sheet S in the recording device 101.

[0027] In S501, the sheet S is fed. Specifically, the CPU 301 detects that the user has set the roll sheet R in the recording device 101. The CPU 301 then rotates the roll sheet R with the roll drive motor 308. This supplies the sheet S from the roll sheet R to the transport roller 203. The sheet detection sensor 202, located upstream of the transport roller 203, then detects that the sheet S has reached the transport roller 203. When the sheet detection sensor 202 detects that the sheet S has reached the transport roller 203, the CPU 301 stops driving the roll drive motor 308. At the position where the sheet S is detected by the sheet detection sensor 202, the sheet S is being transported to a position opposite the media sensor 206 and the ultrasonic transmitter 207. After that, the CPU 301 proceeds to processing S502.

[0028] In step S502, the CPU 301 performs sensing. Specifically, the CPU 301 measures the characteristics of the sheet S by controlling the media sensor 206 and the ultrasonic transmitter 207 via the sensor control unit 302. As shown in Figure 4, the media sensor 206 has a CIS (Contact Image Sensor) 401 and a microphone 402. A roller 403 is positioned opposite the CIS 401. The ultrasonic transmitter 207 is positioned opposite the microphone 402. The CPU 301 lowers the media sensor 206, which is separated from the sheet S, using the media sensor lifting motor 313, so that the sheet S can be gripped using the CIS 401 and the roller 403. By gripping the sheet S, the characteristics of the sheet S can be measured stably. With the sheet S gripped by the CIS 401 and the roller 403, the CPU 301 transports the sheet S again and reads the surface image of the sheet S using the CIS 401. Then, when sensing is complete, the CPU 301 moves the media sensor 206 away from the seat S by raising it using the media sensor lifting motor 313.

[0029] Examples of surface images of sheet S acquired using CIS401 are shown in Figures 6(a) and (b). Examples of electrical signals of ultrasound transmitted through sheet S, acquired using ultrasonic transmitter 207 and microphone 402, are shown in Figures 6(c) and (d).

[0030] CIS401 is a line sensor extending in the width direction of the sheet S, and acquires one-dimensional (one row) image data. The CPU301, with the sheet S held between CIS401 and roller 403, acquires image data of the sheet S using CIS401 while synchronously driving the roll drive motor 308 and the transport roller drive motor 309. By reading the image of the sheet S with CIS401 while transporting the sheet S in this way, two-dimensional image data as shown in Figures 6(a) and (b) can be acquired. Figure 6(a) is an example of a surface image of Japanese paper, and Figure 6(b) is an example of a surface image of synthetic paper. In Figures 6(a) and (b), the CIS direction corresponds to the width of CIS401 (the width in the X direction intersecting the sheet S), and the transport direction corresponds to the amount of sheet S transported measured by CIS401. Although an example using a one-dimensional sensor as CIS401 is shown here, a two-dimensional sensor may also be used to measure the surface image of the sheet S. Furthermore, while measuring the surface image with CIS401, ultrasonic electrical signals shown in Figures 6(c) and 6(d) are obtained by the ultrasonic transmitter 207 and microphone 402 (sound-receiving sensor). Figure 6(c) is an example of an ultrasonic electrical signal transmitted through Japanese paper, and Figure 6(d) is an example of an ultrasonic electrical signal transmitted through synthetic paper. In this embodiment, an example of obtaining an ultrasonic electrical signal in conjunction with measuring the surface image with CIS401 is described, but it is not limited to this. The measurement of the surface image and the acquisition of the ultrasonic electrical signal may be performed separately. Also, the ultrasonic electrical signal may be acquired without transporting the sheet S. After that, the CPU 301 proceeds to processing S503.

[0031] In S503, the CPU 301 derives characteristic values ​​that indicate the characteristics of the recording medium. In this embodiment, an example of deriving feature quantities of the recording medium as characteristic values ​​will be used for explanation. That is, the CPU 301 uses a characteristic value derivation method stored in memory 305 in advance to derive characteristic values ​​(feature quantities) related to the surface information of the sheet S and characteristic values ​​(feature quantities) related to the cross-sectional information of the sheet S from the characteristics of the sheet S measured in S502 described above. In other words, the CPU 301 derives feature quantities related to the surface information of the sheet S and feature quantities related to the cross-sectional information of the sheet S from the surface image of the sheet S and the ultrasonic electrical signal.

[0032] In this embodiment, the number of features derived in S503 is not limited. For example, in this embodiment, a total of 15 types of features are obtained in S503, consisting of 10 types of features (1 to 10) derived from the surface information of the sheet S and 5 types of features (11 to 15) derived from the cross-sectional information. That is, a total of 15 types of features, from the 1st feature to the 15th feature, are derived. Hereafter, the 1st feature will be referred to as feature 1, the 2nd feature as feature 2, and so on.

[0033] The CPU 301 derives several feature quantities 1 to 10 related to the surface information of sheet S from the surface image of sheet S as shown in Figures 6(a) and (b). For example, feature quantity 1 is brightness. Brightness is derived as the average value of all pixel values. Feature quantity 2 is the unevenness in the CIS direction. Unevenness in the CIS direction is derived as the average of the absolute values ​​of the differences between adjacent pixel values ​​in the CIS direction. Feature quantity 3 is the unevenness in the transport direction. Unevenness in the transport direction is derived as the average of the absolute values ​​of the differences between adjacent pixel values ​​in the transport direction. Feature quantity 4 is the standard deviation of all pixel values. Feature quantities 5 to 10 are extracted features obtained by texture analysis using GLCM (Gray-Level Co-Occurrence Matrix) of specific pixels. GLCM is a matrix of the frequencies of pixel values ​​of surrounding pixels relative to the pixel value of a specific pixel. Feature quantities 5 to 10 correspond to features calculated by performing various calculations from the calculated GLCM. For example, feature 5 is the calculated value of ASM (Angular Second Moment). ASM is a feature that takes the sum of the squares of the GLCM components and represents the uniformity of the image. Feature 6 is the calculated value of Entropy. Entropy represents the randomness of the luminance values ​​of the GLCM. Feature 7 is the calculated value of Dissimilarity. Dissimilarity represents the absolute value of the difference between the central pixel value and the surrounding pixel value of the GLCM and represents the uniformity of the image. Feature 8 is the calculated value of Homogeneity. Homogeneity, like Dissimilarity, represents the non-uniformity of the image. Feature 9 is the calculated value of Contrast. Contrast indicates the intensity between the central pixel and its surrounding pixels of the GLCM. Feature 10 is the calculated value of Maximum Probability. Maximum Probability is the maximum value of the GLCM. The above are examples of multiple features related to the surface information of sheet S.

[0034] Next, we will explain several features related to the cross-sectional information of sheet S. The CPU 301 derives several features 11 to 15 related to the cross-sectional information of sheet S from the electrical signals of ultrasound transmitted through sheet S, as shown in Figures 6(c) and (d). For example, feature 11 is peak 1. Peak 1 is derived as the maximum voltage value during the period from time t1 to time t2. Feature 12 is peak 2. Peak 2 is derived as the maximum voltage value during the period from time t2 to time t3. Feature 13 is peak 3. Peak 3 is derived as the maximum voltage value during the period from time t3 to time t4. Although described here as the maximum voltage value, it may also be described as the minimum voltage value. Feature 14 is the value obtained by calculating the ratio between the peak value of feature 11 and the peak value of feature 12. Feature 15 is the value obtained by calculating the ratio between the peak value of feature 12 and the peak value of feature 13. The cross-sectional information of sheet S corresponds to information such as the sheet's thickness and basis weight.

[0035] Thus, in S503, the CPU 301 can derive 15 feature quantities relating to the surface information and cross-sectional information of sheet S from the characteristics of sheet S measured in S502.

[0036] Let's explain an example of the trends in specific features. In this embodiment, we will explain using Japanese paper (washi) and synthetic paper as examples. Below is an example of the relationship between the features of the two. Brightness is higher for synthetic paper than for Japanese paper. The unevenness in the CIS direction is greater for Japanese paper than for synthetic paper. The unevenness in the transport direction is greater for Japanese paper than for synthetic paper. For example, a sheet S with a white color and a flat surface will have higher brightness. Also, a sheet with greater surface unevenness will have greater unevenness in both the CIS direction and the transport direction. Note that depending on the type of sheet, there may be either a vertical or horizontal grain, and some sheets will have greater unevenness in only one direction. The peak value of the ultrasonic electrical signal decreases as the sheet thickness increases. Japanese paper is thicker than synthetic paper. Therefore, the peak value is higher for synthetic paper than for Japanese paper. Also, even with the same thickness, the peak value changes depending on the cross-section (the material or density that makes up the sheet). Specifically, using a material (medium) that has a high acoustic impedance tends to result in a lower peak value.

[0037] In S504, the CPU 301 selects features. As mentioned above, in this embodiment, 15 features are derived in S503. Rather than using all of these features for the classification process described later (i.e., inference processing using the trained model), the classification process is performed using the types of features selected in S504. If the classification process is performed using all features, the trained model may become bloated and various costs may increase. For this reason, a process is performed to select only some of the derived features, rather than using all of them.

[0038] An example of the feature selection process in this embodiment will be explained using Figures 7 and 8. As mentioned above, in this embodiment, it is possible to acquire a total of 15 types of features related to the surface information and cross-sectional information of the sheet S. In this embodiment, it is possible to select the types of features to be used for each trained model from among the 15 types of features that can be acquired. In this embodiment, the types of features used by the trained model are multiple types of features. More specifically, six types of features are used for each trained model. That is, six types of features are selected for each trained model. At least some of the six types of features selected for each trained model are different from each other.

[0039] Figure 7 is a block diagram showing an example of a configuration for selecting the type of features to be input to a trained model when estimating the type of recording medium, that is, when performing classification processing. The block diagram in Figure 7 is a block diagram showing a part of the configuration of the recording device 101. In the block diagram shown in Figure 7, the recording device 101 includes a feature extraction unit 701, selection flag storage units 702, 703, 704, selection units 705, 706, 707, and a trained model 352. The trained model 352 includes a first trained model 708 for performing the rough classification in S505 described later, a second trained model 709 for performing the first detailed classification in S508 described later, and a third trained model 710 for performing the second detailed classification in S509 described later.

[0040] The selection flag storage units 702, 703, and 704 are implemented by the memory 305. The feature extraction unit 701 and the selection units 705 to 707 are implemented by the CPU 301 of the recording device 101 reading the program 351 stored in the memory 305, etc., into the RAM 320 and executing it, thereby allowing the CPU 301 to function as these respective units.

[0041] As described above, the feature extraction unit 701 performs the process of extracting features in S503. The selection flag storage units 702, 703, and 704 are storage units that store flag information for selecting features to be used for estimation. The selection units 705, 706, and 707 select features based on the flag information in the selection flag storage units 702, 703, and 704. The first trained model 708 is a trained model that uses the features selected by the selection unit 705 as input information. The second trained model 709 is a trained model that uses the features selected by the selection unit 706 as input information. The third trained model 710 is a trained model that uses the features selected by the selection unit 707 as input information.

[0042] Figure 8 illustrates the flag information stored in the selection flag storage units 702, 703, and 704. Figure 8 shows information indicating flags for selecting a specific type of feature from the 15 types of features mentioned above. In this embodiment, the flag information is represented by "0" or "1" for each of the 15 types of features. In this embodiment, the type of feature for which the flag information is "1" is defined as the feature to be used for estimation. The type of feature for which the flag information is "0" is defined as not to be used for estimation. Thus, the flag information is selection information that defines the type of feature to be used.

[0043] For example, in the first pre-trained model, features 2, 3, and 4 (roughness), feature 6 (entropy), feature 7 (dissimilarity), and feature 11 (USS (ultrasonic sensor) peak value 1) are used for estimation. In the second pre-trained model, features 1 (luminance), feature 6 (entropy), feature 8 (homogeneity), feature 10 (maximum probability), feature 11 (USS peak value 1), and feature 12 (USS peak value 2) are used for estimation. In the third pre-trained model, features 1 (luminance), feature 5 (ASM), feature 7 (dissimilarity), feature 8 (homogeneity), feature 9 (contrast), and feature 10 (maximum probability) are used for estimation. Based on this flag information, the features selected by the selection units 705, 706, and 707 are input to the first trained model 708, the second trained model 709, and the third trained model 710, respectively.

[0044] In this way, in S504 of this embodiment, the features to be used for classification processing (inference processing) are selected from the features obtained in S503. The flag information shown in Figure 8 may be predetermined, or it may be changed as appropriate during retraining, which will be described later. In addition, although the example in Figure 8 describes an example in which the number of types of features used in each trained model is 6, the number of types of features used in a trained model may be any number, for example, it may be 1. Furthermore, different numbers of types of features may be used for each trained model.

[0045] In S505, CPU301 performs a coarse classification of the sheet S. Coarse classification is a type of classification that is less detailed than the detailed classification described later. In S505, CPU301 uses the first pre-trained model 708 for coarse classification, which is stored in memory 305, and the estimation table shown in Figure 9 to estimate the type of sheet S from the six features derived in S503 and selected in S504. To estimate the type of sheet S, CPU301 obtains the output value y from the first pre-trained model 708 for coarse classification. The first pre-trained model 708 for coarse classification receives the six features related to sheet S derived in S503 and selected in S504 as input and outputs the probability that the sheet S is of the type to be estimated as an array for each type of sheet S. Each element of the output array is associated with a type of sheet S. That is, the index of each element of the output array is associated with each type of sheet. In this embodiment, the index of the element with the highest probability in the output array is used as the output value y of the first pre-trained model 708 for rough classification. The type of sheet S associated with the output value y then becomes the estimation result.

[0046] As shown in the estimation table in Figure 9, in S505, when the output value y obtained by inputting the features of sheet S into the first pre-trained model 708 for coarse classification is 0, the CPU 301 estimates the type of sheet S to be recording medium 1. Similarly, the CPU 301 estimates the type of sheet S to be recording medium 2 when the output value y is 1, recording medium 3 when the output value y is 2, recording medium 4 when the output value y is 3, and recording medium 5 when the output value y is 4. When the output value y is 5, the estimated type of sheet S is recording medium 6 and recording medium 7. In this case, recording medium 6 and recording medium 7 are collectively referred to as the first recording medium group. That is, when the output value y is 5, the CPU 301 estimates the type of sheet S to be the first recording medium group. Similarly, when the output value y is 6, the estimated type of sheet S is recording medium 8 and recording medium 9. In this case, recording medium 8 and recording medium 9 are collectively referred to as the second recording medium group. That is, when the output value y is 6, the CPU 301 estimates the type of sheet S to be the second recording medium group. After processing in S505, the CPU 301 proceeds to processing in S506.

[0047] Thus, in S505, when the output value y from the first pre-trained model 708 for coarse classification is between 0 and 4, the CPU 301 can uniquely estimate the type of sheet S as recording medium 1 to 5, respectively. On the other hand, when the output value y is 5, the CPU 301 estimates the types of sheet S to be recording medium 6 and recording medium 7. In other words, in this case, the CPU 301 estimates that the type of sheet S is classified into the first recording medium group in coarse classification. Similarly, when the output value y is 6, the CPU 301 estimates that the type of sheet S is classified into the second recording medium group in coarse classification. That is, when the output value y is 5 or 6, the CPU 301 cannot uniquely estimate the type of sheet S and uniquely estimates the type of sheet S as the type of a group of recording media composed of multiple types of recording media. In this embodiment, the feature quantities corresponding to recording media 6 and 7 are close values ​​within a predetermined range. Similarly, the feature quantities corresponding to recording media 8 and 9 are close values ​​within a predetermined range. Therefore, the estimation using the first pre-trained model 708 for rough classification is configured to obtain estimation results that combine multiple recording media.

[0048] In S506, the CPU 301 determines, based on the estimation result of the type of sheet S in S505, whether to perform the first detailed classification in S508 or the second detailed classification in S509, as described later. Specifically, if the estimation result in S505 corresponds to the first recording medium group, the CPU 301 determines to perform the first detailed classification and proceeds to the process in S508. If the estimation result in S505 corresponds to the second recording medium group, the CPU 301 determines to perform the second detailed classification and proceeds to the process in S509. If the estimation result in S505 does not correspond to either the first or second recording medium group, the CPU 301 proceeds to the process in S507. In S507, the CPU 301 displays the estimation result on the operation panel 102.

[0049] In S508, the CPU 301 uses the second pre-trained model 709 for first-level classification, which is stored in memory 305, and the estimation table shown in Figure 9 to estimate the type of sheet S from the six features derived in S503 and selected in S504. In other words, in S508, the CPU 301 uses the features selected for first-level classification from the total of 15 features derived from the six features used in the estimation of coarse classification of the type of sheet S in S505. The output value y obtained in S508 is the output value of a different second pre-trained model than the first pre-trained model used in S505. Here, a different pre-trained model means that it is a model that has been trained using different data as the data used for training. In order to estimate the type of sheet S, the CPU 301 obtains the output value y of the second pre-trained model, similar to S505 described above.

[0050] As shown in the estimation table in Figure 9, in S508, when the output value y obtained by inputting the features of sheet S into the second pre-trained model 709 for first detailed classification is 0, the CPU 301 estimates the type of sheet S to be recording medium 6. Similarly, when the output value y is 1, the CPU 301 estimates the type of sheet S to be recording medium 7. After that, the CPU 301 proceeds to processing in S507. In this way, in S508, the CPU 301 can uniquely estimate the type of sheet S, which was estimated as the first recording medium group in S505, as either recording medium 6 or recording medium 7.

[0051] In S509, the CPU 301 uses the third pre-trained model 710 for second-level classification, which is stored in memory 305, and the estimation table shown in Figure 9, to estimate the type of sheet S from the six features derived in S503 and selected in S504. In other words, in S509, the CPU 301 uses the features selected for second-level classification from the total of 15 features derived from the six features used in the rough classification estimation of the type of sheet S in S505. The output value y obtained in S509 is the output value of a different pre-trained model than those obtained in S505 and S508. In S509, in order to estimate the type of sheet S, the output value y of the third pre-trained model for second-level classification is obtained, similar to S505 and S508 described above.

[0052] As shown in the estimation table in Figure 9, in S509, when the output value y obtained by inputting the features of sheet S into the third trained model 710 of the second detailed classification is 0, the CPU 301 estimates the type of sheet S to be recording medium 8. Similarly, when the output value y is 1, the CPU 301 estimates the type of sheet S to be recording medium 9. After that, the CPU 301 proceeds to processing in S507. In this way, in S509, the CPU 301 can uniquely estimate the type of sheet S, which was estimated as the second recording medium group in S505, as either recording medium 8 or recording medium 9.

[0053] As mentioned above, in S507, following S508 and S509, the CPU 301 displays the estimation result on the operation panel 102. When the processing of S507 is completed, the CPU 301 proceeds to S510.

[0054] In S510, CPU301 creates training data used to update the trained model. Training data is a type of training parameter used to train the trained model. This training data includes the features of Sheet S (all 15 types of features) obtained in S504 and information about the type of Sheet S (estimation result) obtained in S507. In other words, the training data includes first features (first values), which are the features used in the estimation process, and second features (second values), which are the features not used in the estimation process. Both the first and second features may contain multiple types of features. In S511, CPU301 saves the created training data to the specified storage location. Even with the same type of storage medium, characteristic values ​​may not be exactly the same due to individual differences. Therefore, even if the type of Sheet S is estimated correctly, the characteristic values ​​used and the estimation result are saved as training data and used to update the trained model, allowing the trained model to be optimized as needed. The training data created and saved here includes the data used in each trained model. For example, let's assume that the estimation result in S507 is recording medium 6. In this case, the training data for the first trained model used for rough classification will use estimation results indicating that it is the first group of recording media. On the other hand, the training data for the second trained model used for the first detailed classification will use estimation results indicating that it is recording medium 6. Thus, when detailed classification is performed, the training data for detailed classification will be created and saved separately from the training data for rough classification.

[0055] Furthermore, it is preferable that the storage location for the training data be the same as the storage location to which the device or system that updates the trained model belongs. This is to reduce the various costs required for updating the trained model. Therefore, in a configuration where the trained model is updated internally within the recording device 101, the storage location for the training data may be the memory 305 shown in Figure 3. In a configuration where the trained model is updated using an external device other than the recording device 101, the storage location for the training data is the machine learning device 170 (see Figure 17). The machine learning device may be the user's PC (Personal Computer), a server PC, or a server system consisting of multiple servers. When transferring training parameters from the recording device 101 to an external storage device, the transfer may be done via the USB port 304 or the LAN (Local Area Network) port 314.

[0056] <Screen displaying the estimated results> Figure 10 shows an example of an estimation result screen displayed on the operation panel 102 by the CPU 301 in S507. The estimation result screen 1000 shown in Figure 10 includes a message 1001 that indicates to the user that the recording medium "my PVC 1" has been estimated as the estimation result and that "my PVC 1" is being fed. When the OK button 1002 is pressed, the CPU 301 proceeds to S510. Here, an example is shown in which one type of recording medium is displayed as the estimation result, but it may also be configured to display up to three of the top recording medium types with high estimation degrees and allow the user to select which recording medium it is. If the user selects and specifies a recording medium type from among the recording medium types with high estimation degrees, the training data created in S510 will include the recording medium type selected by the user.

[0057] Area 1005 displays the types of recording media that have been used most recently. Area 1005 may be configured to allow the selection of any recording media from it. In this case as well, the selected type of recording media will be included in the training data created in S510. Button 1010 is a button for selecting other paper types and a button for registering new types of recording media. When button 1010 is pressed, a screen (not shown) for entering the name of the recording media type is displayed, and the new type of recording media is registered using the entered name. That is, the new type of recording media is added to the estimation table shown in Figure 9. In this case, the training data created in S510 will also include the newly added type of recording media.

[0058] In creating the training data for S510, the type of recording medium selected as the estimation result and all 15 features derived in S503 are created as training data. Similarly, for new recording media, the type of that new recording medium and all 15 features derived in S503 are created as training data.

[0059] <Pre-trained model> Figure 11 is a schematic diagram of a DNN (Deep Neural Network). The three pre-trained models 352 of this embodiment will be explained with reference to Figure 11.

[0060] The trained model 352 in this embodiment is a DNN as shown in the schematic diagram in Figure 11. The DNN receives data in the input layer 1101, propagates the data through the hidden layer 1102, and outputs the data in the output layer 1103. Each layer has multiple nodes indicated by circles. The input data is propagated towards the output layer while being weighted and biased between the nodes of each layer. Adjusting parameters such as weighting and bias so that a specified output is produced for a specified input is called training the model. The trained model is called the trained model. The dataset of input data and associated output data used to train the model is called the training data, as mentioned above.

[0061] In this embodiment, the input data for the training data consists of six features prepared for each type of sheet S. The six features are those derived in S503 and selected in S504, corresponding to the types of features described above. In this embodiment, an example is described in which the same number of features are used in training all models, but the embodiment is not limited to this example. The input layer 1101 of the trained model has six nodes corresponding to the input data of the training data. Each node is input to each feature.

[0062] In this embodiment, the output data of the training data is an integer value indicating the type of sheet S to be estimated. When there are two types of sheet S to be estimated, the output data of the training data will be 0 or 1. When there are seven types of sheet S to be estimated, the output data of the training data will be between 0 and 6. When actually training the model, the integer values ​​are converted into one-hot vectors and used. In this embodiment, a different combination of sheet S types is used for each model to be trained. The output layer 1103 of the trained model has the same number of nodes as the number of sheet S types to be estimated. Each node outputs the probability that sheet S is of each sheet type. If the output of the trained model is considered as an array, then each element of the output array can be thought of as the probability that it is of each sheet type. If each element of the output array is associated with each sheet type, then the index of each element is also associated with each sheet type. In this embodiment, the estimation result of the trained model is the index of the element with the highest probability.

[0063] The first pre-trained model 708 is a pre-trained model for coarse classification used in S505 as described above. The types of sheets S to be estimated are all recording media from recording media 1 to recording media 9. However, considering that it is difficult to estimate all recording media with high accuracy using a single pre-trained model, types of sheets S with similar characteristics are grouped together in advance. In this embodiment, recording media 6 and recording media 7 are grouped together as the first recording media group. Similarly, recording media 8 and recording media 9 are grouped together as the second recording media group. The output data for the coarse classification training data is an integer value indicating the type of sheet S, with recording media 1 being 0, recording media 2 being 1, recording media 3 being 2, recording media 4 being 3, recording media 5 being 4, the first recording media group being 5, and the second recording media group being 6.

[0064] The trained model created using the above training data is designated as the first trained model 708 for coarse classification. As shown in Figure 9, the first trained model 708 for coarse classification outputs 0 to 4 as the estimated result when features corresponding to recording media 1 to 5 are input. When features corresponding to recording media 6 or 7 are input, 5 is output as the estimated result. When features corresponding to recording media 8 or 9 are input, 6 is output as the estimated result.

[0065] The second pre-trained model 709 is the pre-trained model for the first detailed classification used in S508 as described above. The types of sheets S to be estimated are recording media 6 and recording media 7. The output data for the training data of the first detailed classification is an integer value indicating the type of sheet S, with recording media 6 being 0 and recording media 7 being 1.

[0066] The trained model, which was trained using the above training data, is designated as the second trained model 709 for first-level classification. As shown in Figure 9, the second trained model 709 for first-level classification outputs 0 or 1 as the estimation result when a feature corresponding to recording medium 6 or recording medium 7 is input.

[0067] The third pre-trained model 710 is the pre-trained model for the second detailed classification used in S509 as described above. The types of sheets S to be estimated are recording media 8 and recording media 9. The output data for the training data of the second detailed classification is an integer value indicating the type of sheet S, with recording media 8 being 0 and recording media 9 being 1.

[0068] The trained model, which has been trained using the above training data, is designated as the third trained model 710 for the second detailed classification. As shown in Figure 9, the third trained model 710 for the second detailed classification outputs 0 or 1 as the estimation result when a feature corresponding to recording medium 8 or recording medium 9 is input.

[0069] The three pre-trained models described above are generated by the machine learning device 170. The machine learning device 170 can record the pre-trained models and the calculation methods necessary for estimation using the pre-trained models into memory 305 via the USB port 304, input / output IF 303, and CPU 301. The CPU 301 measures the characteristics of the sheet S using the media sensor 206 and ultrasonic transmitter 207, derives feature quantities related to the surface information and cross-sectional information of the sheet S from the measurement data, and estimates the type of sheet S by inputting the derived feature quantities into the pre-trained model.

[0070] The explanation so far has described an example in which estimation processing is performed using a trained model created by the machine learning device 170 based on a predetermined type of recording medium and the flag information of the corresponding selection flag. The following sections will explain the update method for updating the aforementioned selection flag and the update method for the trained model. Note that if only the trained model needs to be updated, it can be retrained using the training data saved in S511 during the estimation (classification) process. However, for example, a new recording medium may be used in the recording device 101. Furthermore, even with the same recording medium, differences in production lots or deterioration during storage may cause changes in characteristics (feature quantities). In such cases, the estimation (classification) process may not be able to appropriately estimate the type of recording medium because there is no difference in feature quantities despite the use of different recording media. Therefore, in this embodiment, a process to update the selection flag is performed. This allows for the optimization of the features used in the trained model. That is, the types of features used as input data in the trained model are changed. As mentioned above, all 15 types of features are stored as training data. In other words, both features used and unused features in the inference process of the trained model are stored. Therefore, even if the type of features used changes due to the update of the selection flag, the corresponding features still exist as training data, allowing the trained model to be retrained using the modified features.

[0071] The selection flag update process and the learning model update process may be performed by the user via the operation panel 102. Alternatively, they may be performed periodically using the timer function in the recording device 101. If the selection flag update process results in the use of different types of features than before, the corresponding learning model update process (retraining process) will also be performed.

[0072] <How to update the selection flag> An example of updating the selection flags (flag information) stored in the selection flag storage units 702, 703, and 704 will be described. Specifically, an example of changing the type of feature used will be described. In this embodiment, a first method using the FilterMethod and a second method using the WrapperMethod will be described as methods for selecting the type of feature. The first method may be used alone, the second method may be used alone, or the first and second methods may be combined. In this embodiment, the update of the selection flags will be performed by the machine learning device 170.

[0073] Figure 17 shows an example of a machine learning device 170. As mentioned above, the machine learning device 170 is, for example, a PC. The CPU 1701 performs calculations, decisions, and control of data and instructions according to the software stored in the ROM 1702, RAM 1703, or hard disk 1704. The RAM 1703 is used as temporary storage when the CPU 1701 performs various processing. The hard disk 1704 stores the operating system (OS), application software, and programs. The system bus 1706 exchanges data between the CPU 1701, ROM 1702, RAM 1703, and hard disk 1704, etc. The network connection unit 1705 uses a wireless LAN or wired LAN compliant with standards such as IEEE 802.11a, and enables data exchange with devices on the same network and the Internet using protocols such as TCP / IP. Note that the configuration shown in Figure 17 is merely an example, and the configuration of the machine learning device 170 is not limited to this example.

[0074] The first method using the FilterMethod will be explained using Figures 12 and 13. Figure 12 is a flowchart of the FilterMethod. The process shown in Figure 12 is realized when the CPU 1701 of the machine learning device 170 reads a program stored in ROM 1702 or the like into RAM 1703, and the CPU 1701 executes it. The process shown in Figure 12 is executed in the machine learning device 170 based on, for example, the machine learning device 170 receiving a request to update the selection flag from the recording device 101. Note that the flag information shown in Figure 8 is assumed to have been sent from the recording device 101 to the machine learning device 170 when the update request was made.

[0075] The FilterMethod is a technique that calculates the correlation between multiple input features and excludes those with high correlations from the input. In this embodiment, it is assumed that the features to be used are selected by deciding which features to exclude from the 15 types of features mentioned above as input. However, explaining with 15 examples would be complex, so to simplify the explanation, we will describe an example using 5 types of features as input.

[0076] Figure 12 shows an example of calculating the correlation coefficient of five recording media features and performing feature weighting. In S1201, the CPU 1701 of the machine learning device 170 reads the training data stored in S511. As mentioned above, this training data includes the type of recording media (brand information) and all 15 types of features associated with that type of recording media. Here, as mentioned above, we assume that it includes five types of features as a simplified example.

[0077] In S1202, the CPU 1701 selects two feature amounts, namely, the feature amount x and the feature amount y, from the teacher data acquired in S1201. In S1203, the CPU 1701 calculates the correlation coefficient r of the two feature amounts selected in S1202. This correlation coefficient r can be calculated from the formula of Sxy / Sx*Sy. Sxy is the covariance of x and y. Sx is the standard deviation of x. Sy is the standard deviation of y.

[0078] In S1204, the CPU 1701 compares the absolute value of the correlation coefficient r calculated in S1203 with a specific value N. If the correlation coefficient r < N, it proceeds to S1205. If the correlation coefficient r >= N, it proceeds to S1206. As an example of this embodiment, let N be 0.8. In this embodiment, from among a plurality of types of feature amounts, any two feature amounts are selected, and a combination of feature amounts with a correlation coefficient r smaller than 0.8 is determined to have a low correlation. As a result, in S1205, the CPU 1701 sets the weighting to +0 for the said combination. That is, no weight is given. On the other hand, a combination of feature amounts with a correlation coefficient r of 0.8 or more is determined to have a high correlation. As a result, in S1206, the CPU 1701 sets the weighting to +1 for the said combination. After S1205 or S1206, the CPU 1701 proceeds to S1207.

[0079] In S1207, the CPU 1701 determines whether the calculation of the correlation coefficient r for all combinations of feature amounts has been completed. If it has been completed, the CPU 1701 proceeds to S1208. If it has not been completed, the CPU 1701 returns to S1202 and repeats the process of calculating the correlation coefficient of the feature amounts of the unprocessed combinations.

[0080] FIG. 13 is a diagram showing an example of the correlation coefficient obtained by obtaining the correlation with each of the other types of feature amounts for five types of feature amounts in this way, and the result of weighting.

[0081] In S1208, CPU1701 compares the weighted sum, which is the result of the weighted calculation in S1205 and S1206, with a predetermined value (let's call it XX). For example, as shown in Figure 13, the weighted sum for feature 1 is the sum of the weights between it and the other features, features 2 through 5, and in the example in Figure 13, it is "0". For feature 2, it is the sum of the weights between it and the other features, features 1, 3 through 5, and in the example in Figure 13, it is "3". In S1208, if CPU1701 finds that the weighted sum is greater than the predetermined value XX, it removes that feature from the training data. A weighted sum greater than the predetermined value XX means that the feature has a relatively high correlation with the other features. In other words, since it can be replaced by other features, this feature is not considered a useful feature for estimating the type of recording medium. If the predetermined value XX is, for example, 3, then as shown in Figure 13, features 2 and 4 can be determined to have a high correlation with the other features and are therefore excluded.

[0082] Next, in S1209, CPU1701 sets the selection flag corresponding to the excluded feature in the flag information to "1" and the selection flag corresponding to the excluded feature to "0". Then, CPU1701 terminates the processing of the flowchart shown in Figure 12.

[0083] Let's provide a supplementary explanation with a specific example. Consider the case where the flag information used in the selection flag storage unit 702 of the first pre-trained model 708 for coarse classification is updated. In this case, the classification results obtained from the estimation results using the first pre-trained model 708 for coarse classification (i.e., coarse classifications 0 to 6 in the estimation table in Figure 9) are used as training data. The processing shown in Figure 12 is performed using the features (all 15 types of features) obtained when these coarse classifications 0 to 6 are acquired. The processing in Figure 12 is performed for each individual training data point, and the features that were not ultimately excluded may be updated as flag information. Alternatively, the average value of each feature for each classification result of coarse classifications 0 to 6 may be used as input information for Figure 12. Furthermore, if a new type of recording medium is added, for example, training data labeled with coarse classification 7 for that additional recording medium may be included.

[0084] In this way, by excluding and filtering out combinations of features that have a high correlation, it is possible to select the features to be used for training. In this embodiment, since multiple pre-trained models are used, the selection flag update process using FilterMethod may be applied to each pre-trained model. In this example, five features were explained as an example, but there is no need to limit the number of input features.

[0085] In the process shown in Figure 12, the number of features to exclude (in other words, the number of features to keep) can be made variable by changing a predetermined value XX. Here, we have explained an example of updating the selection flag using the results shown in Figure 12, but as will be described later, the Filer Method shown in Figure 12 may also be used in combination with the Wrapper Method to update the selection flag.

[0086] Figures 14 and 15 illustrate the second method using the variable reduction method of the Wrapper Method. The variable reduction method is a technique that involves creating a provisional trained model by reducing the number of input features, and then repeatedly evaluating the created trained model to select the features. Figure 14 is a flowchart of the variable reduction method of the Wrapper Method. Figure 15 shows the combination of features and the coefficient of determination R, which is an evaluation metric for the trained model created with that combination. 2 This is a correspondence table. In this embodiment, it is assumed that the feature quantities to be used are selected from the 15 types of feature quantities mentioned above as input information. However, explaining with 15 examples would be complex, so to simplify the explanation, we will explain an example where 5 types of feature quantities are used as input information. In this embodiment, a combination of up to 4 types of feature quantities will be selected from these 5 types. The processing entity and trigger that starts the flowchart shown in Figure 14 are the same as in the example explained in the flowchart in Figure 12.

[0087] In S1401, CPU1701 reads the training data stored in S511. This training data includes the type of recording medium (brand information) and all 15 types of features associated with that recording medium type. For simplicity, as mentioned earlier, it is assumed that five types of features are included.

[0088] In S1402, the CPU 1701 provisionally sets a combination of the five features read in S1401. In S1403, the CPU 1701 generates a provisional pre-trained model using the multiple provisional combinations (combinations of features and brand information of recording media) set in S1402 as training data. For example, provisional training data is prepared by provisionally combining recording media 6 and 7, which are the estimated recording media types included in the training data, with the five features corresponding to those recording media 6 and 7. Then, a pre-trained model is generated such that when the five features are input, either recording media 6 or 7 is estimated. In this embodiment, it is assumed that the pre-trained model in S1402 is generated by training using the DNN shown in Figure 11.

[0089] In S1404, the CPU 1701 calculates the coefficient of determination R 2 , which is an evaluation index of the generated provisional learned model. The coefficient of determination R 2 is an index indicating the goodness of fit of regression analysis, and it is determined that the accuracy is higher as it approaches 1. In S1405, the CPU 1701 determines whether all the coefficients of determination R 2 have been calculated for the combination of feature amounts read in S1401. If all have been calculated, it proceeds to S1406; if not all have been calculated, it returns to S1402.

[0090] When returning to S1402, the CPU 1701 reduces one feature amount. For example, in this example, four combinations of feature amounts are temporarily set, and S1403 to S1405 are repeated again. For example, teacher data that temporarily combines the recording media 6 and 7 and the four feature amounts corresponding to the recording media 6 and 7 is temporarily prepared. And a learned model is generated such that the recording media 6 or 7 is estimated when four feature amounts are input.

[0091] In S1406, the CPU 1701 selects the combination with the highest coefficient of determination R 2 . FIG. 15 is a diagram showing the results of calculating the coefficient of determination R 2 from among combinations of five types of feature amounts and combinations of four types of feature amounts. In the first loop process, the coefficient of determination R 2 is calculated using five types of feature amounts, and "1" is assigned to feature amounts 1 to 5, and the coefficient of determination R 2 is 0.78. That is, it is assumed that "1" is assigned to the feature amounts used in FIG. 15 and "0" is assigned to the feature amounts not used. From the second to the sixth time, it is calculated using four types of feature amounts. That is, from the second to the sixth time, the coefficient of determination R 2 is calculated for combinations in the form of excluding different feature amounts respectively. For example, in the second loop process, "1" is assigned to the used feature amounts 2 to 5, and the coefficient of determination R 2 is 0.71. The CPU 1701 selects the coefficient of determination R 2The system selects the combination that yields the highest result.

[0092] In the example in Figure 15, the highest coefficient of determination R 2 This is 0.84, the result of the fifth loop. The features 1, 3, 4, and 5 used in this fifth loop are selected as the optimal combination of features. In other words, in the example in Figure 15, feature 2 is considered to be noise, and the combination of features excluding feature 2 is determined to be the combination with the highest accuracy.

[0093] Finally, in S1407, CPU 1701 sets the selection flags corresponding to the features included in the selected optimal feature combination in the flag information to "1", and sets the selection flags corresponding to the excluded features to "0". Then, CPU 1701 terminates the processing of the flowchart shown in Figure 14.

[0094] Let's provide a supplementary explanation with a specific example. Consider the case where the flag information used in the selection flag storage unit 702 of the first pre-trained model 708 for coarse classification is updated. In this case, the classification results obtained from the estimation results using the first pre-trained model 708 for coarse classification (i.e., coarse classifications 0 to 6 in the estimation table in Figure 9) are used as training data. The processing shown in Figure 12 is performed using the features (all 15 types of features) obtained when these coarse classifications 0 to 6 are acquired. The processing in Figure 12 is performed for each individual training data point, and the features that were not ultimately excluded may be updated as flag information. Alternatively, the average value of each feature for each classification result of coarse classifications 0 to 6 may be used as input information for Figure 12. Furthermore, if a new type of recording medium is added, for example, training data labeled with coarse classification 7 for that additional recording medium may be included.

[0095] In this way, the WrapperMethod can select the optimal combination from multiple features by creating a provisional learning model and repeatedly evaluating it.

[0096] The above describes a method for selecting the optimal combination of features from a range of 5 to 4 types, but in this embodiment, it is not necessary to limit the number of types of features.

[0097] The feature selection method in this embodiment may be either the first method using the FilterMethod or the second method using the WrapperMethod, or a combination of both. An example of a combination of both is described below. First, the first method using the FilterMethod is used to select 10 features from a total of 15 types, and then the second method using the WrapperMethod is performed using those 10 features. The FilterMethod is a process that selects the optimal minimum number of features, but it does not actually evaluate the trained model. Therefore, by repeatedly generating and evaluating trained models using the WrapperMethod with the features selected by the FilterMethod, it is possible to determine the appropriate selection flags to be used in each trained model. Conversely to the above example, the first method may be performed after the second method.

[0098] <Updating a trained model> Figure 16 shows an example of the process for updating a trained model. Figure 16(a) is a transition diagram of the process for updating a trained model. Figure 16(b) is a flowchart showing an example of the process for updating a trained model in a system for updating trained models. In this embodiment, a series of processes for updating a trained model are performed using a recording device 101 and a machine learning device 170. The step numbers shown in Figure 16(a) and Figure 16(b) refer to the same steps.

[0099] S1701 and S1704 are processes performed by the recording device 101. S1702 and S1703 are processes performed by the machine learning device 170.

[0100] In S1701, the recording device 101 sends a request for update processing to the machine learning device 170. That is, the recording device 101 outputs an instruction to the machine learning device 170 to update the trained model. In S1702, the machine learning device 170 reads the training data to be used for the update. The machine learning device 170 has training data of a predetermined type of recording medium stored in it. The machine learning device 170 also reads the flag information of the trained model to be updated. The flag information is information about the combination of features selected by using at least one of the first method using the FilterMethod and the second method using the WrapperMethod described above. This flag information may be stored in the machine learning device 170 together with the training data, or it may be sent in the update processing request from the recording device 101.

[0101] Next, in S1703, the machine learning device 170 updates the trained model using the acquired training data based on the selection flag information. The update of the trained model may also be done using a machine learning method with a DNN (Deep Neural Network) as described in Figure 11. The input data for the DNN's training data consists of the features of each recording medium. The output data for the DNN's training data contains information indicating the type of recording medium. In this way, the machine learning device updates the trained model by retraining it with training data related to the recording medium to be estimated. Finally, in S1704, after detecting the completion of the update of the trained model in S1703, the recording device 101 replaces the trained model in the recording device 101 with the new model.

[0102] The process of updating a trained model will be explained using a specific example. As shown in the estimation table in Figure 9, recording media 1 to 9 are registered as sheet types, and trained models corresponding to these have been generated. Now, let's consider the case where a new recording media 10 is added. We will assume that this new recording media 10 is classified as rough classification 5 in the output of the first trained model for rough classification. In other words, we will assume that the new recording media 10 is estimated to belong to the same first recording media group as recording media 6 and 7. In this case, in the display process of the estimation process results in S507, recording media 6 or 7 will be displayed as the estimation result based on the result of the first detailed classification. The user registers the new recording media 10 as a new recording media through the estimation results screen. We will then explain the case where retraining is performed using the training data, including the training data for this new recording media 10.

[0103] Ideally, the new recording medium 10 should be classifiable by the first pre-trained model for coarse classification. Therefore, in the first pre-trained model for coarse classification, flag information is updated and retrained using training data that includes the training data for the new recording medium 10. However, as described above, the new recording medium 10 is classified as coarse classification 5 corresponding to the first group of recording media. Therefore, flag information is updated and retrained in the second pre-trained model using the training data for the new recording medium 10 and the training data for recording media 6 and 7 corresponding to the second pre-trained model for first detailed classification.

[0104] As mentioned above, even with the same type of recording medium, the classification results may differ from previous classification results due to differences in lot numbers, etc. Therefore, in the embodiment described above, a process is performed to add the feature data of the recording medium currently being used by the user to the training data. Then, by updating the flag information of the trained models for coarse and detailed classification and retraining the trained models, the type of recording medium can be accurately determined. In addition, although the above explanation used differences on the recording medium side as an example, the acquired feature quantities may also change due to deterioration of the sensor on the recording device 101 side, etc. In such cases as well, the flag information update process and the retraining process of the trained models are useful.

[0105] In this embodiment, an example using three pre-trained models has been described, but the invention is not limited to this example. For example, consider a case where both the newly added new recording media 11 and 12 are estimated to be new recording media 11. In this case, the new recording media 11 and 12 will constitute a third group of recording media, and a new trained model for the third group of recording media will be generated. Thus, if the estimation results of the trained models correspond to multiple recording media, additional trained models may be added and the processing performed.

[0106] <<Other Embodiments>> The embodiments described above are examples of implementing the Disclosure and do not limit it. For example, the Disclosure may be applied not only to recording devices that eject ink onto a sheet to form an image, but also to scanners that read images on a sheet, or post-processing machines that process sheets.

[0107] The estimation of the sheet type is not limited to the CPU 301 mounted on the recording device 101; it may also be performed by a scanner, post-processing machine, or PC, etc.

[0108] The detailed classifications (second estimation) shown in S507 and S508 are not limited to two; one or more classifications are acceptable. For example, if the types of sheet S that are prone to misestimation in the rough classification (first estimation) in S504 are divided into five groups, five detailed classifications corresponding to each group may be established. Furthermore, the types of recording media within each group's detailed classification are not limited to two; two or more classifications are acceptable. For example, if there are three types of sheet S that are prone to misestimation in the rough classification, and these three types can be accurately estimated in the detailed classification, these three types may be classified as one group in the rough classification.

[0109] The machine learning device may be mounted on the recording device 101. Alternatively, the recording device 101 may generate a trained model. The training data used when generating the trained model may be stored in the memory 305.

[0110] The aforementioned features are not limited to 15; the color or thickness of the sheet S may also be used. Furthermore, although the above embodiment described estimating the type of sheet S from 15 features, it is not limited to this. For example, the type of sheet S may be estimated from at least one feature relating to the surface information of the sheet S and at least one feature relating to the cross-sectional information of the sheet S. Alternatively, data acquired by the media sensor 206, etc., may be used directly as input data without deriving any features. The output data for the training data is not limited to the type of sheet; the sheet model name or a sheet name defined by the user may also be used.

[0111] The trained model may be stored outside the recording device 101. For example, it may be stored on a PC connected via the input / output IF 303 and USB port 304 of the recording device 101. Furthermore, the trained model is not limited to a DNN, but may also be a decision tree, etc. Also, in the above embodiment, the estimation of the type of sheet S was explained using the example of applying the trained model to coarse classification and detailed classification, but it is not limited to this. For example, the trained model may be applied to either coarse classification or detailed classification. Specifically, the type of sheet S may be estimated by applying the trained model to coarse classification and deriving the index y in detailed classification. In other words, the type of sheet S may be estimated by combining the trained model and the index y.

[0112] The embodiments described above describe a configuration using a pre-trained model for coarse classification and a pre-trained model for detailed classification, but the system is not limited to this example. When changing the type of recording medium to be estimated, it is sufficient that the number of recording media to be estimated falls within a predetermined range. Then, the pre-trained models should be updated so that the determined recording media can be estimated. For example, in a state where a pre-trained model for coarse classification and a pre-trained model for detailed classification are being used, there may be cases where the pre-trained model for detailed classification is no longer needed after changing the recording medium to be estimated. That is, there may be cases where the type of recording medium can be uniquely estimated using only the pre-trained model for coarse classification. Conversely, in a state where the type of recording medium can be uniquely estimated using only the pre-trained model for coarse classification, changing the recording medium to be estimated may necessitate the pre-trained model for detailed classification. Alternatively, in a state where the type of recording medium can be uniquely estimated using only the pre-trained model for coarse classification, changing the recording medium to be estimated may still result in the same case. Thus, the embodiments described above are also applicable in configurations where only a pre-trained model for coarse classification, i.e., a single pre-trained model, is used. Furthermore, even in configurations where a single pre-trained model is used, as described above, by training the pre-trained model to estimate the types of recording media within a predetermined range, the recording media can be estimated with high accuracy.

[0113] Furthermore, in the embodiments described above, an example was explained in Figure 5 in which training data is created in S510 and training data is saved in S511. In addition, an example was explained in Figure 16 in which training data corresponding to each recording medium is pre-stored. In order to improve learning accuracy, it is preferable to create training data in S510 and save training data in S511 as described in the embodiments, but the trained model may be updated using pre-stored training data. That is, an implementation that does not perform S510 and S511 may be adopted. Alternatively, processes that perform S510 and S511 and processes that do not perform them may be mixed. That is, training data in an appropriately thinned form may be used.

[0114] This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0115] The disclosure of this embodiment includes configurations represented by the following examples of information processing devices, control methods for information processing devices, and programs.

[0116] <Configuration 1> An acquisition means for obtaining multiple characteristic values ​​of the recording medium by measuring the recording medium being fed, A registration means that registers the type of recording medium to be estimated, An estimation means that estimates the type of recording medium to be fed, from among the multiple characteristic values ​​obtained, based on a first characteristic value which is a characteristic value selected based on selection information that defines the characteristic value to be used, using an estimation unit, from among the multiple types of recording media registered in the registration means. A means for changing the aforementioned selection information, Output means for outputting an instruction to update the estimation unit using the first characteristic value indicated by the selection information, An information processing device characterized by comprising:

[0117] <Configuration 2> The information processing apparatus according to Configuration 1, characterized in that when the selection information is changed by the changing means, the output means outputs an instruction to update the estimation unit using the first characteristic value indicated by the changed selection information.

[0118] <Structure 3> The information processing apparatus according to configuration 1 or 2, characterized in that the modification means modifies the selected information by at least one of a first method using a Filter Method and a second method using a Wrapper Method.

[0119] <Structure 4> The information processing apparatus according to configuration 3, characterized in that the modification means applies the second method after applying the first method, or applies the first method after applying the second method.

[0120] <Composition 5> The estimation unit is a trained model configured to output output data indicating the type of recording medium corresponding to the input data when input data corresponding to characteristic values ​​is input. The information processing apparatus according to any one of configurations 1 to 4, characterized in that the estimation means inputs the first characteristic value as input data to the estimation unit to obtain the type of recording medium output from the estimation unit as an estimation result.

[0121] <Composition 6> The information processing apparatus according to configuration 5, characterized in that when the selection information is changed by the changing means, the output means outputs an instruction to update the trained model based on training data which includes input data corresponding to the changed first characteristic value and output data indicating the type of recording medium corresponding to the input data.

[0122] <Composition 7> The estimation of the first recording medium is performed using the first value, which is the first characteristic value among the multiple characteristic values ​​acquired by the acquisition means. The information processing apparatus according to configuration 5 or 6, wherein the characteristic value of the input data corresponding to the output data indicating the type of the first recording medium, used in updating the trained model, is a characteristic value indicated by the selection information, among characteristic values ​​including the first value used in estimating the first recording medium and a second value measured when acquiring the first value but not used in estimating the first recording medium.

[0123] <Structure 8> The information processing apparatus according to configuration 7, characterized in that at least one of the first value and the second value each includes a plurality of types of values.

[0124] <Composition 9> The information processing apparatus according to configuration 7 or 8, characterized in that, when the estimation means estimates that the type of recording medium is the first recording medium, training data is stored which associates all of the multiple characteristic values ​​acquired by the acquisition means, including the first value and the second value, with the fact that the type of recording medium is the first recording medium.

[0125] <Composition 10> The information processing apparatus according to any one of configurations 1 to 9, characterized in that the acquisition means acquires feature quantities relating to the surface information of the paper-fed recording medium and feature quantities relating to the cross-sectional information of the paper-fed recording medium, and acquires the characteristic value based on the acquired feature quantities.

[0126] <Composition 11> The information processing apparatus according to configuration 10, wherein the acquisition means acquires feature quantities relating to the surface information from a sensor that acquires a surface image of the recording medium, and acquires feature quantities relating to the cross-sectional information of the recording medium from a sensor that acquires an electrical signal of ultrasonic waves transmitted through the recording medium.

[0127] <Composition 12> The information processing apparatus according to configuration 10 or 11, characterized in that the surface information includes information regarding at least one of the brightness and unevenness of the paper-fed recording medium.

[0128] <Composition 13> The information processing apparatus according to any one of configurations 10 to 12, characterized in that the cross-sectional information includes information regarding at least one of the thickness and basis weight of the paper-fed recording medium.

[0129] <Composition 14> The information processing apparatus according to any one of configurations 1 to 13, further comprising a display control means for displaying the results estimated by the estimation means.

[0130] <Composition 15> The information processing apparatus according to configuration 14, characterized in that the registration means registers a new type of recording medium in response to a user's operation on the result displayed by the display control means.

[0131] <Composition 16> The estimation unit is configured to estimate the group of recording media, including the second and third recording media, as one of the types of recording media. The information processing apparatus according to any one of configurations 1 to 15, characterized in that, when the estimation result by the estimation unit indicates the group of recording media, the estimation means estimates the type of recording media to be fed, from the group of recording media including the second recording media and the third recording media, using a second estimation unit different from the estimation unit, based on the characteristic value acquired by the acquisition means.

[0132] <Composition 17> The information processing apparatus according to configuration 16, characterized in that the second estimation unit is a trained model trained using training data corresponding to the types of recording media included in the group of recording media, including the second recording media and the third recording media.

[0133] <Composition 18> The information processing device according to configuration 16 or 17, characterized in that the selection information used in the second estimation unit is different from the selection information used in the estimation unit.

[0134] <Composition 19> A step of obtaining multiple characteristic values ​​of the recording medium by measuring the recording medium being fed, The steps include: estimating the type of recording medium to be fed using an estimation unit from among a registered list of recording medium types, based on a first characteristic value which is selected from among the multiple characteristic values ​​obtained, based on selection information that defines the characteristic value to be used; The step of changing the aforementioned selection information, Output means for outputting an instruction to update the estimation unit using the first characteristic value indicated by the selection information, A control method for an information processing device, characterized by having the following features.

[0135] <Composition 20> A program for causing a computer to function as one of the means of the information processing device described in any one of configurations 1 to 18. [Explanation of Symbols]

[0136] 101 Recording device 206 Media Sensors 207 Ultrasonic Transmitter 301 CPU 352 pre-trained models

Claims

1. An acquisition means for obtaining multiple characteristic values ​​of the recording medium by measuring the recording medium being fed, A registration means that registers the type of recording medium to be estimated, An estimation means that estimates the type of recording medium to be fed, from among the multiple characteristic values ​​obtained, based on a first characteristic value which is a characteristic value selected based on selection information that defines the characteristic value to be used, using an estimation unit, from among the multiple types of recording media registered in the registration means. A means for changing the aforementioned selection information, Output means for outputting an instruction to update the estimation unit using the first characteristic value indicated by the selection information, An information processing device characterized by comprising:

2. The information processing apparatus according to claim 1, characterized in that when the selection information is changed by the changing means, the output means outputs an instruction to update the estimation unit using the first characteristic value indicated by the changed selection information.

3. The information processing apparatus according to claim 1, characterized in that the modification means modifies the selected information by at least one of a first method using the Filter Method and a second method using the Wrapper Method.

4. The information processing apparatus according to claim 3, characterized in that the modification means applies the second method after applying the first method, or applies the first method after applying the second method.

5. The estimation unit is a trained model configured to output output data indicating the type of recording medium corresponding to the input data when input data corresponding to characteristic values ​​is input. The information processing apparatus according to claim 1, characterized in that the estimation means inputs the first characteristic value as input data to the estimation unit to obtain the type of recording medium output from the estimation unit as an estimation result.

6. The information processing apparatus according to claim 5, wherein, when the selection information is changed by the changing means, the output means outputs an instruction to update the trained model based on training data including input data corresponding to the changed first characteristic value and output data indicating the type of recording medium corresponding to the input data.

7. The estimation of the first recording medium is performed using the first value, which is the first characteristic value among the multiple characteristic values ​​acquired by the acquisition means. The information processing apparatus according to claim 5, wherein the characteristic value of the input data corresponding to the output data indicating the type of the first recording medium, used in updating the trained model, is a characteristic value indicated by the selection information, among characteristic values ​​including the first value used to estimate the first recording medium and the second value measured when acquiring the first value but not used to estimate the first recording medium.

8. The information processing apparatus according to claim 7, characterized in that at least one of the first value and the second value each includes a plurality of types of values.

9. The information processing apparatus according to claim 7, characterized in that, when the estimation means estimates that the type of recording medium is the first recording medium, training data is stored which associates all of the multiple characteristic values ​​acquired by the acquisition means, including the first value and the second value, with the fact that the type of recording medium is the first recording medium.

10. The information processing apparatus according to claim 1, wherein the acquisition means acquires feature quantities relating to the surface information of the paper-fed recording medium and feature quantities relating to the cross-sectional information of the paper-fed recording medium, and acquires the characteristic value based on the acquired feature quantities.

11. The information processing apparatus according to claim 10, wherein the acquisition means acquires feature quantities relating to the surface information from a sensor that acquires a surface image of the recording medium, and acquires feature quantities relating to the cross-sectional information of the recording medium from a sensor that acquires an electrical signal of ultrasonic waves transmitted through the recording medium.

12. The information processing apparatus according to claim 10, characterized in that the surface information includes information regarding at least one of the brightness and unevenness of the paper-fed recording medium.

13. The information processing apparatus according to claim 10, characterized in that the cross-sectional information includes information regarding at least one of the thickness and basis weight of the paper-fed recording medium.

14. The information processing apparatus according to claim 1, further comprising a display control means for displaying the results estimated by the estimation means.

15. The information processing apparatus according to claim 14, characterized in that the registration means registers a new type of recording medium in response to a user's operation on the result displayed by the display control means.

16. The estimation unit is configured to estimate the group of recording media, including the second and third recording media, as one of the types of recording media. The information processing apparatus according to claim 1, characterized in that, when the estimation result by the estimation unit indicates the group of recording media, the estimation means estimates the type of recording media to be fed, based on the characteristic value acquired by the acquisition means, from the group of recording media including the second recording media and the third recording media, using a second estimation unit different from the estimation unit.

17. The information processing apparatus according to claim 16, characterized in that the second estimation unit is a trained model trained using training data corresponding to the types of recording media included in the group of recording media, including the second recording media and the third recording media.

18. The information processing device according to claim 16, characterized in that the selection information used in the second estimation unit is different from the selection information used in the estimation unit.

19. A step of obtaining multiple characteristic values ​​of the recording medium by measuring the recording medium being fed, The steps include: estimating the type of recording medium to be fed using an estimation unit from among a registered list of recording medium types, based on a first characteristic value which is selected from among the multiple characteristic values ​​obtained, based on selection information that defines the characteristic value to be used; The step of changing the aforementioned selection information, Output means for outputting an instruction to update the estimation unit using the first characteristic value indicated by the selection information, A control method for an information processing device, characterized by having the following features.

20. A program for causing a computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 18.

Images