Recognition system, recognition method, program, learning method, learned model, distilled model, and learning dataset generation method

Abstract

The recognition system stores a learned model (M10) including first and second model sections (M11, M12). The first model section (M11) outputs, for input of a first resolution image (P10) of an object (100), a second resolution image (P20) equivalent to an image obtained by converting the first resolution image (M10) to a higher resolution and a difference image (P30) of the first resolution image (P10) and the second resolution image (P20). The second model section (M12) outputs, for input of the second resolution image (P20) and the difference image (P30), a feature amount (F11) of the object (100). The arithmetic circuit receives a first resolution image (P10) as a target image and supplies the received target image to the learned model (M10) to calculate a feature amount (F11) of the object (100) appearing in the target image.

Description

Technical Field

[0001] This disclosure generally relates to recognition systems, recognition methods, programs (computer programs), learning methods, learned models, distillation models, and methods for generating learning datasets. This disclosure particularly relates to recognition systems, recognition methods, programs (computer programs), learning methods, learned models, distillation models, and methods for generating learning datasets related to objects displayed in images. Background Technology

[0002] Non-Patent Document 1 discloses a face recognition technique using deep learning. Non-Patent Document 1 discloses that, in order to reduce the impact of low-resolution images used in face recognition, a generative network (super-resolution network) is used to generate high-resolution images from low-resolution images, and face recognition is performed based on the high-resolution images obtained through the super-resolution network.

[0003] Existing technical documents

[0004] Non-patent literature

[0005] Non-patent document 1: Bayram Bayramli, et al., “FH-GAN: Face Hallucination and Recognition using Generative Adversarial”, [online], May 16, 2019, [retrieved September 3, 2020], Cornell University, Internet <URL: https: / / arxiv.org / abs / 1905.06537> Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] This disclosure provides an identification system, identification method, program, learning method, learned model, distillation model, and method for generating learning datasets that can improve the accuracy of object identification.

[0008] Solutions for solving technical problems

[0009] The recognition system disclosed herein includes a storage device for storing a learned model and a computational circuit capable of accessing the storage device. The learned model includes a first model unit and a second model unit. The first model unit is a model that has been learned to output a second resolution image and a difference image as input to a first resolution image. The first resolution image is an image of an object displayed at the first resolution. The second resolution image is equivalent to an image obtained by converting the first resolution image to a second resolution higher than the first resolution. The difference image is an image equivalent to the difference between the first resolution image and the second resolution image. The second model unit is a model that has been learned to output feature quantities of the object as input to the second resolution image and the difference image. The computational circuit performs: an acquisition process to acquire the first resolution image as an object image; and an inference process to provide the object image acquired in the acquisition process to the learned model so that the learned model calculates the feature quantities of the object displayed in the object image.

[0010] One aspect of the recognition method disclosed herein is an recognition method executed by a computational circuit capable of accessing a storage device storing a learned model. The learned model includes a first model unit and a second model unit. The first model unit is a model that has been learned to output a second resolution image and a difference image as input to a first resolution image. The first resolution image is an image of an object displayed at the first resolution. The second resolution image is equivalent to an image obtained by converting the first resolution image to a second resolution higher than the first resolution. The difference image is an image equivalent to the difference between the first resolution image and the second resolution image. The second model unit is a model that has been learned to output feature quantities of the object as input to the second resolution image and the difference image. The recognition method includes: an acquisition process that acquires the first resolution image as an object image; and an inference process that provides the object image received in the acquisition process to the learned model, so that the learned model calculates feature quantities of the object displayed in the object image.

[0011] One aspect of this disclosure is a program for causing an arithmetic circuit to perform an identification method.

[0012] The learning method of this disclosure includes a preparation step of preparing a model and a learning step of performing machine learning using the model prepared in the preparation step. The model includes a first model unit, a second model unit, and a third model unit. The first model unit is a model that outputs a second resolution image and a difference image as input to a first resolution image. The first resolution image is an image of an object displayed at the first resolution. The second resolution image is equivalent to an image obtained by converting the first resolution image to a second resolution higher than the first resolution. The difference image is an image equivalent to the difference between the first resolution image and the second resolution image. The second model unit is a model that outputs feature quantities of the object as input to the second resolution image and the difference image from the first model unit. The third model unit is a model that outputs the result of object recognition as input to the feature quantities of the object from the second model unit. The learning step includes using machine learning on a learning dataset with the first resolution image as input and the result of object recognition displayed in the first resolution image as the correct answer, to enable the model to learn the relationship between the feature quantities of the first resolution image and the object displayed in the first resolution image.

[0013] The learning model disclosed herein includes a first model unit and a second model unit. The first model unit is a model that, given an input of a first-resolution image, outputs a second-resolution image and a difference image. The first-resolution image is an image displaying an object at a first resolution. The second-resolution image is equivalent to an image obtained by converting the first-resolution image to a second resolution, which is higher than the first resolution. The difference image is an image representing the difference between the first-resolution image and the second-resolution image. The second model unit is a model that, given input of the second-resolution image and the difference image, outputs feature quantities of the object.

[0014] One aspect of the distillation model disclosed herein is generated by distillation of a learned model.

[0015] The identification system of this disclosure includes a storage device for storing a distillation model and a computational circuit capable of accessing the storage device. The computational circuit performs an acquisition process and an inference process. The acquisition process is the process of acquiring a first-resolution image as an object image. The inference process is the process of feeding the object image acquired in the acquisition process to the distillation model so that the distillation model calculates the feature quantities of the object appearing in the object image.

[0016] The method for generating a learning dataset disclosed herein includes a first step, a second step, a third step, and a fourth step. The first step is to obtain a reference image of the object to be displayed. The second step is to convert the reference image into a low-resolution image with a lower resolution than the reference image. The third step is to generate a difference image, which is equivalent to the difference between the reference image and the low-resolution image. The fourth step is to generate a learning dataset with the low-resolution image as input and the reference image and the difference image as the correct answers.

[0017] The effects of the invention

[0018] According to the aspects disclosed herein, it can achieve the effect of improving the accuracy of object identification. Attached Figure Description

[0019] Figure 1 This is a block diagram of an identification system according to one embodiment.

[0020] Figure 2 This is an illustration of a recognition method using a learned model executed in a recognition system.

[0021] Figure 3 This is a flowchart of the identification method.

[0022] Figure 4 This is a flowchart of a learning method used to generate a learned model for use in a recognition system.

[0023] Figure 5 This is an illustration of a learning method.

[0024] Figure 6 A block diagram of a learning system for implementing learning methods. Detailed Implementation

[0025] (1) Implementation method

[0026] (1-1)Overview

[0027] Figure 1 An identification system 10 according to one embodiment is shown. The identification system 10 is a system for identifying an object 100. Specifically, the identification system 10 is used to acquire data of an image of the object 100 displayed from the camera system 200, provide the identification result of the object 100 displayed in the image to the control system 300, and execute an action corresponding to the identification result through the control system 300.

[0028] Object 100 is a visually recognizable object that is identified by the recognition system 10. In this embodiment, object 100 is envisioned as a human being. Object 100 is not limited to a human being, but may also be a living being such as an animal. Object 100 is not limited to living beings, but may also be inanimate objects. Examples of inanimate objects include moving objects such as vehicles such as two-wheeled vehicles, automobiles, ships, and aircraft, as well as drones. Object 100 is not limited to the entirety of an object, but may also be a part of an object.

[0029] The camera system 200 is a system for generating data to display an image of an object 100. In this embodiment, "display object 100" refers to at least a portion of the display object 100, not all of it. For example, if the object 100 is a person, an image displaying a person's face can be considered an image of the display object 100. The camera system 200 is communicatively connected to the recognition system 10 and can provide data of the image of the display object 100 to the recognition system 10. The camera system 200 includes one or more cameras (digital cameras).

[0030] The control system 300 is a system capable of performing actions corresponding to the recognition result of object 100. In one example, the control system 300 can be used to control doors of facilities such as factories, shops, and buildings (the entire building or individual floors). If the recognition result of object 100 is that object 100 is a person permitted to pass through the door, then the control system 300 can unlock the door. In another example, the control system 300 can be used to control the actions (movements) of a robot. The control system 300 can determine the robot's actions based on the recognition result of object 100. The control system 300 can communicatively connect to the recognition system 10 and can receive the recognition result of object 100 from the recognition system 10. The control system 300 includes a computer system, which includes one or more memories and one or more processors, etc.

[0031] like Figure 1 As shown, the recognition system 10 includes a storage device 12 for storing the learned model M10 and a processing circuit 13 capable of accessing the storage device 12. For example... Figure 2As shown, the learned model M10 includes a first model unit M11 and a second model unit M12. The first model unit M11 is a model that has been learned as follows: Taking as input a first-resolution image P10 of the object 100 displayed at a first resolution, it outputs a second-resolution image P20 and a difference image P30, which are equivalent to converting the first-resolution image P10 into a second-resolution image that is higher than the first resolution. The difference image P30 is an image representing the difference between the first-resolution image P10 and the second-resolution image P20. The second model unit M12 is a model that has been learned as follows: Taking as input the second-resolution image P20 and the difference image P30 from the first model unit M11, it outputs the feature quantity F11 of the object 100. (The text repeats itself here.) Figure 3 As shown, the arithmetic circuit 13 performs: acquisition process S11, acquiring the first resolution image P10 as an object image; and inference process S12, providing the object image acquired in acquisition process S11 to the learned model M10, so that the learned model M10 calculates the feature quantity F11 of the object 100 displayed in the object image.

[0032] Thus, when the recognition system 10 extracts the feature quantity F11 of the object 100 from the first resolution image P10 of the displayed object 100, it utilizes both a second resolution image P20 (which has a higher resolution than the first resolution image P10) and a difference image P30. Since the second resolution image P20 has a higher resolution than the first resolution image P10, it can be described as an image obtained by capturing the same object 100 as the first resolution image P10 at a closer distance. Empirically, it is expected that capturing an image of the object 100 at a closer distance can improve the accuracy of object 100 recognition. Considering this, it is believed that the information lost due to the increased distance from the object 100 has a significant impact on the accuracy of object 100 recognition. For example, if the first resolution image P10 and the second resolution image P20 are images of a human face, the information lost due to the increased distance from the object 100 could include detailed information related to the eyes, nose, mouth, etc. On the other hand, it is believed that information such as the facial contours is maintained between the first resolution image P10 and the second resolution image P20. Since the difference image P30 is equivalent to the difference between the first resolution image P10 and the second resolution image P20, it can be said to represent the information lost from the second resolution image P20 due to the increased distance from the object 100. The fully learned model M10 used by the recognition system 10 utilizes both the difference image P30 and the second high-resolution image P20 to extract the feature quantity F11 of the object 100 from the first resolution image P10 displaying the object 100. Therefore, the recognition system 10 can improve the accuracy of recognizing the object 100.

[0033] (1-2) Details

[0034] (1-2-1) Identification System

[0035] The identification system 10 will be described in detail below. Figure 1 As shown, the identification system 10 includes an interface 11, a storage device 12, and a computing circuit 13. In this embodiment, the identification system 10 is implemented through a server.

[0036] Interface 11 is used for inputting information to and outputting information from the identification system 10. Interface 11 includes an input / output device 111 and a communication device 112. The input / output device 111 functions as an input device for inputting information from a user and as an output device for outputting information to a user. The input / output device 111 has one or more human-machine interfaces. Examples of human-machine interfaces include keyboards, pointing devices (mice, trackballs, etc.), touchpads, etc., as input devices, displays, speakers, etc., and touch panels, etc. The communication device 112 can communicatively connect to external devices or systems. In this embodiment, the communication device 112 is used for communication with the camera system 200 and the control system 300 via a communication network. The communication device 112 has one or more communication interfaces. The communication device 112 can connect to a communication network and has the function of communicating via the communication network. The communication device 112 operates according to a predetermined communication protocol. The predetermined communication protocol can be selected from various known wired and wireless communication standards.

[0037] Storage device 12 is used to store information utilized by arithmetic circuit 13 and information generated by arithmetic circuit 13. Storage device 12 includes one or more storage units (non-temporary storage media). The storage unit may be, for example, any of a hard disk drive, optical disk drive, and solid-state drive (SSD). In addition, the storage unit may be any of an internal type, an external type, and a NAS (network-attached storage) type.

[0038] The information stored in storage device 12 includes the learned model M10 and database D10. Figure 1 The diagram shows that storage device 12 stores all the states of the learned model M10 and database D10. The learned model M10 and database D10 do not need to be stored in storage device 12 all the time; they only need to be stored in storage device 12 when required by the arithmetic circuit 13.

[0039] After learning, model M10 is used in recognition system 10 for the recognition of object 100. For example... Figure 2As shown, after learning, model M10 performs the following learning: For input to the first resolution image P10, it outputs the feature quantity F11 of the object 100 displayed in the first resolution image P10. The learned model M10 includes a first model unit M11 and a second model unit M12.

[0040] The first model unit M11 is a model that has undergone the following learning process (learned model): Taking a first-resolution image P10 as input, it outputs a second-resolution image P20 and a difference image P30. The first-resolution image P10 is an image displaying the object 100 at the first resolution. In this embodiment, the object 100 is a person. The first-resolution image P10 is conceived as an image displaying a portion, rather than all, of the features of the object 100. The features are those that contribute to the recognition of the object 100; in this embodiment, this is conceived as the face of the object 100. The second-resolution image P20 is equivalent to an image obtained by converting the first-resolution image P10 to a second resolution higher than the first resolution. This is unrelated to the method of obtaining the second-resolution image P20, but rather means that the second-resolution image P20 is identical or substantially identical in content to the image obtained by converting the first-resolution image P10 to a second resolution higher than the first resolution. To elaborate further, it can be said that although the second-resolution image P20 displays the image of the object 100 with the same composition as the first-resolution image P10, it is an image with a higher resolution than the first-resolution image P10. In other words, although the object 100 in the second-resolution image P20 has the same composition as the first-resolution image P10, it is displayed at a second resolution, which is higher than the first resolution. In other words, the first-resolution image P10 is equivalent to converting the second-resolution image P20 to a first resolution, which is lower than the second resolution. This is unrelated to the method of obtaining the first-resolution image P10; it means that the content of the first-resolution image P10 is the same or substantially the same as the image obtained by converting the second-resolution image P20 to a first resolution, which is lower than the second resolution. To elaborate further, it can be said that although the first-resolution image P10 displays the image of the object 100 with the same composition as the second-resolution image P20, it is an image with a lower resolution than the second-resolution image P20. In other words, although the object 100 in the first-resolution image P10 has the same composition as the second-resolution image P20, it is displayed at a lower resolution than the second resolution. In the relationship between the first-resolution image P10 and the second-resolution image P20, the first-resolution image P10 is a low-resolution image of the object 100, and the second-resolution image P20 is a high-resolution image of the object 100. For example, the first resolution is 22×26 pixels, and the second resolution is 112×96 pixels. The difference image P30 is the image representing the difference between the first-resolution image P10 and the second-resolution image P20. Although the object 100 in the first-resolution image P10 and the second-resolution image P20 have the same composition, they are displayed at different resolutions.Therefore, it can be said that the difference image P30 is an image showing the difference in appearance of the object 100 caused by the difference in resolution between the first resolution image P10 and the second resolution image P20. The difference in resolution can be considered as the difference in distance to the object 100. Therefore, it can be said that the difference image P30 represents the information lost from the second resolution image P20 in the first resolution image P10 due to the increased distance to the object 100. Regarding the first model unit M11, details will be described later. The first model unit M11 is obtained using a model constructed with a neural network from a learned model generated by executing machine learning (supervised learning) using a learning dataset, which takes the first resolution image P10 as input and the second resolution image P20 and the difference image P30 as correct answers. In this embodiment, the first model unit M11 can also be described as a super-resolution network because it outputs the second resolution image P20, which has a higher resolution than the first resolution image P10.

[0041] The second model unit M12 is a model that has undergone learning (learned model) and outputs feature quantity F11 of object 100 as input to the second resolution image P20 and the difference image P30. The feature quantity F11 of object 100 is information used in the arithmetic circuit 13 to recognize (identify) object 100. The feature quantity F11 is represented, for example, by an n-dimensional vector (n is any integer). Details regarding the second model unit M12 will be described later. The second model unit M12 is obtained from a learned model generated by executing machine learning (supervised learning) using a learning dataset constructed with a neural network, the learning dataset being input to the second resolution image P20 and the difference image P30, and the result of recognizing object 100 as the correct answer. In this embodiment, since the first resolution image P10 and the second resolution image P200 are images displaying the face of object 100, the second model unit M12 can also be described as a face recognition network.

[0042] After learning model M10, the output of the first model unit M11 is connected to the input of the second model unit M12. Therefore, for the input of the first resolution image P10, the second resolution image P20 and the difference image P30 output from the first model unit M11 are input to the second model unit M12, and the feature quantity F11 is output from the second model unit M12.

[0043] The learned model M10 described above is generated by the learning system 20, which will be discussed later. The method for generating the learned model M10 (the learning method) will be explained in detail in the "(1-2-2) Learning Method" section, which will be discussed later.

[0044] Database D10 is a database related to objects 100 identified (distinguished) in recognition system 10. For each object 100 identified in recognition system 10, the database includes a correspondence between the object 100's identification information and a reference feature quantity F12. The identification information includes an identification number assigned to the object 100. The reference feature quantity F12 is used in comparison with feature quantity F11 obtained through the learned model M10. In this embodiment, the object 100 is a person, and in the case of person A, database D10 contains identification information for person A, and feature quantity F12 corresponds to feature quantity F11 obtained from an image of person A displayed through the learned model M10. Feature quantity F12 is, for example, a vector with the same dimension as feature quantity F11.

[0045] The arithmetic circuit 13 is a circuit that controls the operation of the recognition system 10. The arithmetic circuit 13 is connected to the interface 11 and can access the storage device 12 (that is, it can access the learned model M10 and the database D10). The arithmetic circuit 13 can be implemented, for example, in a computer system including one or more processors (microprocessors) and one or more memories. The function of the arithmetic circuit 13 is implemented by executing a program (stored in one or more memories or storage devices 12) by one or more processors. The program is pre-recorded in the storage device 12, but it can also be provided via telecommunications circuits such as the Internet or recorded on a non-temporary recording medium such as a memory card.

[0046] The arithmetic circuit 13 is configured to perform acquisition processing S11, inference processing S12, and output processing S13 (see reference). Figure 3 ).

[0047] The acquisition process S11 is the process of acquiring the first resolution image P10 as an object image. In this embodiment, the acquisition process S11 indirectly acquires the first resolution image P10 as an object image. The recognition system 10 is connected to the camera system 200 via interface 11. The acquisition process S11 extracts the first resolution image P10 from the image obtained from the camera system 200 via interface 11 to obtain the object image. Specifically, in the acquisition process S11, the area of ​​the object 100 that is displayed is extracted from the image from the camera system 200 as the object image (first resolution image P10). In this embodiment, since the first resolution image P10 is an image of the face of the object 100, the image of the face of the object 100 is extracted from the image from the camera system 200 in the acquisition process S11. The first resolution image P10 can also be directly acquired as an object image in the acquisition process S11. For example, if the first resolution image P10 is given from the camera system 200 to the recognition system 10 through the interface 11, then in the acquisition process S11, the given first resolution image P10 can be acquired as an object image while remaining unchanged.

[0048] The inference process S12 involves feeding the object image acquired in the acquisition process S11 to the learned model M10, so that the learned model M10 can calculate the feature quantity F11 of the object 100 displayed in the object image. Specifically, in this embodiment, the inference process S12 includes recognizing the object 100 based on the feature quantity F11 of the object 100 displayed in the object image. More specifically, in the inference process S12, the consistency (similarity) of the feature quantity F11 and feature quantity F12 is determined by comparing them with the feature quantity F12 in the database D10 of the storage device 12, and the object 100 displayed in the object image is recognized based on this similarity. In this embodiment, since the feature quantity F11 and feature quantity F12 are n-dimensional vectors, the consistency between the feature quantity F11 and feature quantity F12 can be evaluated using cosine similarity or Euclidean distance, etc. As an example, the inference process S12 identifies the object 100 associated with the feature quantity F12 among multiple feature quantities F12 that has the highest consistency with feature quantity F11 and whose consistency is above the threshold as the object 100 displayed in the object image.

[0049] Output processing S13 is the processing of the result of output inference processing S12. In this embodiment, the result of inference processing S12 includes the recognition result R21 of the object 100 displayed in the object image. The recognition system 10 is connected to the control system 300 via the communication device of interface 11. In output processing S13, the result of inference processing S12 is output to the control system 300 via interface 11. In addition, in output processing S13, the result of inference processing S12 can be output to the user via the input / output device of interface 11. For example, in output processing S13, a screen for outputting the result of inference processing S12 can be displayed via the input / output device of interface 11.

[0050] As described above, the recognition system 10 includes a storage device 12 for storing the learned model M10 and an arithmetic circuit 13 capable of accessing the storage device 12. The learned model M10 includes a first model unit M11 and a second model unit M12. The first model unit M11 is a model that has been learned to output a second resolution image P20 and a difference image P30, which are equivalent to converting the first resolution image P10 into a second resolution image that is higher than the first resolution, in response to an input of a first resolution image P10 of the object 100 displayed at a first resolution. The difference image P30 is an image that is equivalent to the difference between the first resolution image P10 and the second resolution image P20. The second model unit M12 is a model that has been learned to output a feature quantity F11 of the object 100, in response to an input of the second resolution image P20 and the difference image P30 from the first model unit M11. Figure 3 As shown, the arithmetic circuit 13 performs: an acquisition process S11, acquiring the first resolution image P10 as an object image; and an inference process S12, providing the object image acquired in the acquisition process S11 to the learned model M10, so that the learned model M10 calculates the feature quantity F11 of the object 100 displayed in the object image. According to this recognition system 10, the accuracy of recognizing the object 100 can be improved.

[0051] In other words, it can be said that the identification system 10 performs the following method (identification method). The identification method is performed by the arithmetic circuit 13, which is capable of accessing the storage device 12, and includes an acquisition process S11 and an inference process S12. According to such an identification method, the accuracy of identifying the object 100 can be improved in the same way as the identification system 10.

[0052] The identification system 10 is implemented using the arithmetic circuit 13. That is, the method (identification method) executed by the identification system 10 can be implemented by the arithmetic circuit 13 executing a program. This program is a computer program used to cause the arithmetic circuit 13 to execute the above-described identification method. According to such a program, the accuracy of identifying the object 100 can be improved in the same way as the identification system 10.

[0053] The learned model M10 used in the recognition system 10 includes a first model unit M11 and a second model unit M12. The first model unit M11 is a model that has been learned to output a second-resolution image P20 and a difference image P30, which are equivalent to converting the first-resolution image P10 into a second-resolution image that is higher than the first resolution, in response to an input of a first-resolution image P10 displaying the object 100 at a first resolution. The difference image P30 is an image representing the difference between the first-resolution image P10 and the second-resolution image P20. The second model unit M12 is a model that has been learned to output a feature quantity F11 of the object 100, in response to an input of the second-resolution image P20 and the difference image P30 from the first model unit M11. Based on this learned model M10, the accuracy of recognizing the object 100 can be improved.

[0054] (1-2-2) Learning Methods

[0055] Next, the learning method used to generate the learned model M10 used in the recognition system 10 will be explained. For example... Figure 4 As shown, the learning method includes preparation process S21 and learning process S22.

[0056] Preparing to process S21 is preparation Figure 5 The processing of model M20 is shown. Model M20 is used to generate the learned model M10. Model M20 includes a first model unit M21, a second model unit M22, a third model unit M23, and a fourth model unit M24.

[0057] The first model unit M21 is a model that takes a first-resolution image P10 of an object 100 displayed at a first resolution as input, and outputs a second-resolution image P20, which is equivalent to converting the first-resolution image P10 into a second-resolution image that is higher than the first resolution, and a difference image P30, which is equivalent to the difference between the first-resolution image P10 and the second-resolution image P20. The first model unit M21 has a neural network structure, and in this embodiment, it has a convolutional neural network (CNN) structure. A CNN has an appropriate number of convolutional layers, pooling layers, activation functions, fully concatenated layers, etc., between the input and output layers. In particular, the first model unit M21 has a structure for generating a super-resolution network such as a first model unit M11 that has learned the model M10. For example, the construction of the first model unit M21 can utilize the reference (Christian Ledig, et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, [online], September 15, 2016, [retrieved September 7, 2020], Cornell University, Internet <URL: https: / / arxiv.org / abs / 1609.04802>). Figure 4 The construction of the generative network is shown in the figure.

[0058] The second model unit M22 is a model for outputting feature quantities F11 of the object 100 as input to the second resolution image P20 and the difference image P30 from the first model unit M21. The second model unit M22 has a neural network structure, and in this embodiment, it has a convolutional neural network (CNN) structure. In particular, the second model unit M22 has a structure for generating a face recognition network such as the second model unit M12 that has completed learning the model M10. For example, the structure of the second model unit M22 can utilize the face recognition network structure illustrated in Non-Patent Document 1. In this embodiment, the second resolution image P20 is input to the input layer of the second model unit M22, while the difference image P30 is not input to the input layer of the second model unit M22. The difference image P30 is converted into parameters through a convolutional layer or the like, different from the input layer of the second model unit M22, for use in the second model unit M22. Regarding parameters, examples include parameters (γ, β used in batch normalization) for converting normalized data of the image input to the input layer with inherent scale and offset. For example, the reference (TaesungPark, et al., "Semantic Image Synthesis with Spatially-Adaptive Normalization", [online], March 18, 2019, [retrieved September 7, 2020], Cornell University, Internet <URL: https: / / arxiv.org / abs / 1903.07291) discloses the use of tensors obtained by convolution of segment masks in convolutional layers as parameters γ and β for batch normalization.

[0059] The third model unit M23 is a model that takes the feature quantity F11 of the object 100 as input from the second model unit M22 and outputs the recognition result (recognition result) R21 of the object 100. The third model unit M23 is a recognizer, and can use algorithms such as K-nearest neighbors (KNN) or support vector machine (SVM).

[0060] The fourth model unit M24 is a model that takes the second resolution image P20 as input and outputs a determination result (determination result) R22 of whether the second resolution image P20 is an image generated by the first model unit M21. The fourth model unit M24 has a neural network structure, and in this embodiment, it has a convolutional neural network (CNN) structure. The fourth model unit M24 has a structure for generating a recognition network. The fourth model unit M24 is used together with the first model unit M21 to form a generative adversarial network (GAN). As the structure of the fourth model unit M24, the above-mentioned references can be utilized. Figure 4 The structure of the identification network is shown in the figure.

[0061] like Figure 4 As shown, the preparation process S21 includes the generation step S210 and the pre-learning step S211.

[0062] Generation step S210 is the step of generating the learning dataset (first learning dataset) D21. The first learning dataset D21 is a learning dataset that takes a first resolution image P10 as input and a second resolution image P20 and a difference image P30 as the correct answers. In other words, generation step S210 is a learning dataset generation method used to generate the learning dataset D21.

[0063] like Figure 4 As shown, the generation step S210 includes a first step S210a, a second step S210b, a third step S210c, and a fourth step S210d.

[0064] The first step S210a is the step of acquiring a reference image of the object 100 to be displayed. The reference image is an image of the object 100 displayed at a second resolution. That is, it can also be said that the first step S210a is the step of acquiring a second-resolution image P20. However, the reference image is not an image obtained by converting the first-resolution image P10 to a second resolution higher than the first resolution, but rather is conceived as an image obtained by photographing the object 100 at a second resolution, or an image obtained by converting a third resolution (higher than the second resolution) to a second resolution. In the following text, as needed, the second-resolution image P20 obtained by the first model section M11 of the model M10 after learning will be represented by reference numeral P21. The second-resolution image P20 obtained in the first step S210a will be represented by reference numeral P22. The second-resolution image P20 obtained by the first model section M21 of the model M20 will be represented by reference numeral P23.

[0065] The second step S210b is to convert the reference image (second resolution image P22) into a low-resolution image with a resolution lower than the reference image. In this embodiment, the reference image is converted into a low-resolution image with a first resolution lower than the second resolution. That is, it can also be said that the second step S210b is to convert the second resolution image P22 obtained in the first step S210a into a first resolution image to generate a first resolution image P10. Here, the low-resolution image is not an image obtained by converting a fourth resolution (which is lower than the first resolution) to a first resolution image obtained by photographing the object 100 at the first resolution or a fourth resolution (which is lower than the first resolution) to the first resolution image, but rather is conceived as an image obtained by converting the second resolution image P22 into a first resolution image with a resolution lower than the second resolution. In the following text, in order to distinguish between the first resolution image P10, which is a low-resolution image generated from the second resolution image P22, and the first resolution image P10, such as an image obtained by photographing the object 100 at the first resolution, the latter is indicated by reference numeral P11, and the former is indicated by reference numeral P12, as needed. In the second step 210b, a first resolution image (low-resolution image) P12 is generated, whose image size is smaller than the second resolution image (reference image) P22 obtained in the first step 210a. The conversion from the second resolution image P22 to the first resolution image P12 can be achieved by known methods such as average interpolation.

[0066] The third step, S210c, is to generate a difference image P30, which is equivalent to the difference between the reference image (second resolution image P22) and the low-resolution image (first resolution image P12). Alternatively, the third step, S210c, is to generate the difference image P30 from the second resolution image P22 obtained in the first step, S210a, and the first resolution image P12 generated in the second step, S210b. In the third step, S210c, the first resolution image P12 generated in the second step, S210b, is enlarged to the same image size as the second resolution image P22 obtained in the first step, S210a. In the third step, S210c, the difference image P30 is generated based on the difference in pixel values ​​between the enlarged first resolution image P12 and the second resolution image P22 obtained in the first step, S210a. Therefore, the difference between pixel values ​​can be easily obtained from the first resolution image P11 and the second resolution image P22. The enlargement of the first resolution image P12 can be achieved using known methods such as bicubic interpolation. The range of pixel values ​​in the difference image P30 is narrower than the range of pixel values ​​in the first resolution image P11 and the second resolution image P22. For example, if the range of pixel values ​​in the first resolution image P11 and the second resolution image P22 is 0 to 255, the range of pixel values ​​in the difference image P30 can be set to 0 to 15. That is, in the third step S210c, the grayscale of the image where the difference between the magnified first resolution image P12 and the second resolution image P22 obtained in the first step S210a remains unchanged is reduced to generate the difference image P30. Thus, the portion where there is a difference between the first resolution image P12 and the second resolution image P22 is clearly reflected in the difference image P30. In the following text, the difference image P30 obtained by the first model unit M11 after learning the model M10 will be represented by reference numeral P31, as needed. The difference image P30 generated in the third step 210c will be represented by reference numeral P32. The differential image P30 obtained by the first model part M21 of model M20 is denoted by the attached figure label P33.

[0067] The fourth step, S210d, is the step of generating the (first) learning dataset D21. The first learning dataset D21 is a learning dataset that takes a low-resolution image (first resolution image P12) as input and a reference image (second resolution image P22) and a difference image P32 as the correct answers. In other words, the fourth step, S210d, can also be described as the step of generating a learning dataset that takes the first resolution image P12 generated in the second step, S210b, as input, and the second resolution image P22 obtained in the first step, S210a, and the difference image P32 generated in the third step, S210c, as the correct answers.

[0068] In the pre-learning step S211, the first model unit M21 learns the relationship between the first resolution image P10 (P12), the second resolution image P20 (P22), and the difference image P30 (P32) using the learning dataset D21 generated in the generation step S210. Thus, the first model unit M21 learns the input for the first resolution image P10 (P12) and outputs the second resolution image P23 and the difference image P33. The pre-learning step S211 may also include pre-learning for the second model unit M22 and the third model unit M23 as needed. Alternatively, the second model unit M22 and the third model unit M23 may learn the relationship between the second resolution image P20, the difference image P30, and the recognition result R21 using a learning dataset with the second resolution image P20 and the difference image P30 as input and the recognition result R21 as the correct answer. When the pre-learned model is used as the second model unit M22 and the third model unit M23, the pre-learning of the second model unit M22 and the third model unit M23 in the pre-learning step S211 is not required. The pre-learning step S211 may also include pre-learning of the fourth model unit M24 as needed. For example, the fourth model unit M24 may be pre-trained to some extent using the second resolution image P22 obtained in the first step 210a and the second resolution image P23 generated in the first model unit M21. When the pre-learned model is used as the fourth model unit M24, the pre-learning of the fourth model unit M24 in the pre-learning step S211 is not required.

[0069] Learning step S21 includes using machine learning with the (second) learning dataset D22 to enable model M20 to learn the relationship between the first resolution image P12 and the feature quantity F21 of the object 100 displayed in the first resolution image P12. The second learning dataset D22 is a learning dataset that takes the first resolution image P12 as input and the recognition result (recognition result R21) of the object 100 displayed in the first resolution image P12 as the correct answer. Learning step S21 may also include performing additional learning of at least one of the first model unit M21 and the fourth model unit M24 based on the judgment result (judgment result) R22 output from the fourth model unit M24.

[0070] In this way, the learned parameters of the first model unit M21 and the second model M22 can be obtained from the machine learning model M20. Using the learned parameters of the first model unit M21 and the second model M22, along with the inference procedures for the first model unit M21 and the second model M22, an assembly can be performed. Figure 2 The model M10 is shown after learning is complete.

[0071] The learning methods described above, for example, can be achieved through... Figure 6The learning system 20 shown is executed. The learning system 20 includes an interface 21, a storage device 22, and a computing circuit 23. In this embodiment, the learning system 20 is implemented using a server.

[0072] Interface 21 is used for inputting and outputting information to and from the learning system 20. Interface 21 includes an input / output device 211 and a communication device 212. The input / output device 211 functions as an input device for inputting information from a user and as an output device for outputting information to a user. The input / output device 211 has one or more human-machine interfaces. Examples of human-machine interfaces include input devices such as keyboards, pointing devices (mice, trackballs, etc.), and touchpads; output devices such as displays and speakers; and input / output devices such as touch panels. The communication device 212 can communicatively connect to external devices or systems. The communication device 212 has one or more communication interfaces. The communication device 212 can connect to a communication network and has the function of communicating through the communication network. The communication device 212 operates according to a predetermined communication protocol. The predetermined communication protocol can be selected from various known wired and wireless communication standards.

[0073] Storage device 22 is used to store information utilized by arithmetic circuit 23 and information generated by arithmetic circuit 23. Storage device 22 includes one or more storage units (non-temporary storage media). The storage unit may be, for example, any of a hard disk drive, optical disk drive, and solid-state drive (SSD). In addition, the storage unit may be any of an internal type, an external type, and a NAS (network-attached storage) type.

[0074] The information stored in storage device 22 includes model M20 and learning data D20. The learning data D20 includes a first learning dataset D21 and a second learning dataset D22. Figure 6 The diagram shows the complete state of the model M20 and the learning data D20 stored in the storage device 22. The model M20 and the learning data D20 do not need to be stored in the storage device 22 all the time, but only when needed in the arithmetic circuit 23.

[0075] The arithmetic circuit 23 is a circuit that controls the operation of the learning system 20. The arithmetic circuit 23 is connected to the interface 21 and can access the storage device 22 (that is, it can access the model M20 and the learning data D20). The arithmetic circuit 23 can be implemented, for example, in a computer system including one or more processors (microprocessors) and one or more memories. The function of the arithmetic circuit 23 is achieved by executing a program (stored in one or more memories or storage devices 22) by one or more processors. The program is pre-recorded in the storage device 22, or it can be provided via telecommunications circuits such as the Internet or recorded on a non-temporary recording medium such as a memory card.

[0076] The operational circuit 23 is configured to execute the learning method (including preparation step S21 and learning step S22).

[0077] As described above, the learning system 20 includes a computation circuit 23. The computation circuit 23 executes a learning method including a preparation process S21 and a learning step S22. In the preparation process S21, a model M20 is prepared, and in the learning step S22, machine learning is performed using the model M20 prepared in the preparation step S21. The model M20 includes a first model unit M21, a second model unit M22, and a third model unit M23. The first model unit M21 is a model that, for input to a first-resolution image P20 of the object 100 displayed at a first resolution, outputs a second-resolution image P20 equivalent to an image obtained by converting the first-resolution image P20 to a second resolution higher than the first resolution, and a difference image P30 equivalent to the difference between the first-resolution image P10 and the second-resolution image P20. The second model unit M21 is a model that, for input to the second-resolution image P20 and the difference image P30 from the first model unit M22, outputs a feature quantity F21 of the object 100. The third model unit M23 is a model that outputs a recognition result R21 of the object 100 as input to the feature quantity F21 of the object 100 from the second model unit M22. The learning step S22 includes using machine learning with a learning dataset D21, which takes a first resolution image P10 as input and the recognition result R21 of the object 100 displayed in the first resolution image P10 as the correct answer, to enable model M20 to learn the relationship between the first resolution image P10 and the feature quantity F21 of the object 100 displayed in the first resolution image P10. According to this learning system 20, the accuracy of object 100 recognition can be improved.

[0078] In other words, it can be said that the learning system 20 performs the following method (learning method). The learning method is executed by the arithmetic circuit 23 and includes a preparation step S21 and a learning step S22. According to this learning method, the accuracy of recognizing the object 100 can be improved in the same way as the learning system 20.

[0079] The learning system 20 is implemented using the arithmetic circuit 23. That is, the method (learning method) executed by the learning system 20 can be implemented by the arithmetic circuit 23 executing a program. This program is a computer program used to cause the arithmetic circuit 23 to execute the aforementioned learning method. Based on such a program, the accuracy of identifying the object 100 can be improved in the same way as the learning system 20.

[0080] (1-3) Performance Evaluation

[0081] Experiments were conducted to evaluate the performance of the trained model M10 in face recognition. In the experiments, to relatively evaluate the performance of the trained model M10 in face recognition, [the following steps were taken]. Figure 2 In the learned model M10 shown, the performance of face recognition was also evaluated using a learned model that omitted the difference image P30. This learned model omitting the difference image P30 is considered to be the learned model of Non-Patent Document 1. For the performance evaluation data for face recognition, the LFW dataset (http: / / vis-www.cs.umass.edu / lfw / ), one of the publicly available benchmarks for face recognition, was used. 3000 sets of image pairs of the same person but different categories and 3000 sets of image pairs of different people were prepared, totaling 6000 sets, and the correct resolution rate was evaluated. In the learned model of Non-Patent Document 1, the correct resolution rate was 98.64%, compared to 98.82% for the learned model M10. This demonstrates that the error rate can be improved by 10% using the learned model M10.

[0082] (2) Variation

[0083] The embodiments disclosed herein are not limited to the embodiments described above. Various modifications can be made to the above embodiments, depending on the design, etc., as long as they achieve the technical problem of this disclosure. Modifications to the above embodiments are listed below. The modifications described below can be appropriately combined and applied.

[0084] In a variation, within the recognition system 10, the computation circuit 13 may generate a reuse model by performing additional learning on the learned model M10, and then use the reuse model to perform inference processing S12. In other words, the computation circuit 13 may also perform learning processing that involves additional learning (relearning) of the learned model M10.

[0085] In a variation, in the recognition system 10, the storage device 12 may also store a distillation model instead of the learned model M10. The distillation model is a model generated by distilling the learned model M10. In this case, the recognition system 10 includes a storage device 12 for storing the distillation model and an arithmetic circuit 13 capable of accessing the storage device 12. The arithmetic circuit 13 executes an acquisition process S11 and an inference process S12. The acquisition process S11 is the process of acquiring the first resolution image P10 as an object image. The inference process S12 is the process of providing the object image acquired in the acquisition process S11 to the distillation model so that the distillation model calculates the feature quantity F11 of the object 100 displayed in the object image. Even in this case, it is possible to improve the accuracy of recognizing the object 100.

[0086] In one variation, in the recognition system 10, the acquisition process S11 displays a screen for inputting a first resolution image P10, for example, through the input / output device of interface 11, allowing the user to input the first resolution image P10 according to the instructions on the screen. The input of the first resolution image P10 may include not only inputting the first resolution image P10 from an external device or system into the recognition system 10, but may also include data determined from data stored in the recognition system 10 to be used as the first resolution image P10. The inference process S12 may not necessarily proceed until the recognition result R11 is obtained; the feature quantity F11 may also be used as the result of the inference process. The output process S13 may also display a screen for outputting the result of the inference process, for example, through the input / output device of interface 11.

[0087] In a variation, in the learning method, the second-resolution image P23 and the difference image P33 from the first model unit M21 of model M20 can also be input as two-channel images to the input layer of the second model unit M22. In this case, the second-resolution image P21 and the difference image P31 from the first model unit M11 of the learned model M10 can also be input as two-channel images to the input layer of the second model unit M12.

[0088] In a variation, the difference image P30 may also be an image in which the difference between the pixel values ​​of the first resolution image P10 and the second resolution image P20 is itself set as the pixel values.

[0089] In a variation, preparation step S21 is not mandatory, as the training dataset D21 can also be prepared in advance. Instead of end-to-end learning of model M20 in learning step S22, it can be set up as performing separate learning of the first model part M21 through the fourth model part M24 of model M20. Model M20 may also exclude the fourth model part M24. In other words, learning using a GAN is not mandatory.

[0090] In one variation, it is not necessary for the recognition system 10 and the learning system 20 to be implemented through different computer systems. The recognition system 10 and the learning system 20 can also be implemented through a single computer system.

[0091] In one variation, it is not necessary for both the interface 11 of the identification system 10 and the interface 21 of the learning system 20 to have input / output devices and communication devices.

[0092] In one variation, each of the identification system 10 and the learning system 20 can also be implemented using a computer system with multiple servers or the like. That is, it is not necessary for multiple functions (components) of each of the identification system 10 and the learning system 20 to be integrated into a single enclosure; the components of each of the identification system 10 and the learning system 20 can also be distributed across multiple enclosures. Furthermore, at least a portion of the functions of each of the identification system 10 and the learning system 20, such as a portion of the functions of the computational circuits 13 and 23, can be implemented using the cloud (cloud computing) or the like.

[0093] (3) aspects

[0094] As can be seen from the above embodiments and variations, this disclosure includes the following aspects. In order to clearly show the correspondence with the embodiments, reference numerals are provided in parentheses below.

[0095] The first aspect is a recognition system (10) comprising: a storage device (12) for storing a learned model (M10); and an arithmetic circuit (13) capable of accessing the storage device (12). The learned model (M10) includes a first model unit (M11) and a second model unit (M12). The first model unit (M10) is a model that has been learned to output a second resolution image (P20) and a difference image (P30) in response to an input of a first resolution image (P10). The first resolution image (P10) is an image of an object (100) displayed at a first resolution. The second resolution image (P20) is equivalent to an image obtained by converting the first resolution image (P10) to a second resolution higher than the first resolution. The difference image (P30) is equivalent to the difference between the first resolution image (P10) and the second resolution image (P20). The second model unit (M12) is a model that has been learned as follows: given the second resolution image (P20) and the difference image (P30) as inputs, it outputs the feature quantity (F11) of the object. The arithmetic circuit (13) performs an acquisition process (S11) and an inference process (S12). In the acquisition process (S11), the first resolution image (P10) is acquired as an object image. In the inference process (S12), the object image acquired in the acquisition process (S11) is given to the learned model (M10) so that the learned model (M10) calculates the feature quantity (F11) of the object (100) displayed in the object image. According to this aspect, the accuracy of object (100) recognition can be improved.

[0096] The second aspect is based on the recognition system (10) of the first aspect. In the second aspect, the inference process (S12) includes recognizing the object based on the feature quantity (F11) of the object displayed in the object image. According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0097] The third aspect is an identification system (10) based on the first or second aspect. In the third aspect, the arithmetic circuit (13) performs an output process (S13) that outputs the result of the inference process (S12). According to this aspect, the result of the inference process (S12) can be displayed.

[0098] The fourth aspect is a recognition method executed by an arithmetic circuit (13) capable of accessing a storage device (12) storing a learned model (M10). The learned model (M10) includes a first model unit (M11) and a second model unit (M12). The first model unit (M10) is a model that has been learned to output a second resolution image (P20) and a difference image (P30) as input to a first resolution image (P10). The first resolution image (P10) is an image of an object (100) displayed at a first resolution. The second resolution image (P20) is equivalent to an image obtained by converting the first resolution image (P10) to a second resolution higher than the first resolution. The difference image (P30) is equivalent to the difference between the first resolution image (P10) and the second resolution image (P20). The second model unit (M12) is a model that has been learned to output a feature quantity (F11) of the object as input to the second resolution image (P20) and the difference image (P30). The recognition method includes an acquisition process (S11) and an inference process (S12). In the acquisition process (S11), the first resolution image (P10) is acquired as an object image. In the inference process (S12), the object image acquired in the acquisition process (S11) is given to the learned model (M10) so that the learned model (M10) calculates the feature quantity (F11) of the object (100) appearing in the object image. According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0099] The fifth aspect is a procedure for causing the arithmetic circuit (13) to execute a recognition method based on the fourth aspect. According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0100] The sixth aspect is a learning method, comprising: a preparation step (S21) of preparing a model (M20); and a learning step (S22) of performing machine learning using the model (M20) prepared in the preparation step (S21). The model (M20) includes a first model unit (M21), a second model unit (M22), and a third model unit (M23). The first model unit (M21) is a model for outputting a second resolution image (P20) and a difference image (P30) as input to a first resolution image (P10). The first resolution image (P10) is an image displaying an object (100) at a first resolution. The second resolution image (P20) is equivalent to an image obtained by converting the first resolution image (P10) to a second resolution higher than the first resolution. The difference image (P30) is an image equivalent to the difference between the first resolution image (P10) and the second resolution image (P20). The second model unit (M22) is a model that outputs the feature quantity (F21) of the object as input to the second resolution image (P20) and the difference image (P30) from the first model unit (M21). The third model unit (M23) is a model that outputs the recognition result (R21) of the object (100) as input to the feature quantity (F21) of the object (100) from the second model unit (M22). The learning step (S22) includes machine learning using a learning dataset (D22) with the first resolution image (P10) as input and the recognition result (R21) of the object (100) displayed in the first resolution image (P10) as the correct answer, so that the model (M20) learns the relationship between the first resolution image (P10) and the feature quantity (F21) of the object displayed in the first resolution image (P10). According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0101] The seventh aspect is based on the learning method of the sixth aspect. In the seventh aspect, the preparation step (S21) includes a generation step (S210) and a pre-learning step (S211). The generation step (S210) is a step of generating a learning dataset (D21) with the first resolution image (P10) as input and the second resolution image (P20) and the difference image (P30) as correct answers. The pre-learning step (S211) is a step of using the learning dataset (D21) generated in the generation step (S210) to enable the first model unit (M21) to learn the relationship between the first resolution image (P10), the second resolution image (P20), and the difference image (P30). According to this aspect, the accuracy of identifying the object (100) can be improved.

[0102] The eighth aspect is based on the recognition system (10) of the seventh aspect. In the eighth aspect, the generation step (S210) includes a first step (S210a), a second step (S210b), a third step (S210c), and a fourth step (S210d). The first step (S210a) is to acquire the second resolution image (P20). The second step (S210b) is to convert the second resolution image (P20) acquired in the first step (S210a) into an image of the first resolution to generate the first resolution image (P10). The third step (S210c) is to generate the difference image (P30) from the second resolution image (P20) acquired in the first step (S210a) and the first resolution image (P10) generated in the second step (S210b). The fourth step (S210d) is to generate a learning dataset (D21) that takes the first resolution image (P10) generated in the second step (S210b) as input, the second resolution image (P20) prepared in the first step (210a), and the difference image (P30) generated in the third step (S210c) as the correct answers. According to this aspect, the accuracy of identifying the object (100) can be improved.

[0103] The ninth aspect is based on the learning method of the eighth aspect. In the ninth aspect, the third step (S210c) enlarges the first resolution image (P10) generated in the second step (S210b) to the same image size as the second resolution image (P20) acquired in the first step (S210a). In the third step (S210c), the difference image (P30) is generated based on the difference in pixel values ​​between the enlarged first resolution image (P10) and the second resolution image (P20) acquired in the first step (S210a). According to this aspect, the accuracy of identifying the object (100) can be improved.

[0104] The tenth aspect is based on the learning method of the ninth aspect. In the tenth aspect, the range of pixel values ​​in the differential image (P30) is narrower than the range of pixel values ​​in the magnified first resolution image (P10) and the second resolution image (P20) acquired in the first step (S210a). According to this aspect, the accuracy of identifying the object (100) can be improved.

[0105] The eleventh aspect is a learning method based on any one of the sixth to tenth aspects. In the eleventh aspect, the model (M20) includes a fourth model unit (M24), which outputs a determination result (R22) as to whether the second resolution image (P20) is an image generated by the first model unit (M21) as input to the second resolution image (P20). The learning step (S22) includes performing additional learning of at least one of the first model unit (M21) and the fourth model unit (M24) based on the determination result (R22) output from the fourth model unit (M24). According to this aspect, the accuracy of identifying the object (100) can be improved.

[0106] The twelfth aspect is a learned model (M10) comprising a first model unit (M11) and a second model unit (M12). The first model unit (M10) is a model that has been learned to output a second resolution image (P20) and a difference image (P30) as input to a first resolution image (P10). The first resolution image (P10) is an image displaying an object (100) at a first resolution. The second resolution image (P20) is an image obtained by converting the first resolution image (P10) to a second resolution higher than the first resolution. The difference image (P30) is an image representing the difference between the first resolution image (P10) and the second resolution image (P20). The second model unit (M12) is a model that has been learned to output a feature quantity (F11) of the object as input to the second resolution image (P20) and the difference image (P30). According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0107] The thirteenth aspect is a distillation model generated by distillation based on the learned model (M10) of the twelfth aspect. According to this aspect, the accuracy of identifying the object (100) can be improved.

[0108] The fourteenth aspect is a recognition system (10) comprising: a storage device (12) storing a distillation model based on the thirteenth aspect; and a computational circuit (13) capable of accessing the storage device (12). The computational circuit (13) performs an acquisition process (S11) and an inference process (S12). The acquisition process (S11) is a process of acquiring a first resolution image (P10) as an object image. The inference process (S12) is a process of providing the object image acquired in the acquisition process (S11) to the distillation model so that the distillation model calculates the feature quantity (F11) of the object (100) displayed in the object image. According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0109] The fifteenth aspect is a method for generating a learning dataset, comprising a first step (S210a), a second step (S210b), a third step (S210c), and a fourth step (S210d). The first step (S210a) is to acquire a reference image (P22) of the object to be displayed (100). The second step (S210b) is to convert the reference image (P22) into a low-resolution image (P12) with a lower resolution than the reference image (P22). The third step (S210c) is to generate a difference image (P30) corresponding to the difference between the reference image (P22) and the low-resolution image (P12). The fourth step (S210d) is to generate a learning dataset (D11) with the low-resolution image (P12) as input and the reference image (P22) and the difference image (P30) as correct answers. According to this aspect, the accuracy of recognizing the object (100) can be improved.

[0110] It should be noted that the second or third aspect can also be appropriately modified to apply to the fourth and fourteenth aspects.

[0111] (4) Terminology

[0112] In this disclosure, the terms related to machine learning are defined and used as follows.

[0113] "Learning-complete model" refers to an "inference procedure" that incorporates "learning-complete parameters".

[0114] "Learned parameters" refer to the parameters (coefficients) obtained from learning using the training dataset. These parameters are generated by mechanically adjusting the training dataset in a learning program for a specific purpose. Although learned parameters are adjusted to suit the learning objective, they are merely simple parameters (numerical values, etc.) on their own; they only function as a learned model when incorporated into the inference process. For example, in deep learning, the primary parameters among the learned parameters, such as those used to assign weights to connections between nodes, fall into this category.

[0115] "Inference procedure" refers to a procedure that enables the application of learned parameters to output a certain result for an input. For example, it is a procedure that defines a series of operations for applying learned parameters obtained as a learning result to an image given as input and outputting a result (authentication, judgment) for that image.

[0116] "Learning datasets," also known as training datasets, refer to data that has undergone secondary processing. To facilitate the analysis of learning methods used as objects, these datasets are generated by preprocessing the original data, such as removing missing or outlier values, adding different data such as label information (correct answer data), or combining these data through transformation and processing. Sometimes, learning datasets also include data that has been transformed from the original data, i.e., "virtualized" data.

[0117] "Raw data" refers to data initially obtained from users, suppliers, other businesses, research institutions, etc., which has been transformed and processed to be read into a database.

[0118] "Learning program" refers to a program that executes an algorithm to find certain rules from a learning dataset and generate a model that represents those rules. Specifically, it refers to a program that specifies the order in which the computer executes the program in order to achieve learning through the learning method employed.

[0119] "Additional learning" refers to regenerating learned parameters by applying different learning datasets to an existing learned model for further learning.

[0120] "Reuse model" refers to an inference procedure that incorporates learned parameters that are regenerated through supplementary learning.

[0121] "Distillation" refers to using the input and output results of an existing learned model as the learning dataset for a new learned model to generate new learned parameters.

[0122] The term "distillation model" refers to an inference procedure that incorporates learned parameters that are regenerated through distillation.

[0123] Industrial availability

[0124] This disclosure can be applied to recognition systems, recognition methods, programs (computer programs), learning methods, learned models, distillation models, and methods for generating learning datasets. Specifically, this disclosure can be applied to recognition systems, recognition methods, programs (computer programs), learning methods, learned models, distillation models, and methods for generating learning datasets related to objects displayed in images.

[0125] Explanation of reference numerals in the attached figures

[0126] 10. Identification System

[0127] 12 Storage devices

[0128] 13 Operational Circuit

[0129] M10 learning completed model

[0130] M11 First Model Section

[0131] M12 Second Model Section

[0132] M20 model

[0133] M21 First Model Department

[0134] M22 Second Model Section

[0135] M23 Third Model Department

[0136] M24 Fourth Model Department

[0137] D21 Learning Dataset

[0138] D22 Learning Dataset

[0139] P10 First resolution image

[0140] P20 Second Resolution Image

[0141] P30 Differential Image

[0142] F11 and F12 features

[0143] The results of R11 and R21 recognition (recognition results)

[0144] The result of R22 determination (determination result)

[0145] S11 Acquisition and Processing

[0146] S12 Inference Processing

[0147] S13 Output Processing

[0148] S21 Preparation Steps

[0149] S210 Generation Steps

[0150] S210a First Step

[0151] S210b Second Step

[0152] S210c Third Step

[0153] S210d Step 4

[0154] S211 Pre-learning steps

[0155] S22 Learning Steps

[0156] 100 Objects.

Claims

1. An identification system, comprising: Storage device for storing the learned model; as well as The arithmetic circuitry is capable of accessing the storage device. The learned model includes: The first model unit performs the following learning: For an input of a first-resolution image of an object displayed at a first resolution, it outputs a second-resolution image equivalent to converting the first-resolution image to a second resolution higher than the first resolution, and a difference image equivalent to the difference between the first-resolution image and the second-resolution image, which is the image obtained by enlarging the first-resolution image to the same image size as the second-resolution image; and The second model unit performs the following learning: For the input of the second resolution image and the difference image, it outputs the feature quantities of the object. The arithmetic circuit performs: The acquisition process involves obtaining the first resolution image as an object image; and The inference process involves feeding the object image acquired in the acquisition process to the learned model, so that the learned model can calculate the feature quantities of the object appearing in the object image.

2. The identification system according to claim 1, wherein, The inference process includes identifying the object based on the feature quantities of the object appearing in the object image.

3. The identification system according to claim 1 or 2, wherein, The arithmetic circuit performs output processing to output the result of the inference process.

4. A recognition method, executed by a computational circuit capable of accessing a storage device storing a learned model, wherein, The learned model includes: The first model unit performs the following learning: For an input of a first-resolution image of an object displayed at a first resolution, it outputs a second-resolution image equivalent to converting the first-resolution image to a second resolution higher than the first resolution, and a difference image equivalent to the difference between the first-resolution image and the second-resolution image, which is the image obtained by enlarging the first-resolution image to the same image size as the second-resolution image; and The second model unit, taking the second resolution image and the difference image as input, outputs the feature quantities of the object. The identification method includes: The acquisition process involves obtaining the first resolution image as an object image; and The inference process involves feeding the object image acquired in the acquisition process to the learned model, so that the learned model can calculate the feature quantities of the object appearing in the object image.

5. A program product for causing the arithmetic circuit to perform the identification method of claim 4.

6. A learning method, comprising: Preparation steps, prepare the model; as well as The learning step involves performing machine learning using the model prepared in the preparation step. The model includes a first model section, a second model section, and a third model section. The first model unit is a model that takes a first-resolution image of an object displayed at a first resolution as input, and outputs a second-resolution image equivalent to converting the first-resolution image to a second resolution higher than the first resolution, and a difference image equivalent to the difference between the first-resolution image and the second-resolution image, which is an image obtained by enlarging the first-resolution image to the same image size as the second-resolution image. The second model unit is a model for outputting feature quantities of the object from inputs of the second resolution image and the difference image from the first model unit. The third model unit is a model used to output the recognition result of the object in response to the input of the feature quantities of the object from the second model unit. The learning step includes using machine learning on a learning dataset that takes the first resolution image as input and the results of recognizing objects displayed in the first resolution image as correct answers, to enable the model to learn the relationship between the feature quantities of the first resolution image and the objects displayed in the first resolution image.

7. The learning method according to claim 6, wherein, The preparation steps include: The generation step involves generating a learning dataset that uses the first resolution image as input and the second resolution image and the difference image as correct answers; and The pre-learning step uses the learning dataset generated in the generation step to enable the first model to learn the relationship between the first resolution image, the second resolution image, and the difference image.

8. The learning method according to claim 7, wherein, The generation steps include: The first step is to acquire the image at the second resolution. The second step is to convert the second resolution image obtained in the first step into an image of the first resolution to generate the first resolution image; The third step involves generating the difference image from the second resolution image obtained in the first step and the first resolution image generated in the second step; and The fourth step is to generate a learning dataset that takes the first resolution image generated in the second step as input, and the second resolution image prepared in the first step and the difference image generated in the third step as correct answers.

9. The learning method according to claim 8, wherein, In the third step, the first resolution image generated in the second step is enlarged to the same image size as the second resolution image obtained in the first step, and the difference image is generated based on the difference in pixel values ​​between the enlarged first resolution image and the second resolution image obtained in the first step.

10. The learning method according to claim 9, wherein, The range of pixel values ​​in the differential image is narrower than the range of pixel values ​​in the magnified first resolution image and the second resolution image obtained in the first step.

11. The learning method according to any one of claims 6 to 10, wherein, The model includes a fourth model unit that takes the second resolution image as input and outputs a result determining whether the second resolution image is an image generated by the first model unit. The learning step includes performing additional learning on at least one of the first model unit and the fourth model unit based on the result of the determination output from the fourth model unit.

12. A learned model, comprising: The first model unit performs the following learning: for input of a first-resolution image of an object displayed at a first resolution, it outputs a second-resolution image that is equivalent to converting the first-resolution image into a second-resolution image that is higher than the first resolution, and a difference image that is equivalent to the difference between the first-resolution image and the second-resolution image, which is equivalent to enlarging the first-resolution image to the same image size as the second-resolution image. as well as The second model unit performs the following learning: for the input of the second resolution image and the difference image, it outputs the feature quantity of the object.

13. A distillation model generated by distillation of the learned model as described in claim 12.

14. An identification system, comprising: Storage device for storing the distillation model as described in claim 13; and The arithmetic circuitry is capable of accessing the storage device. The arithmetic circuit performs: The acquisition process involves obtaining the first resolution image as an object image; and The inference process involves feeding the object image acquired in the acquisition process to the distillation model, so that the distillation model can calculate the feature quantities of the object appearing in the object image.

15. A method for generating a learning dataset, comprising: The first step is to obtain a reference image of the object to be displayed; The second step is to convert the reference image into a low-resolution image with a lower resolution than the reference image. The third step is to generate a difference image that is equivalent to the difference between the reference image and an image in which the low-resolution image is enlarged to the same size as the reference image; as well as The fourth step is to generate a learning dataset that uses the low-resolution image as input and the reference image and the difference image as the correct answers.

Images