Image recognition device, image recognition method, and object recognition model

Abstract

We provide image recognition technology that can recognize unknown images with high accuracy. [Solution] The object recognition unit 20 recognizes objects in the input image using an object recognition model. The recognition accuracy determination unit 40 determines the recognition accuracy of objects in the input image. The image conversion unit 60 converts the input image if the object recognition accuracy does not meet a predetermined threshold. The image is labeled based on its features and converted into a supervised image. The transfer learning unit 80 is supervised The object recognition model is updated by using images as training data for transfer learning. ru.

Description

Technical Field

[0001] The present invention relates to image recognition technology.

Background Art

[0002] In an apparatus for recognizing an object from an image, an image with low recognition accuracy is classified as an unknown image that is difficult to recognize. For an unknown image, a human identifies and labels the objects in the image, and a manual annotation operation is performed.

[0003] Patent Document 1 discloses a system that acquires an unknown image for which a learned model has not yet been created, selects a learned model of a known image whose imaging conditions are similar to the acquired unknown image from the learned models, analyzes the unknown image using the selected learned model, and provides the result of the image analysis.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Manual annotation takes a lot of time and cost, so it is not suitable for practical use.

[0006] The present invention has been made in view of such a situation, and an object thereof is to provide an image recognition technology capable of recognizing an unknown image with high accuracy.

Means for Solving the Problems

[0007] ​​​​ To solve the above problems, an image recognition device in one aspect of the present invention uses an object recognition model The object recognition unit recognizes objects in the input image, and the object recognition unit recognizes objects in the input image. A recognition accuracy determination unit that determines the accuracy, and the input that determines the recognition accuracy of the object is below a predetermined threshold. The image is labeled based on the features of the input image and converted into a supervised image. A supervised image conversion unit and an object recognition model that uses the supervised image as training data It includes a transfer learning unit that performs transfer learning on the object recognition model and updates the object recognition model.

[0008] Another aspect of the present invention is an image recognition method. This method uses an object recognition model to input The steps include recognizing an object in a force image and determining the accuracy of object recognition in the input image. The steps include: and for the input image in which the recognition accuracy of the object does not meet a predetermined threshold, The steps include: labeling the input image based on its features and converting it into a supervised image; and The object recognition model is transferred and learned using the supervised images as training data. This includes the step of updating the recognition model.

[0009] Yet another aspect of the present invention is an object recognition model. This object recognition model is a computer An object recognition model that causes a data source to recognize objects in an input image, wherein the recognition accuracy of the object For the input image in which the value is less than a predetermined threshold, a label is applied based on the feature quantities of the input image. The system is updated using transfer learning with supervised images marked with a "ru" as training data.

[0010] Furthermore, any combination of the above components, or any expression of the present invention, may be used to describe a method, apparatus, system, or recording medium. The conversion between the human body, computer programs, etc., is also valid as an embodiment of the present invention. Yes.

Advantages of the Invention

[0011] According to the present invention, an image recognition technology capable of recognizing unknown images with high accuracy can be provided. It can be done.

Brief Description of the Drawings

[0012] [Figure 1] It is a configuration diagram of an image recognition device according to an embodiment. [Figure 2] Figs. 2(a) to 2(d) are diagrams showing an example of an image input to the object recognition unit of Fig. 1. [Figure 3] Figs. 3(a) to 3(d) are diagrams for explaining the results of the object recognition unit of Fig. 1 recognizing the objects in the images of Figs. 2(a) to 2(d) based on a learned object recognition model. [Figure 4] It is a diagram for explaining a labeled image with a label attached to an unknown image. [Figure 5] It is a flowchart for explaining an image recognition procedure by the image recognition device of Fig. 1.

Modes for Carrying Out the Invention

[0013] Fig. 1 is a configuration diagram of an image recognition device 100 according to an embodiment. The image recognition device 100 includes an input unit 10, an object recognition unit 20, an object recognition model storage unit 30, a recognition accuracy determination unit 40, an output unit 50, a labeled image conversion unit 60, a feature-label database 70, a transfer learning unit 80, and a pre-training dataset storage unit 90.

[0014] The input unit 10 acquires an image of an object to be recognized and supplies it to the object recognition unit 20.

[0015] In the object recognition model storage unit 30, a recognition accuracy that has been pre-learned with a pre-training dataset A highly accurate object recognition model is stored. The pre-trained dataset storage unit 90 contains objects The pre-training dataset used to train the body recognition model is stored.

[0016] The object recognition unit 20 uses the trained object recognition model stored in the object recognition model storage unit 30. The object recognition unit 20 recognizes objects in the input image. The recognition result is supplied to the recognition accuracy determination unit 40. The recognition result includes the class of at least one object. This includes candidate names and their recognition accuracy.

[0017] If the object recognition accuracy by the object recognition unit 20 is above a predetermined threshold, the recognition result is output by the output unit. The signal is supplied to unit 50. The output unit 50 outputs the recognition result of the input image.

[0018] The recognition accuracy determination unit 40 determines the recognition accuracy of objects in the input image, and if the object recognition accuracy is Images with low recognition accuracy that do not meet a predetermined threshold are supplied to the supervised image conversion unit 60 as unknown images. ru.

[0019] The supervised image conversion unit 60 applies the following to the unknown image supplied by the recognition accuracy determination unit 40: The unknown image is labeled based on its features and converted into a supervised image, and the transfer learning unit 80 It supplies to [the system]. An example of a feature used to label unknown images is an object recognition model. This is the intermediate output when an image is input to the trained object recognition model stored in the memory unit 30. Alternatively, the features of the hidden layers in the later stages of the neural network may be used.

[0020] The feature-label database 70 is a database that stores pairs of image features and labels. The supervised image transformation unit 60 refers to the feature-label database 70, Obtain the label corresponding to the feature that most closely resembles the features of the unknown image, and use the obtained label. It is used to transform unknown images into supervised images. As an example of a feature-label database, The image is input to the pre-trained object recognition model stored in the object recognition model storage unit 30. The intermediate output in this case is the feature quantities of the hidden layer in the later stages of the neural network, and its image A database that stores images in association with their labels may also be used.

[0021] The transfer learning unit 80 uses supervised images supplied from the supervised image conversion unit 60 for pre-training. A new dataset is added to the pre-training dataset stored in the dataset memory unit 90. The object recognition model is constructed and transferred to a new dataset, and the updated object recognition model is then used for transfer learning. The data is stored in the object recognition model storage unit 30.

[0022] Here, the pre-training dataset storage unit 90 is used for pre-training the object recognition model. If datasets such as big data are available, use those datasets for pre-training. It is stored as data. The dataset used for pre-training the object recognition model is utilized. If it cannot be used, the object recognition unit 20 recognizes the object with high recognition accuracy, and known input Images may be stored as a pre-training dataset. In this case, the transfer learning unit 80 For the pre-training dataset of known images stored in the pre-training dataset storage unit 90 Then, a new dataset is constructed by adding labeled supervised images to the unknown images. We will transfer-learn an object recognition model using the Tan dataset.

[0023] In transfer learning, as an example, the final output layer of a pre-trained neural network is used Replace the layer with a new one, and use the new dataset as training data to modify the parameters of the new layer. By retraining the model, a new neural network is generated.

[0024] The object recognition unit 20 improves the recognition accuracy by using an updated object recognition model. Even when an image containing an object is input, it can recognize that object with high accuracy. This makes it possible to recognize objects of an unknown class.

[0025] Figures 2(a) to 2(d) show examples of images input to the object recognition unit 20. Figure 2(a) is an image of a person, Figure 2(b) is an image of a motorcycle, Figure 2(c) is an image of a car, Figure 2(d) is an image of a bicycle. There are other types of input images as well; for example, there are a total of 1 Let's assume there are 0 types.

[0026] The initial object recognition model was designed to utilize a large number of images of three types of subjects: people, motorcycles, and cars. Let's assume that these three types of images were used as a pre-trained dataset.

[0027] Figures 3(a) to 3(d) show the object recognition unit 20 based on the learned object recognition model. This figure illustrates the results of object recognition in the images shown in Figures 2(a) to 2(d).

[0028] The initial object recognition model was pre-trained on three classes: people, motorcycles, and cars. Since this has been done, as a result of the recognition, the image in Figure 2(a) shows "person" as shown in Figure 3(a). As shown in Figure 3(b), the image in Figure 2(b) is labeled "motorcycle", and as shown in Figure 3(c) As shown in Figure 2(c), the label "car" is identified in the image. However, the initial object recognition The model has not learned about the bicycle class, as shown in Figure 3(d). The recognition results for the image in Figure 2(d) are inaccurate, and the label remains unknown.

[0029] The supervised image transformation unit 60 extracts the feature quantities from the image in Figure 2(d) and performs a feature quantity-label data analysis. Refer to Base 70 and find the feature that most closely resembles the feature in the image in Figure 2(d). The bell is retrieved. In this case, the retrieved label is "bicycle". Supervised image transformation unit 60 is "bicycle" obtained from the feature-label database 70, as shown in Figure 4. Labels are added to the image in Figure 2(d) to convert it into a supervised image.

[0030] The transfer learning unit 80 uses the supervised images in Figure 4 as new training data to perform object recognition on the model. A new object recognition model is generated by transferring learning from the previous model. The system will be able to identify four types of vehicles: people, motorcycles, cars, and bicycles. In this way, whenever an unknown image with low recognition accuracy is detected, the labeled supervised image is generated. The supervised images are converted to a new training data, and the object recognition model is then subjected to transfer learning using the supervised images as new training data. This will allow us to identify all 10 classes.

[0031] Here, only supervised images labeled for images of the unknown class are used as training data. When an object recognition model is subjected to transfer learning using this method, images of known classes that were previously able to be identified will be processed. It is possible that the system may fail to recognize the unknown in the supervised images of the known class. Object recognition using a new dataset with added supervised images of classes as training data. It is preferable to perform transfer learning on the model. For example, transfer learning using images of bicycles. If you do this, you can use a pre-trained dataset of supervised images of people, motorcycles, and cars, and then apply the supervision of bicycles to the same dataset. A new dataset was constructed by adding supervised images, resulting in data containing four types of supervised images. The object recognition model is subjected to transfer learning using the set as training data. This ultimately results in This allows us to generate an object recognition model that can correctly recognize all types of images.

[0032] Figure 5 is a flowchart illustrating the image recognition procedure performed by the image recognition device 100.

[0033] The object recognition unit 20 inputs the image to be recognized into the trained object recognition model, and in the image Recognize the object (S10).

[0034] If the object recognition accuracy is above a predetermined threshold (N in S20), the recognition result label is output. (S30), return to step S10 and input another image.

[0035] If the object recognition accuracy is below a predetermined threshold, for example, the accuracy of the first candidate label... If the rate is less than 50% (Y in S20), the input image is treated as an unknown image for image feature extraction. This will be done (S40).

[0036] The supervised image transformation unit 60 extracts the features of the unknown image from the feature-label database 70. The matching label is obtained (S50), and the obtained label is assigned to the unknown image as a teacher. Generate an image with the attached (S60).

[0037] Until a predetermined number of supervised images of objects of the same class have been accumulated, for example, 30 images (N in S70) Then, repeat the procedure from step S10 to step S60. A predetermined number of sheets, in this case 30 sheets. When supervised images of the same class accumulate (Y in S70), the transfer learning unit 80 performs the following prior learning A new dataset is created by adding a predetermined number of supervised images to the dataset used for learning. Then, transfer learning is performed on the trained object recognition model using the new dataset (S80). The transfer learning unit 80 generates a new object recognition model that has undergone transfer learning, and the object recognition model memory Store in memory section 30 (S90).

[0038] In the above explanation, transfer learning is performed after a predetermined number of supervised images of objects of the same class have been accumulated. However, transfer learning can also be performed each time a supervised image is generated.

[0039] The various processes of the image recognition device 100 described above utilize hardware such as the CPU and memory. Of course, it can be realized using ROM (Read-Only Memory). Firmware stored in flash memory, etc., and software on computers, etc. This can also be achieved through wearables. Specifically, through firmware programs and software. The program may also be provided by recording it on a recording medium that can be read by a computer, etc. Alternatively, data can be sent and received to and from the server via a wireless network, or via terrestrial or satellite radio. It is also possible to transmit and receive data broadcasts as digital broadcasting.

[0040] In the conventional configuration, annotation work is performed manually on unknown images, and during transfer learning... It was necessary to add it as training data. In the image recognition device 100 of this embodiment, learning Based on the recognition results of the completed object recognition model, unknown images with low recognition accuracy are detected, and the unknown images are... The images are automatically converted into supervised images, and these converted supervised images are added as new training data for learning. We retrain a portion of the already completed object recognition model. This allows us to automatically recognize unknown images with high accuracy. It will become possible to recognize it in degrees.

[0041] The present invention has been described above based on embodiments. The embodiments are illustrative and their respective components The fact that various variations are possible in the combination of constituent elements and each processing process, and such variations Those skilled in the art will understand that this also falls within the scope of the present invention. [Explanation of symbols]

[0042] 10 Input unit, 20 Object recognition unit, 30 Object recognition model storage unit, 40 Recognition precision Degree determination unit, 50 Output unit, 60 Supervised image conversion unit, 70 Feature quantity-label data Database, 80 transfer learning units, 90 pre-trained dataset storage units, 100 images recognition device.

Claims

1. An object recognition unit that recognizes objects in an input image using an object recognition model, A recognition accuracy determination unit that determines the recognition accuracy of an object in the input image, For the input image in which the recognition accuracy of the object falls below a predetermined threshold, the characteristics of the input image It includes a supervised image conversion unit that labels based on characteristics and converts them into supervised images. An image recognition device characterized by the following.

2. The object recognition unit performs the following actions on the input image where the object recognition accuracy is above a predetermined threshold: The present invention is characterized in that it outputs the label of the object in the input image. Image recognition device.

3. The steps involve using an object recognition model to recognize objects in the input image, The steps include determining the accuracy of object recognition in the input image, For the input image in which the recognition accuracy of the object falls below a predetermined threshold, the characteristics of the input image The method is characterized by including the step of labeling based on characteristics and converting them into supervised images. A computer-based image recognition method.

4. The steps involve using an object recognition model to recognize objects in the input image, The steps include determining the accuracy of object recognition in the input image, For the input image in which the recognition accuracy of the object falls below a predetermined threshold, the characteristics of the input image The computer performs the steps of labeling based on characteristics and converting them into supervised images. An image recognition program characterized by performing the following actions.

Images