Machine learning data collection system, machine learning data collection method, and program

Abstract

To provide a machine learning data collection system, a machine learning data collection method and a program for reducing the burden on individuals when generating learning data of handwritten information.SOLUTION: A machine learning data collection system disclosed herein has a service provision server comprising an answer acquisition unit 12 for acquiring digitized handwritten answers to tasks for learning or leisure for a user, and an answer evaluation unit 13 for rating the answers as answers to the tasks. A machine learning data collection server 20 has a data registration unit 23 for generating and registering machine learning data based on the answers.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a machine learning data collection system, a machine learning data collection method, and a program. Background Art

[0002] Machine learning using neural networks or the like sometimes requires a large amount of training data generated by individuals. For example, in machine learning for character recognition of handwritten characters, if training is performed using handwritten character data obtained from a large number of writers, the recognition accuracy for characters with idiosyncratic writing styles may decrease. Therefore, in the character recognition method disclosed in Patent Document 1, the recognition rate is improved by limiting writers to a single individual and providing an individual-specific recognition unit. Prior Art Documents Patent Documents

[0003] Patent Document 1 Japanese Unexamined Patent Publication No. Hei 1-267790 Summary of the Invention Problems to be Solved by the Invention

[0004] However, when a large amount of training data generated by an individual is used, there is a problem that a burden is imposed on the individual who generates a large amount of training data.

[0005] The present invention has been made in view of such circumstances, and provides a machine learning data collection system, a machine learning data collection method, and a program capable of reducing the burden imposed on an individual when generating training data. Means for Solving the Problems

[0006] This invention was made to solve the above-mentioned problems, and one aspect of the present invention is a machine learning data collection system comprising: an answer acquisition unit that acquires digitized handwritten answers to tasks for the user's learning or entertainment; an answer evaluation unit that assigns an evaluation to the answers as answers to the tasks; and a data registration unit that generates and registers machine learning data based on the answers.

[0007] Another aspect of the present invention is the machine learning data collection system described above, comprising a likelihood calculation unit that calculates the likelihood that the answer is correct using the problem, the correct answer to the problem, or the evaluation, and the data registration unit generates the machine learning data based on the answers extracted from the answers based on the likelihood.

[0008] Another aspect of the present invention is the machine learning data collection system described above, wherein the handwritten response is a character, a drawn image, or a colored image.

[0009] Another aspect of the present invention is the machine learning data collection system described above, comprising a task generation unit that generates the task based on the machine learning data to be generated and notifies the user who will create the answer.

[0010] Another aspect of the present invention is a machine learning data collection method comprising the steps of: obtaining a digitized handwritten response that is a user's answer to a task for learning or entertainment; assigning an evaluation to the response as an answer to the task; and generating and registering machine learning data based on the response.

[0011] Another aspect of the present invention is a program that causes a computer to function as an answer acquisition unit that acquires digitized handwritten answers to tasks for the user's learning or entertainment, an answer evaluation unit that assigns an evaluation to the answers as answers to the tasks, and a data registration unit that generates and registers machine learning data based on the answers. [Effects of the Invention]

[0012] This invention makes it possible to reduce the burden on individuals when generating machine learning data. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic block diagram showing the configuration of a machine learning data collection system 100 according to one embodiment of the present invention. [Figure 2] This is a schematic block diagram showing the functional configuration of the service provision server 10 and the machine learning data collection server 20 in the same embodiment. [Figure 3] This table shows a first example of machine learning data stored in the machine learning database 30 in the same embodiment. [Figure 4] This table shows a second example of machine learning data stored in the machine learning database 30 in the embodiment. [Figure 5] This table shows a third example of machine learning data stored in the machine learning database 30 in the embodiment. [Figure 6] This is a sequence diagram illustrating the operation of the machine learning data collection system 100 in the same embodiment. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a schematic block diagram showing the configuration of a machine learning data collection system 100 according to one embodiment of the present invention. The machine learning data collection system 100 comprises a service provision server 10, a machine learning data collection server 20, and a machine learning database 30. The service provision server 10, the machine learning data collection server 20, and the machine learning database 30 are realized by a computer running an application.

[0015] Furthermore, one or more of the service providing server 10, the machine learning data collection server 20, and the machine learning database 30 may be implemented by one or more computers. In addition, one or more of the service providing server 10, the machine learning data collection server 20, and the machine learning database 30 may be implemented by one or more applications. The user terminal 40 is a computer such as a smartphone, a tablet terminal, or a personal computer, and may be implemented by executing a general-purpose web browser or the like, or may be implemented by executing a dedicated application.

[0016] The service providing server 10 is communicably connected to a plurality of user terminals 40 and the machine learning data collection server 20 via a network 50. The service providing server 10 causes the user terminal 40 to present a task for learning or entertainment for the user of the user terminal 40, and acquires, from the user terminal 40, the user's answer to the task, which is a digitized handwritten answer.

[0017] Examples of tasks for learning or entertainment include those that require answers in text, such as transcription of characters such as kanji and words, reading and writing of kanji, fill-in-the-blank sentence questions, crossword puzzles, quizzes, and number places. In addition, there are other tasks that require answers as drawn images, such as drawing and copying, and tasks that require answers by coloring images, such as coloring.

[0018] The machine learning data collection server 20 is communicably connected to the service providing server 10 via a network 50. The machine learning data collection server 20 acquires answers from the service providing server 10, generates machine learning data based on the answers, and registers the machine learning data in the machine learning database 30. Furthermore, the machine learning data collection server 20 may also acquire, from the service providing server 10 together with the above-described answers, a task or a correct answer to the task necessary for generating the machine learning data.

[0019] The machine learning database 30 stores machine learning data registered by the machine learning data collection server 20. The machine learning database 30 may be directly connected to the machine learning data collection server 20, or may be connected via a network 50 or the like. The machine learning database 30 provides the stored machine learning data as data for performing machine learning.

[0020] FIG. 2 is a schematic block diagram showing the functional configuration of a service providing server 10 and a machine learning data collection server 20 according to the present embodiment. As shown in FIG. 2, the service providing server 10 includes a task presentation unit 11, an answer acquisition unit 12, and an answer evaluation unit 13. The task presentation unit 11 (task generation unit) notifies a user terminal 40 of a task to cause the user terminal 40 to present the task. Note that the task may be based on a request from the data acquisition unit 21. For example, the data acquisition unit 21 notifies the task presentation unit 11, as a data request, of a character code indicating a required Chinese character (e.g., "chi (ground)") based on the completeness of collected machine learning data, machine learning accuracy using the collected machine learning data, or the like. The task presentation unit 11 notifies the user terminal 40 of, as a task, handwriting of the Chinese character corresponding to the character code, an idiom including the Chinese character (e.g., "chi-qiu (earth)"), or a sentence. For idioms and sentences, the task presentation unit 11 searches a dictionary to acquire them, for example.

[0021] Further, the data acquisition unit 21 may notify the task presentation unit 11 of an image or a name of an object required as a drawn image, and the task presentation unit 11 may notify the user terminal 40 of, as a task, drawing of the object corresponding to the image or name. Further, the data acquisition unit 21 may notify the task presentation unit 11 of an uncolored line drawing image, and the task presentation unit 11 may notify the user terminal 40 of, as a task, coloring the line drawing image. Note that the task may be notified to the user by mail. When notified of a name from the data acquisition unit 21, the task presentation unit 11 searches for and acquires an image associated with the name, for example. Further, when notifying a user of a line drawing image, the task presentation unit 11 may acquire a colored image from the data acquisition unit 21, convert the colored image into a line drawing, and notify the user of the line drawing image.

[0022] The user terminal 40 presents the task notified by the task presentation unit 11 to the user. The user terminal 40 digitizes the user's handwritten response to the task and transmits it to the response acquisition unit 12. Digitization may be performed, for example, by scanning a handwritten response on paper with a digital camera or scanner, by capturing a handwritten response on a touch panel with a stylus pen, or by capturing a handwritten response using a pointing device such as a mouse.

[0023] The response acquisition unit 12 acquires the response by receiving the response that has been digitized and transmitted by the user terminal 40. The response evaluation unit 13 assigns an evaluation to the response acquired by the response acquisition unit 12 as an answer to the task notified by the task presentation unit 11, and notifies the user terminal 40 of the evaluation result. This evaluation may be an evaluation that the response has been submitted, or an evaluation of whether the response is correct or not. Furthermore, the determination of whether the response is correct or not may be made by a human, or by a human if the response evaluation unit 13 has determined that the response is incorrect using character recognition technology or the like.

[0024] The data acquisition unit 21 acquires responses from the response acquisition unit 12. The data acquisition unit 21 may also acquire only responses that have been determined to be correct by the response evaluation unit 13. Furthermore, the data acquisition unit 21 acquires the tasks or correct answers to the tasks necessary for generating machine learning data from the task presentation unit 11, in association with the aforementioned responses. The data acquisition unit 21 may also acquire the evaluation results of the acquired responses from the response evaluation unit 13.

[0025] The likelihood calculation unit 22 calculates the likelihood that the answer obtained by the answer acquisition unit 12 is correct, using the task, the correct answer to the task, or the evaluation. For example, if the answer is an image of handwritten characters, the likelihood calculation unit 22 recognizes the characters in the image using character recognition technology and calculates the likelihood. When the likelihood calculation unit 22 exceeds a predetermined threshold, it considers the image of the characters as a candidate for machine learning data. Here, the threshold used is a lower value than that used in general character recognition. This makes it possible to include images that are highly distinctive and would not be recognized or would be misrecognized in general character recognition as candidates for machine learning data.

[0026] Furthermore, the likelihood calculation unit 22 may, when the likelihood of a recognition result matching the correct answer acquired by the data acquisition unit 21 exceeds a predetermined threshold, consider the image of the character as a candidate for machine learning data. For example, suppose that as a result of character recognition in a certain image, the likelihood of character A matching the correct answer is X, and the likelihood of character B not matching the correct answer is Y. In this case, the likelihood calculation unit 22 may, if X exceeds a predetermined threshold, consider the image as a candidate for machine learning data even if the likelihood of X is lower than Y.

[0027] Furthermore, when performing character recognition, the likelihood calculation unit 22 may use a learning model that has been trained using machine learning data registered in the machine learning database 30 as machine learning data up to that point. In addition, this learning model may be prepared for each user. That is, the data acquisition unit 21 acquires information that identifies the user of the answer in association with the answer, and a learning model for a particular user may be generated by training using machine learning data based on that user's answers. This makes it possible to obtain a learning model that reflects the individual habits of each user. The same applies when the answer is not a character image.

[0028] Furthermore, if the answer is a colored image, the likelihood calculation unit 22 may calculate the likelihood from the color difference between the answer and, for example, a common color such as the sky, forest, or animal, or the color of the image considered to be the correct answer. Alternatively, regions that make up the image, such as the sky, forest, or animal, may be set in advance, and the likelihood may be calculated from the color difference between each region and the color of the image considered to be the correct answer. Or, the likelihood calculation unit 22 may not be provided or used, and all answers may be included as candidates for machine learning data.

[0029] Furthermore, if the answer is a drawn image, instead of character recognition and correct answer as in the case of a character image, the likelihood calculation unit 22 may use object recognition technology (image recognition technology) and the task (the name of the object to be drawn, or the image recognition result of the task image) to recognize objects in the drawn image and perform the same processing as in the case of a character image. In addition, if the task is an image, the likelihood calculation unit 22 may calculate the similarity between the task image and the answer's drawn image and use this similarity as the likelihood.

[0030] The data registration unit 23 registers the responses identified as candidate machine learning data by the likelihood calculation unit 22 in the machine learning database 30, associating them with the task or the correct answer to the task. At this time, the data registration unit 23 may also associate information that identifies the user. If the answer is a colored image, for example, each element that makes up the image, such as the sky, forest, or animals, may be registered in the machine learning database 30. Furthermore, if the answer is a drawn image, each recognized object may be registered in the machine learning database 30.

[0031] Figure 3 is a table showing a first example of machine learning data stored in the machine learning database 30 in this embodiment. The first example shown in Figure 3 is an example where the answer is a character image. In this first example, the machine learning database 30 stores "character code A1" and "character image B1" in association, "character code A2" and "character image B2" in association, and "character code A3" and "character image B3" in association.

[0032] For example, if the task involves reading and writing kanji, fill-in-the-blank sentences, crossword puzzles, quizzes, or number-paste puzzles, then these character codes represent the character codes of the correct answers, and these character images represent the images of the answers. Similarly, if the task involves writing, then these character codes represent the character codes of the task, and these character images represent the images of the answers.

[0033] Figure 4 is a table showing a second example of machine learning data stored in the machine learning database 30 in this embodiment. The second example shown in Figure 4 is an example where the answer is a drawn image. In this second example, the machine learning database 30 stores "Name C1" in association with "Drawn Image D1", "Name C2" in association with "Drawn Image D2", and "Name C3" in association with "Drawn Image D3". These names are, for example, the names of the objects assigned to be drawn and the names of the objects in the images assigned to be copied.

[0034] Figure 5 is a table showing a third example of machine learning data stored in the machine learning database 30 in this embodiment. The third example shown in Figure 5 is an example where the answer is a colored image. In this third example, the machine learning database 30 stores "Name E1" in association with "Colored Image F1", "Name E2" in association with "Colored Image F2", and "Name E3" in association with "Colored Image F3". These names are the names of the regions that make up the line drawing image that was the subject of the problem.

[0035] Note that the character images, drawing images, and colored images in Figures 3 to 5 do not have to be in image format; they can be in any data format. Furthermore, they may be converted into features for machine learning.

[0036] Figure 6 is a sequence diagram illustrating the operation of the machine learning data collection system 100 in this embodiment. First, the machine learning data collection server 20 sends a data request S1 indicating the necessary data to the service provider server 10. Based on the data request S1, the service provider server 10 generates a task S2 and notifies the user terminal 40. The user terminal 40 displays the notified task S2 and sends the user's answer S3 to the service provider server 10.

[0037] The service provider server 10 evaluates the response S3 as an answer to the task S2 and notifies the user terminal 40 of the evaluation result S4. The service provider server 10 also sends the response S3 along with the task S2 or correct answer S5 to the machine learning data collection server 20. The machine learning data collection server 20 generates machine learning data S6 based on the response S3 and the task S2 or correct answer S5 and registers it in the machine learning database 30.

[0038] In this way, the machine learning data collection system 100 obtains answers to learning or entertainment tasks and assigns an evaluation to those answers as answers to the tasks. Then, the machine learning data collection system 100 generates machine learning data based on those answers. In this way, individuals who generate answers can obtain evaluation results for their answers to the tasks, thus reducing the burden on individuals when generating machine learning data.

[0039] Furthermore, the machine learning data collection system 100 in this invention may be realized by recording a program for realizing the functions of the machine learning data collection system 100 shown in Figure 1 onto a computer-readable recording medium, and then loading and executing the program recorded on this recording medium into a computer system. The term "computer system" here includes hardware such as an OS (Operating System) and peripheral devices. The term "computer system" also includes a WWW (World Wide Web) system equipped with a homepage provisioning environment (or display environment).

[0040] Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs (Read Only Memory), CD-ROMs (Compact Disc - Read Only Memory), and storage devices such as hard disks built into computer systems. In addition, "computer-readable recording media" also includes volatile memory (RAM (Random Access Memory)) within computer systems that act as servers or clients when programs are transmitted via networks such as the Internet or communication lines such as telephone lines, which retain programs for a certain period of time.

[0041] Furthermore, the above program may be transmitted from a computer system that stores the program in a memory device or the like to another computer system via a transmission medium or by transmission waves within the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Also, the above program may be for the purpose of realizing only a part of the functions described above. Furthermore, it may be a so-called differential file (differential program) that can realize the above functions in combination with a program already recorded in the computer system.

[0042] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and may include design modifications and the like that do not depart from the spirit of this invention. [Explanation of Symbols]

[0043] 10 Service Provider Servers 11 Assignment presentation section 12 Answer acquisition part 13. Response Evaluation Department 20 Machine Learning Data Collection Server 21 Data Acquisition Unit 22 Likelihood calculation unit 23 Data Registration Section 30 Databases for Machine Learning 40 User terminals 100 Machine Learning Data Collection Systems

Claims

1. A response acquisition unit that acquires digitized handwritten responses to tasks for the user's learning or entertainment purposes, A response evaluation unit that assigns an evaluation to the aforementioned response as an answer to the aforementioned problem, A data registration unit that generates and registers machine learning data based on the above response, The task generation unit generates the aforementioned task based on the machine learning data to be generated and notifies the user who will create the answer. A machine learning data collection system equipped with the following features.

2. The system includes a likelihood calculation unit that calculates the likelihood that the aforementioned answer is correct, using the aforementioned task, the correct answer to the task, or the aforementioned evaluation. The data registration unit generates the machine learning data based on the responses extracted from the responses based on the likelihood. A machine learning data collection system according to claim 1.

3. The machine learning data collection system according to claim 1, wherein the handwritten response is a character, a drawn image, or a colored image.

4. A method for collecting machine learning data, The process involves obtaining a digitized handwritten response to a task intended for user learning or entertainment, and The steps include: providing an evaluation of the above response as an answer to the above problem; The steps include generating and registering machine learning data based on the above response, The steps include generating the aforementioned problem based on the machine learning data to be generated and notifying the user who will create the aforementioned answer, and A machine learning data collection method having [specific features].

5. Computers, A response acquisition unit that acquires digitized handwritten responses to tasks for the user's learning or entertainment purposes. A response evaluation unit provides an evaluation of the aforementioned response as an answer to the aforementioned problem. A data registration unit generates and registers machine learning data based on the aforementioned response. Task generation unit generates the aforementioned task based on the machine learning data to be generated and notifies the user who will create the answer. A program designed to function as such.

Images