Learning data selection device, learning data selection method, and program
The learning data sorting device enhances machine learning data selection by identifying user operations and settings to exclude inappropriate data, thereby improving the accuracy of orientation determination processing.
Patent Information
- Application Number
- JP2023543526
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-08-24
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-08-24
AI Technical Summary
Existing technologies face challenges in selecting useful learning data for machine learning without manual intervention or visual inspection, particularly in image orientation correction, due to the mixing of inappropriate data types such as corrected, uncorrected, and incorrectly oriented images, which affects accuracy.
A learning data sorting device that identifies user operations on data files to determine their suitability for machine learning by evaluating user reliability, commonality, and specific settings, such as disabling automatic correction, to select high-quality learning data.
Enables the selection of useful learning data without manual intervention, improving the accuracy of orientation determination processing by focusing on user operations and data file characteristics.
Smart Images

Figure 0007577867000001 
Figure 0007577867000002 
Figure 0007577867000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a training data selection device, a training data selection method, and a program. [Background technology]
[0002] For example, Patent Document 1 discloses an inspection system that uses an evaluation data set to evaluate the judgment performance of a first classifier before re-learning or additional learning and a second classifier after re-learning or additional learning, and judges whether the judgment performance of the second classifier has deteriorated compared to the first classifier based on the evaluation result.Then, an inspection system is disclosed that, when the judgment performance of the second classifier has deteriorated compared to the first classifier, judges the quality of a product by using the first classifier without using the second classifier.
[0003] Patent Document 2 also discloses a machine learning data collection system 3 including a classification model holding means 22 that arranges a plurality of trained classification models to be used in a user environment 2, a model distribution means 26 that distributes the classification models to the user environment 2, a classification result holding means 23 that performs classification using labeled data sent from the user environment 2 as input and holds the classification results including whether the classification is correct or not and the accuracy rate for each input data, an optimal model recommendation means 21 that presents an appropriate classification model for the input data based on the classification result for each classification model, and a teacher data recording means 28 that records the input data as teacher or test data for the classification model.
[0004] Furthermore, Patent Document 3 discloses a teacher data collection device that collects data related to a specific field to be used as teacher data for machine learning, the teacher data collection device comprising: a feature calculation unit that calculates a first feature vector, which is a feature vector of reference data related to the specific field that has been registered in advance; a generation unit that generates search conditions to be used for collecting data related to the specific field from the first feature vector; a collection unit that collects data related to the specific field based on the generated search conditions; a similarity calculation unit that calculates a similarity between the second feature vector and the first feature vector when the feature calculation unit calculates a second feature vector, which is a feature vector of the collected data; and an extraction unit that extracts the collected data whose similarity is within a predetermined range as the teacher data. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent Publication No. 2019-158684 [Patent Document 2] Patent Publication No. 2020-194355 [Patent Document 3] Patent Publication 2018-124617 Summary of the Invention [Problem to be solved by the invention]
[0006] The object of the present invention is to provide a learning data selection device that prevents inappropriate learning data from being mixed in. [Means for solving the problem]
[0007] The learning data selection device of the present invention includes a file acquisition unit that acquires a data file, an operation identification unit that identifies a user operation on the data file acquired by the file acquisition unit, and a determination unit that determines whether or not to adopt the data file acquired by the file acquisition unit as learning data for machine learning based on the user operation identified by the operation identification unit.
[0008] Preferably, the system further includes a user evaluation unit which evaluates the reliability of each user based on the operation history of each user, and the judgment unit judges whether or not to adopt the data file as learning data based on the user's operation identified by the operation identification unit and the user's reliability evaluated by the user evaluation unit.
[0009] Preferably, the operation identification unit determines whether or not an operation instructing multiple consecutive scan processes was performed when the data file acquired by the file acquisition unit was generated, and when the operation identification unit determines that an operation instructing multiple consecutive scan processes was performed, further has a commonality evaluation unit that evaluates the commonality of the multiple data files generated by the multiple scan processes, and the judgment unit determines whether or not to adopt the data file as learning data based on the commonality evaluation result by the commonality evaluation unit.
[0010] Preferably, the operation identification unit identifies a user's setting operation at the time of file creation of the data file acquired by the file acquisition unit, and the judgment unit judges whether or not to adopt the data file as learning data based on the user's setting operation identified by the operation identification unit.
[0011] Preferably, the file acquisition unit acquires a data file containing image data as the data file, and the determination unit decides whether to adopt learning data related to an orientation determination process that determines the orientation of an image contained in the data file.
[0012] Preferably, the determination unit determines whether or not to adopt the data file as learning data based on a viewing operation or a modification operation on an image contained in the data file, and further has a learning unit that performs machine learning on orientation determination processing using features of the data file adopted as learning data by the determination unit.
[0013] Preferably, the commonality evaluation unit determines whether the orientations of images included in the multiple data files are consistent, and the determination unit adopts data files determined by the commonality evaluation unit to have consistent image orientations as learning data, and further includes a learning unit that performs machine learning on orientation determination processing using features of the data files adopted as learning data by the determination unit.
[0014] Preferably, the operation identification unit identifies a setting operation related to an automatic correction process of the data file, and the determination unit adopts the data file as learning data when the operation identification unit identifies a setting operation that disables the automatic correction process.
[0015] Preferably, the operation identification unit identifies a setting operation related to an orientation correction process of the data file, and the determination unit further has a learning unit that, when a setting operation that disables the orientation correction process is identified by the operation identification unit, adopts the data file as learning data and performs machine learning on an orientation determination process using features of the data file adopted as the learning data by the determination unit.
[0016] In addition, the learning data selection method of the present invention includes a file acquisition step of acquiring a data file, an operation identification step of identifying a user operation on the data file acquired by the file acquisition step, and a determination step of determining whether or not to adopt the data file acquired by the file acquisition step as learning data for machine learning based on the user operation identified by the operation identification step.
[0017] In addition, the program of the present invention causes a computer to execute a file acquisition step of acquiring a data file, an operation identification step of identifying a user operation on the data file acquired by the file acquisition step, and a determination step of determining whether or not to adopt the data file acquired by the file acquisition step as learning data for machine learning based on the user operation identified by the operation identification step. Effect of the Invention
[0018] It is possible to provide a learning data selection device that prevents inappropriate learning data from being mixed in. [Brief description of the drawings]
[0019] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an image processing system 1. [Diagram 2] 11A and 11B are diagrams illustrating learning data of the orientation determination process. [Diagram 3] 2 is a diagram illustrating an example of a hardware configuration of an image processing server 2. FIG. [Figure 4] 2 is a diagram illustrating an example of a functional configuration of an image processing server 2. FIG. [Diagram 5] 13 is a diagram illustrating an example of an image information table of a data file stored in the image processing server 2. FIG. [Figure 6] 13 is a diagram illustrating an example of a viewing and editing screen provided by a service providing unit 345. FIG. [Figure 7] 13 is a flowchart illustrating a learning process (S10) of the image processing server 2. [Figure 8] 8 is a flowchart for explaining the adoption / rejection determination process (S20) of FIG. 7 in more detail. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of the overall configuration of an image processing system 1. As shown in FIG. As illustrated in FIG. 1, the image processing system 1 includes an image processing server 2, a scanner 4, and a user terminal 6, which are connected to each other via the Internet . The image processing server 2 is, for example, a computer terminal, and performs image processing on the image file received from the scanner 4. The image processing server 2 is an example of a learning data selection device according to the present invention. The scanner 4 is an image reading device that optically reads image data from an original (image display medium) and, for example, transmits a data file (image file) of the read image data to the image processing server 2 via the Internet 7. For example, the scanner 4 is a network scanner and is directly connected to a network such as the Internet 7. The user terminal 6 is a computer terminal operated by a user, and is used to view and edit image data managed by the image processing server 2. The user terminal 6 is, for example, a mobile terminal such as a smartphone or a tablet terminal. The Internet 7 is, for example, an Internet network. In this example, a specific example will be described in which image data is transmitted from the scanner 4 to the image processing server 2 via the Internet 7, but the present invention is not limited to this, and for example, image files may be transmitted and received via a wired connection such as a USB cable, or via a restricted network such as a LAN (Local Area Network). Furthermore, the image processing server 2 may be built into the scanner 4 .
[0021] Next, the background of the present invention will be described. In recent years, with the advancement of machine learning (especially deep learning) technology, efforts to improve image processing accuracy by learning from large amounts of data have become common. One example is the use of a method called CNN (Convolutional Neural Network) to learn character images with various shape patterns, thereby determining the correct orientation of characters and correcting the orientation of images. However, in machine learning, training data with correct answer labels (hereinafter, useful training data) is important, and accuracy does not improve if training data with incorrect answer labels is mixed in, but collecting a large amount of data and manually / visually assigning correct answer labels requires a lot of effort.In addition, although it is possible to make data collection more efficient by using scanned images and orientation determination results of users, users' data cannot be viewed visually to protect privacy. For this reason, there is a need for technology that can select useful learning data without human intervention and without viewing images.
[0022] For example, learning data refers to information such as the example shown in Figure 2. There are multiple explanatory variables that indicate the characteristics of the manuscript, and the correct labels (in this case, "orientation") associated with those explanatory variables are included in the set. When collecting such learning data, the following problems arise. It is difficult to select useful training data from data files acquired from the cloud or locally. In other words, the acquired data includes data other than useful training data. Here, useful learning data corresponds to the following (Data Example 1) and (Data Example 2), and other data corresponds to (Data Example 3). (Data example 1) Data with the image orientation corrected to the correct orientation (corrected data) (Data example 2) Data in which the image orientation has not been corrected (uncorrected data) because the orientation is correct. (Data example 3) Data in which the image orientation is incorrect and has not been corrected (uncorrected data) When determining whether the uncorrected data is (Data Example 2) or (Data Example 3), it is not possible to visually determine the data for the following reasons. - Due to the terms of use, developers cannot view images that contain personal information. - The number of images is large and selection is difficult If (Data Example 3) is included in the machine learning process, the accuracy of the orientation determination process will decrease because the data will be trained with the incorrect correct answer label. In other words, the challenge is to distinguish between (Data Example 2) and (Data Example 3) from uncorrected data.
[0023] Therefore, in the image processing system 1 of this embodiment, whether or not the data file is useful learning data is determined by focusing on the user's operation on the data file. The user's operation on the data file is, for example, an operation performed by the user when generating the data file, an operation performed by the user when viewing or editing the data file, and the like, and in this example, is an operation performed by the user on the scanner 4, and an image viewing operation or editing operation performed by the user on the user terminal 6. Note that in this embodiment, a specific example will be described in which the process of determining the orientation of an image is realized by a machine learning model.
[0024] FIG. 3 is a diagram illustrating an example of a hardware configuration of the image processing server 2. As shown in FIG. As illustrated in FIG. 3, the image processing server 2 includes a CPU 200, a memory 202, a HDD 204, a network interface 206 (network IF 206), a display device 208, and an input device 210, which are connected to each other via a bus 212. The CPU 200 is, for example, a central processing unit. The memory 202 is, for example, a volatile memory, and functions as a main storage device. The HDD 204 is, for example, a hard disk drive device, and serves as a non-volatile storage device for storing computer programs (for example, the image processing program 3 in FIG. 4) and other data files. The network IF 206 is an interface for wired or wireless communication, and realizes communication with the scanner 4 and the user terminal 6, for example. The display device 208 is, for example, a liquid crystal display. The input device 210 is, for example, a keyboard and a mouse.
[0025] FIG. 4 is a diagram illustrating an example of the functional configuration of the image processing server 2. 4, an image processing program 3 is installed and operates on the image processing server 2 of this example. The image processing program 3 is stored in a recording medium such as a CD-ROM, and is installed in the image processing server 2 via this recording medium. The image processing program 3 has a file acquisition unit 300, an operation identification unit 305, a user evaluation unit 310, an orientation determination unit 315, an automatic correction unit 320, a commonality evaluation unit 325, an acceptance / rejection determination unit 330, a feature extraction unit 335, a learning unit 340, and a service provision unit 345. Furthermore, the image processing server 2 accumulates data files (image files) received from the scanner 4 and operation histories for the data files, and manages them as an image information table (described later with reference to FIG. 5). Note that a part or the whole of the image processing program 3 may be realized by hardware such as an ASIC, or may be realized by borrowing some of the functions of an OS (Operating System).
[0026] In the image processing program 3, the file acquisition unit 300 acquires a data file as a candidate for learning data. In this example, the file acquisition unit 300 acquires a data file including image data generated by the scanner 4 via the Internet 7. The acquired data file is stored in the image processing server 2.
[0027] The operation identification unit 305 identifies a user operation on the data file acquired by the file acquisition unit 300. For example, the operation identification unit 305 identifies a user operation when the data file was generated or a user operation when the data file was viewed / edited with respect to the data file acquired by the file acquisition unit 300. The operation identification unit 305 in this example identifies a setting operation on the scanner 4 or the number of originals set at the time of generating the data file, or a viewing operation or an image editing operation on the user terminal 5 at the time of viewing / editing the data file with respect to the data file acquired by the file acquisition unit 300. The user operation identified by the operation identification unit 305 is stored in the image processing server 2 as an operation history.
[0028] The user evaluation unit 310 evaluates the reliability of each user based on the operation history of each user. The operation history for evaluating the reliability of a user is, for example, the number of times images were viewed, the ratio of the number of images viewed to the number of images scanned, the number of times the orientation of an image was manually corrected, the ratio of manual orientation correction to the number of images scanned, the number of manual corrections other than image orientation, or the ratio of manual corrections other than orientation to the number of images scanned. This is because the greater the number or ratio of images viewed by a user, the more likely they are to check the orientation of images. Similarly, it is considered that the greater the number or percentage of times a user manually corrects an image, the greater the possibility that the user has checked the orientation of the image. The user evaluation unit 310 of this example multiplies the number or percentage of images by a predetermined weighting coefficient, and evaluates the reliability of the user based on the sum of these values.
[0029] The orientation determination unit 315 determines the orientation of an image in a data file acquired by the file acquisition unit 300. In this example, the orientation determination unit 315 determines the orientation of an image using a machine learning model improved by the learning unit 340. The input to the machine learning model is the number of character strings in each area of the image (upper left, upper right, lower left, and lower right) and the positions of blank spaces, as exemplified in FIG. 2.
[0030] The automatic correction unit 320 automatically performs correction processing on the data file acquired by the file acquisition unit 300. For example, the automatic correction unit 320 performs image orientation correction processing, image quality correction processing, blank page removal processing, and crop processing on the data file acquired by the file acquisition unit 300 in accordance with the setting operation of the scanner 4. That is, the automatic correction unit 320 can switch between application / non-application of these automatic correction processes in accordance with the setting operation of the user.
[0031] When it is determined that an operation to instruct multiple consecutive scan processes (an operation to set multiple documents on the document tray of the scanner 4 and scan these documents in one batch) has been performed, the commonality evaluation unit 325 evaluates the commonality of multiple data files generated by the multiple scan processes. Generally, documents scanned in one batch often have the same orientation. Therefore, when documents are scanned in one batch and have a high commonality in the data files, it is determined that the data files are useful learning data regardless of whether they have been viewed by a user. Based on the determination result by the orientation determination unit 315, the commonality evaluation unit 325 in this example determines whether the orientations of images included in multiple data files generated by one batch of scanning are consistent.
[0032] The adoption / rejection determination unit 330 determines whether or not to adopt the data file acquired by the file acquisition unit 300 as learning data for machine learning based on the user's operation identified by the operation identification unit 305. The adoption / rejection determination unit 330 is an example of a determination unit according to the present invention. For example, the adoption / rejection determination unit 330 determines whether or not to adopt the data file as learning data based on the user's viewing operation or correction operation on the image identified by the operation identification unit 305 and the evaluation result by the user evaluation unit 310. That is ... user's terminal 6 (a highly reliable user) and the image viewed by the user on the user terminal 6 as useful learning data because the user has confirmed the correct orientation.
[0033] Furthermore, the acceptance / rejection determination unit 330 adopts as learning data a data file for which the operation identification unit 305 has identified that multiple sheets of documents have been scanned in one batch and the commonality evaluation unit 325 has determined that the image orientations are consistent. In this example, the same document ID is assigned to data files read in one batch, so the acceptance / rejection determination unit 330 can determine whether to accept the data files as learning data by comparing the determination results (orientations) of the orientation determination unit 315 for data files assigned the same document ID.
[0034] Furthermore, when the operation specifying unit 305 specifies a setting operation that disables the automatic correction process by the automatic correction unit 320, the adoption / rejection determination unit 330 adopts the data file as learning data. When the operation specifying unit 305 specifies a setting operation that disables the orientation correction process, the adoption / rejection determination unit 330 in this example adopts the data file generated by scanning with this setting as learning data. A data file scanned by a user with the scan setting of automatic orientation correction disabled is likely to be scanned by the user with paper fed in the correct orientation, and can be determined to be useful learning data regardless of whether the user has viewed it or not.
[0035] The feature extraction unit 335 extracts features to be used as learning data from the data file adopted as learning data by the adoption / rejection determination unit 330. In this example, the feature extraction unit 335 extracts the number of character strings and the positions of blank spaces in each area (upper left, upper right, lower left, and lower right) of the image.
[0036] The learning unit 340 performs machine learning for the orientation determination process using the features of the data file adopted as the learning data by the adoption / rejection determination unit 330. The learning unit 340 in this example generates a machine learning model for the orientation determination process based on the features extracted by the feature extraction unit 335 and the orientation of the image.
[0037] The service providing unit 345 displays or edits the data file in response to a user's viewing or editing operation on the data file acquired by the file acquiring unit 300. In this example, the service providing unit 345 provides viewing or editing of an image file as a Web service in response to a request from the user terminal 6. The editing functions provided to the user by the service providing unit 345 include a "skew correction" function and a "trimming" function in addition to an image orientation correction function. The "skew correction" function is a function used to correct a few degrees of skew (misalignment) that occurs when scanning an original, and the "trimming" function is a function used to cut out a part of an image. It is hypothesized that when using these functions, the user checks the image orientation to some extent, and this is added to the feature candidates.
[0038] FIG. 5 is a diagram illustrating an example of an image information table of a data file stored in the image processing server 2. As shown in FIG. 5, the image information table registers a user ID for identifying a user, a document ID for identifying a stack of documents (i.e., a batch), an image ID for identifying an image file, information indicating the contents of manual correction, information indicating the orientation of the image, information indicating the contents of automatic orientation correction, information indicating the browsing history of the image, and setting information indicating the settings at the time of scanning. That is, when a data file (image file) is acquired by the file acquisition unit 300, the image processing server 2 registers the user ID, document ID, image ID, and setting information in the image information table, and when an orientation determination process is performed on the acquired data file, the result of the orientation determination is added, and when the user browses, corrects the orientation, or manually corrects the data file, the image information table is updated in accordance with the user operations. The information indicating the content of the manual correction is, for example, "-1: uncorrected, 0: orientation, 1: deleted, 2: other". The information indicating the image orientation is information indicating the result of the determination by the orientation determination unit 315, for example, "-1: undetermined, 0: 0 degrees, 1: 90 degrees, 2: 180 degrees, 3: 270 degrees, 99: undetermined". The information indicating the content of the automatic orientation correction is information indicating the content of the image orientation correction by the automatic correction unit 320, for example, "-1: uncorrected, 0: 0 degrees, 1: 90 degrees, 2: 180 degrees, 3: 270 degrees". The information indicating the viewing history of the image is, for example, "0: unviewed, 1: viewed". The setting information indicating the settings at the time of scanning is, for example, a difference from the default setting, and is information indicating the content of the setting change by the user.
[0039] FIG. 6 is a diagram illustrating a viewing and editing screen provided by the service providing unit 345, and explaining a method for updating a viewing history, etc. 6, the service providing unit 345 displays a preview screen of the image (for each page) on the user terminal 6, and the operation identifying unit 305 determines that the image has been viewed when the image is loaded, and updates the "viewing history" information in the image information table (FIG. 5). At this time, the service providing unit 345 does not load images of pages that are not displayed on the screen of the user terminal 6, thereby increasing the certainty that the user has confirmed them. Furthermore, when the user manually corrects (orientation, deletion, etc.) an image (per page) on the user terminal 6 and saves it, it is determined that the user has performed the correct image correction operation, and the operation identification unit 305 updates the "manual correction" information in the image information table in Fig. 5 at the time the image is corrected and saved. If the orientation is corrected, the operation identification unit 305 also updates the "orientation correction" information. However, this "orientation correction" information is not used to determine the usefulness of the data, but is used for the correct label. That is, when the viewer is started by the user terminal 6, the image processing server 2 reads the image file and updates the "browsing history" information for the read image file. After that, when manual correction is performed by the user operation, the image processing server 2 updates the "manual correction" information. Every time a new page is displayed in the app by scrolling, the image processing server 2 repeats the above process.
[0040] 7 is a flowchart explaining the learning process (S10) of the image processing server 2. The learning process (S10) is operated periodically (once every two weeks) on the premise that the accumulated image files are deleted periodically, and by selecting data when the image files are deleted, duplication of learning data to be passed to the learning unit 340 can be avoided. As illustrated in FIG. 7, in step 100 (S100), the file acquisition unit 300 (FIG. 4) of the image processing server 2 reads the image files (data files) read by the scanner 4 in order. In step 105 (S105), the operation identifying section 305 refers to the image information table (FIG. 5) for the image file read by the file acquiring section 300, and identifies the user's operation on this image file.
[0041] In step 110 (S110), the user evaluation section 310 refers to the image information table to identify the user associated with this image file, and evaluates the reliability of the user based on the operation history of the identified user.
[0042] In step 20 (S20), the adoption / rejection determination unit 330 determines whether or not to adopt the image file read by the file acquisition unit 300 as learning data, based on the user operation identified by the operation identification unit 305 and the user reliability evaluated by the user evaluation unit 310. In step 115 (S115), if the acceptance / rejection determination unit 330 has adopted the data as learning data, the image processing program 3 proceeds to processing of S120, and if the acceptance / rejection determination unit 330 has rejected the data as learning data, the image processing program 3 proceeds to processing of S130.
[0043] In step 120 (S 120 ), the feature extraction unit 335 extracts features from the image file read by the file acquisition unit 300 . In step 125 (S125), the feature extraction unit 335 outputs the extracted feature and the correct label (image orientation) to the learning unit 340 as learning data.
[0044] In step 130 (S130), the file acquisition unit 300 deletes the image file that was read. In step 135 (S135), the image processing program 3 determines whether processing has been completed for all stored image files, and if processing has been completed for all image files, proceeds to processing of S140, and if there are unprocessed image files, returns to processing of S100 and reads the next image file. In step 140 (S140), the learning unit 340 uses the learning data input from the feature amount extraction unit 335 to generate a machine learning model for the orientation determination process.
[0045] FIG. 8 is a flowchart for explaining the adoption / rejection determination process (S20) of FIG. 7 in more detail. As shown in Fig. 8, in step 200 (S200), the acceptance / rejection determination unit 330 refers to the image information table (Fig. 5) and determines whether the loaded image file is one of a plurality of image files scanned in one batch. This determination is made based on whether there is another image ID in the image information table that is assigned the same document ID. If the acceptance / rejection determination unit 330 determines that the loaded image file is one of a plurality of image files scanned in one batch, the image processing program 3 proceeds to processing of S205, and otherwise proceeds to processing of S210.
[0046] In step 205 (S205), the commonality evaluation unit 325 compares multiple image files scanned in one batch to determine whether the image orientations match. The image orientations can be determined by referring to "orientation" in the image information table (FIG. 5). If the commonality evaluation unit 325 determines that the image orientations match, the image processing program 3 proceeds to processing of S225, and if the image orientations do not match, the image processing program 3 proceeds to processing of S210.
[0047] In step 210 (S210), the acceptance / rejection determination section 330 determines whether or not the reliability evaluation value by the user evaluation section 310 is less than a reference value. When the acceptance / rejection determination section 330 determines that the reliability evaluation value is less than the reference value, the image processing program 3 proceeds to processing of S230, and when the acceptance / rejection determination section 330 determines that the reliability evaluation value is equal to or greater than the reference value, the image processing program 3 proceeds to processing of S215.
[0048] In step 215 (S215), the adoption / rejection determination unit 330 determines whether the loaded image file has been viewed or edited by the user. If the adoption / rejection determination unit 330 determines that the image file has been viewed or edited, the image processing program 3 proceeds to processing of S225, and if the adoption / rejection determination unit 330 determines that the image file has not been viewed or edited, the image processing program 3 proceeds to processing of S220.
[0049] In step 220 (S220), the adoption / rejection determination unit 330 refers to the image information table (FIG. 5) to determine whether automatic orientation correction has been disabled in the settings of the scanner 4. The determination of whether automatic orientation correction has been disabled may refer to the setting information or the "orientation" information. When the adoption / rejection determination unit 330 determines that automatic orientation correction has been disabled, the image processing program 3 proceeds to processing of S225, and when the adoption / rejection determination unit 330 determines that automatic orientation correction has been enabled, the image processing program 3 proceeds to processing of S230. When automatic orientation correction has been intentionally disabled by the user, it is determined that it is highly useful, but since there are cases in which the document cannot be fed in the correct orientation even if the scan settings are disabled, the useful learning data is determined taking into account user reliability.
[0050] In step 225 (S225), the adoption / rejection determination unit 330 decides to adopt the read image file as learning data. That is, the adoption / rejection determination unit 330 adopts the image file as learning data if it is an image file scanned in one batch and all orientation determination results match, if it is an image file viewed or edited by a highly reliable user, or if automatic orientation correction has been disabled by a highly reliable user. In step 230 (S230), the acceptance / rejection determination unit 330 decides to reject the read image file as learning data.
[0051] As described above, according to the image processing system 1 of this embodiment, image files useful as learning data for the orientation determination process can be selected based on the user's operation. In other words, developers can select only the learning data that is useful for machine learning without browsing a large amount of images collected by the developer. This enables stable learning and improves the accuracy of orientation determination.
[0052] Although the embodiment of the present invention has been described, the above embodiment is presented as an example and is not intended to limit the scope of the invention. The above embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. The above embodiment and its modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and gist of the invention. [Explanation of symbols]
[0053] 1. Image Processing System 2. Image Processing Server 3. Image Processing Programs 4 Scanner device 6 User terminals
Claims
1. A file acquisition unit for acquiring a data file; an operation identification unit that identifies a user's operation on the data file acquired by the file acquisition unit; a user evaluation unit that evaluates the reliability of each user based on the operation history of each user; a determination unit that determines whether or not to adopt the data file acquired by the file acquisition unit as learning data for machine learning, based on the user operation identified by the operation identification unit and the user reliability evaluated by the user evaluation unit; A learning data selection device having the above configuration.
2. A file acquisition unit for acquiring a data file; an operation specifying unit that determines whether an operation for instructing to perform a plurality of consecutive scan processes has been performed when the data file acquired by the file acquisition unit is generated; a commonality evaluation unit that, when it is determined by the operation identification unit that an operation instructing to perform multiple consecutive scan processes has been performed, evaluates commonality among multiple data files generated by the multiple scan processes; a determination unit that determines whether or not to adopt the data file acquired by the file acquisition unit as learning data for machine learning based on a result of the commonality evaluation by the commonality evaluation unit; A learning data selection device having the above configuration.
3. A file acquisition unit that acquires a data file including image data; an operation identification unit that identifies a user's operation on the data file acquired by the file acquisition unit; a determination unit that determines whether or not to adopt the data file acquired by the file acquisition unit as learning data for machine learning related to an orientation determination process that determines an orientation of an image included in the data file, based on the user's operation identified by the operation identification unit; A learning data selection device having the above configuration.
4. The determination unit determines whether or not to adopt the data file as learning data based on a viewing operation or a modification operation on an image included in the data file; A learning unit that performs machine learning on orientation determination processing using the feature amount of the data file adopted as learning data by the determination unit. The training data selection device according to claim 3 , further comprising:
5. the commonality evaluation unit determines whether or not the orientations of images included in the plurality of data files are consistent; The determination unit adopts, as learning data, a data file determined by the commonality evaluation unit to have a matching image orientation, A learning unit that performs machine learning on orientation determination processing using the feature amount of the data file adopted as learning data by the determination unit. The training data selection device according to claim 2 , further comprising:
6. A file acquisition unit for acquiring a data file; an operation specifying unit that specifies a setting operation related to an automatic correction process of the data file acquired by the file acquisition unit; a determination unit that adopts the data file acquired by the file acquisition unit as learning data for machine learning when the operation identification unit identifies a setting operation for disabling the automatic correction process; A learning data selection device having the above configuration.
7. The operation identification unit identifies a setting operation related to a direction correction process of the data file, The determination unit adopts the data file as learning data when the operation identification unit identifies a setting operation for disabling the orientation correction process; A learning unit that performs machine learning on orientation determination processing using the feature amount of the data file adopted as learning data by the determination unit. The training data selection device according to claim 6, further comprising:
8. The operation identification unit identifies a setting operation by a user at the time of file generation of the data file acquired by the file acquisition unit, The determination unit determines whether or not to adopt the data file as learning data based on the user's setting operation identified by the operation identification unit.
8. The training data selection device according to claim 3, 4, 6 or 7.
9. a file acquisition step for acquiring a data file; an operation identification step of identifying a user's operation on the data file acquired by the file acquisition step; a user evaluation step of evaluating the reliability of each user based on the operation history of each user; a determination step of determining whether or not to adopt the data file acquired in the file acquisition step as learning data for machine learning, based on the user operation identified in the operation identification step and the reliability of the user evaluated in the user evaluation step; A method for selecting learning data comprising the steps of:
10. a file acquisition step for acquiring a data file; an operation identification step of identifying a user's operation on the data file acquired by the file acquisition step; a user evaluation step of evaluating the reliability of each user based on the operation history of each user; a determination step of determining whether or not to adopt the data file acquired in the file acquisition step as learning data for machine learning based on the user's operation identified in the operation identification step and the reliability of the user evaluated in the user evaluation step; A program that causes a computer to execute the following.
Citation Information
Patent Citations
Teacher data collection apparatus, teacher data collection method and program
JP2018124617A
Inspection system, identification system, and discriminator evaluation device
JP2019158684A
Machine learning device, data processing system, machine learning method, and data processing method
JP2020071271A
Production method of machine learning model and copying device
JP2020096290A
Property image classification device, property image classification method, learning method and program
JP2020112905A
