Data processing device, data processing system, data processing method, and program

Abstract

To provide an apparatus configured to facilitate processing to be executed on data by a user, a system, a method, and a program.SOLUTION: A data processing apparatus includes: acquisition means for acquiring data; determination means for determining the type of the data, based on text information included in the acquired data or text information converted from the data; execution means for executing one or more processes on the data according to the determined type; and control means for controlling data subjected to the one or more processes executed by the execution means to be output as one or more processing results.SELECTED DRAWING: Figure 4

Description

Technical Field

[0007] ,

[0001] The present invention relates to an apparatus, a system, a method for processing data, and a program for causing a computer to execute data processing.

Background Art

[0002] Data such as text, voice, image, and video accumulated by companies and the like is attached with attribute information such as data name, data type, creator, creation date and time, update date and time, and is managed based on these attribute information.

[0003] However, for data obtained by reading paper documents such as claim documents and contract documents with a scanner, even if the attribute information is described in the paper document, the computer cannot identify the attribute information, so the user determines and classifies it. This places a heavy burden on the user's work.

[0004] Therefore, a document attribute extraction sheet in which the types of attributes to be extracted and the description position IDs are set is created, and when scanning a paper document, the sheet is used as a cover and scanned, so that attributes are extracted from the scanned paper document and the extracted attributes are assigned. A technology has been proposed (see, for example, Patent Document 1).

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the above technology, since only attribute information is attached to and stored in the data, the type of data cannot be discriminated, and one or more processes cannot be executed on the data according to the discriminated type.

[0006] The present invention solves the above problems, and an object thereof is to provide an apparatus, a system, a method, and a program capable of executing one or more processes on data according to the discriminated type.

Means for Solving the Problems

[0007] According to the present invention, a data processing device for processing data, A means of acquiring data, A determination means for determining the type of data based on the text information contained in the acquired data or the text information converted from said data, An execution means that performs one or more processes on the data according to the type identified, Control means that controls the output of one or more processed data as processing results, which have been processed by the execution means. The present invention provides a data processing device that includes [a specific feature / function]. [Effects of the Invention]

[0008] According to the present invention, one or more processes can be performed on the data depending on the type that has been identified. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing an example configuration of a data processing system. [Figure 2] A diagram showing an example of a server hardware configuration as a data processing device. [Figure 3] A diagram illustrating the relationships between various software components implemented on a server. [Figure 4] A block diagram showing an example of the server's functional configuration. [Figure 5] A flowchart illustrating an example of the process performed by a server, from receiving data to storing it. [Figure 6] This diagram shows an example of a table that stores data numbers in association with user IDs. [Figure 7] This diagram shows an example of the data extraction settings configured for a given data type. [Figure 8] This diagram shows an example of tagging data based on its identified type and extracted content. [Figure 9] This diagram shows an example of software configuration for different data types. [Figure 10]A flowchart showing an example of the process of converting to text data. [Figure 11] A diagram explaining the types of software and the processes performed by each type of software. [Figure 12] A flowchart showing an example of the process after data storage executed by a server. [Figure 13] A diagram showing an example of a screen displayed on an operation device used by a user. [Figure 14] A diagram showing a first example of a screen displayed when data is selected on an operation device. [Figure 15] A diagram showing an example of a screen displayed when software 1 is selected on the screen shown in FIG. 14. [Figure 16] A diagram showing a second example of a screen displayed when data is selected on an operation device. [Figure 17] A diagram explaining the outline of a quality analysis service in the food industry using a data processing system. [Figure 18] A diagram showing an example of a screen displaying classification results on an operation device. [Figure 19] A flowchart showing the process flow from data acquisition to classification result display executed by a server.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, the present invention will be described with embodiments, but the present invention is not limited to the embodiments described below.

[0011] FIG. 1 is a diagram showing a configuration example of a data processing system. The data processing system includes a server 10 as a data processing device that implements a plurality of software and processes data by executing the software, and an operation device 11 that handles the software stored in the server 10. The data processing system includes a capturing device that acquires data and transmits it to the server 10. The operation device 11 also has a function as a capturing device. Note that the data is not limited to one-page data and may be data of multiple pages.

[0012] The form of the data may be any of image data, audio data, and text data. As an example of audio data, audio data of a customer's voice received at a call center can be cited. As an example of text data, SNS (Social Network Service) information such as blogs and tweets can be cited. The customer's audio data and SNS information include inquiries about products and services, and the inquiries include the content of specified items such as satisfaction, dissatisfaction, and requests for products and services. As an example of image data, image data of a contract, an invoice, a report (such as a business daily report and a work report) can be cited. The image data may be still image data or moving image data. Note that audio data and image data other than text data are converted into text data using speech recognition and character recognition (OCR). Since speech recognition and character recognition are well-known technologies, they are not described in detail here.

[0013] Server 10, operating device 11, and capturing device are connected to network 12 and communicate with each other via network 12. Server 10 can be installed, for example, on the cloud. Multiple software programs implemented by Server 10 include, for example, data conversion software, translation software, invoice issuance software, contract deadline management software, and software for classifying and analyzing inquiry content. These are just examples and are not limited to these programs. Server 10 acquires data via network 12, executes software based on operations from operating device 11 to process the data, and transmits the processing results to operating device 11. Here, data processing involves conversion, classification, and matching to obtain necessary information from the data, and specifically, the above-mentioned software performs data format conversion, translation, and classification and analysis of inquiry content.

[0014] The operating device 11 can be any device that is used by a user of the data processing system and is capable of operating the software implemented on the server 10. Examples of operating devices 11 include smartphones, tablet devices, PCs (Personal Computers), PDAs (Personal Digital Assistants), etc.

[0015] The capturing device can be any device that has the function of acquiring data and transmitting the acquired data to the server 10. Examples of capturing devices include the operating device 11, an MFP (Multi-Function Peripheral) 13, a printer 14, a webcam 15, an imaging device that captures images in all directions (360-degree camera) 16, a microphone 17, and a hearable device 18, which is a computer worn on the ear. Other examples of capturing devices include a PJ (Projector), an IWB (Interactive White Board) 19, industrial machinery, networked home appliances, automobiles (Connected Car), game consoles, wearable PCs, etc.

[0016] Network 12 may be a LAN (Local Area Network), WAN (Wide Area Network), the Internet, etc., and may be either a wired network or a wireless network. Furthermore, Network 12 is not limited to a single network, but may consist of two or more networks connected by relay devices such as routers.

[0017] In the example shown in Figure 1, the server 10, the operating device 11, and the capturing device are each separate devices, but this is not the only option. A single device may have the functions of both the server 10 and the capturing device, and the data processing system may consist of this device and the operating device 11. Alternatively, the data processing system may consist of the server 10 and an operating device that has the functions of a capturing device.

[0018] Figure 2 shows the relationships between the various software implemented in server 10. Server 10 stores various software in order to perform processing on the acquired data. The various software consists of main software 20 and various software 21-24, each with a different function, and the main software 20 and the various software 21-24 work together to perform processing.

[0019] The main software 20 has the function of analyzing the content of text data using natural language processing with AI (Artificial Intelligence). AI is based on machine learning, which learns patterns and other elements from large amounts of data and provides answers to given tasks. Examples of machine learning include deep learning, which constructs a hierarchical model using a neural network that represents the human nervous system and performs inference using the hierarchical model. Neural networks are divided into supervised learning, which learns to optimize itself to correct training data given to input data, and unsupervised learning, which does not require training data. By providing data and performing machine learning, AI achieves highly accurate natural language processing.

[0020] Natural language processing is the technology that enables computers to process natural language used by humans in everyday life. Techniques for processing Japanese include morphological analysis, syntactic analysis, semantic analysis, and contextual analysis. Morphological analysis is the process of dividing a word into the smallest possible units that cannot be further divided into meaningful words. Syntactic analysis is the process of analyzing the relationships between each word divided by morphological analysis, and understanding the dependency structure of a sentence. Semantic analysis is the process of determining which word to select to achieve the correct meaning when a single word may have two or more possible dependencies as determined by syntactic analysis. Contextual analysis is the process of analyzing the relationships between sentences.

[0021] The main software 20 analyzes the content of text data strings using natural language processing and extracts the type of text data and specific information corresponding to that type. The type of text data is classified by document format, purpose, etc. For example, when classified by document format, it could be invoices, contracts, technical documents, reports, etc., or in the food industry, it could be inquiries, food safety, manufacturing and distribution, etc., as a way to utilize customer feedback. Here, the food industry is used as an example, but it is not limited to the food industry and may also be used in the construction industry, transportation and communication industry, finance and insurance industry, retail industry, food and beverage industry, real estate industry, etc. If the type of text data is "invoice," "contract," "technical document," "report," etc., the type can be determined from the characters in that range of the page by pre-training the software using machine learning. Note that it is not limited to a specific range of the page, and the type of text data can also be determined from the analyzed content. For example, if it contains characters such as billing address, billing amount, and payment deadline, the type of text data can be determined to be an invoice.

[0022] When the type of text data falls under the categories of "inquiry," "food safety," and "manufacturing / distribution" for the intended use of customer feedback, it is possible to determine which of these uses it is based on the terminology and context used in the text data. Furthermore, each of these categories can be further subdivided to determine which category it belongs to. The categories shown are just examples of category information.

[0023] Examples of categories under "Inquiries" include "Requests / Suggestions" and "Comments / Complaints." Examples of categories under "Food Safety" include health hazards such as "Digestive Symptoms" and "Headache / Fever." Examples of categories under "Manufacturing and Distribution" include manufacturing and distribution quality issues such as "Foreign Object Contamination" and "Container Damage / Packaging Box Deformation." These categories are just examples and are not exhaustive.

[0024] Specific information, in the case of an invoice, refers to the content of pre-specified items such as the invoice amount, billing address, and payment deadline. This specific information can also be extracted from text and numbers within a specified area of ​​a page by pre-training the system using machine learning to determine where that content resides. Furthermore, similar to the data type, specific information can be extracted from the analyzed content; for example, numbers and characters following text such as the invoice amount, billing address, and payment deadline can be extracted. In the case of a query, the information can be extracted from the analyzed content, specifically as phrases or sentences containing specified terms such as "I want to...", "I hope to...", or "I request...".

[0025] The various software programs 21-24 are software designed to implement various functions. These include software for data conversion, translation, invoice generation, contract deadline management, and software for classifying and analyzing inquiry content.

[0026] The main software 20 passes the acquired data to various software programs 21-24, manages the processed data processed by the various software programs 21-24 in association with it, and also manages the linkage to the various software programs 21-24. The linkage to the various software programs can be managed by management information that specifies which software should process which type of data.

[0027] Figure 3 shows an example of the hardware configuration of server 10. Server 10 has hardware that stores and executes the software shown in Figure 2. Specifically, server 10 is built as a computer and includes a CPU (Central Processing Unit) 30, ROM (Read Only Memory) 31, RAM (Random Access Memory) 32, HD (Hard Disk) 33, and HDD (Hard Disk Drive) controller 34. Server 10 also includes a display 35, an external device connection interface 36, a network interface 37, a data bus 38, a keyboard 39, a pointing device 40, a DVD-RW (Digital Versatile Disk Rewritable) drive 41, and a media interface 42.

[0028] The CPU 30 controls the overall operation of the server 10 and executes the main software 20 and various software programs 21-24. The ROM 31 stores programs used to drive the CPU 30, such as the IPL (Initial Program Loader). The RAM 32 provides the CPU 30's workspace. The HD 33 stores the main software 20 and various software programs 21-24 executed by the CPU 30, the OS (Operating System), data acquired from the capturing device, and processed data obtained by software processing of that data. The HDD controller 34 controls software reading and data writing to the HD 33 according to the CPU 30's control.

[0029] The display 35 displays various information such as cursors, menus, windows, characters, and images. The external device connection interface 36 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printer 14. The network interface 37 is an interface for data communication using the network 12. The data bus 38 is a bus for electrically connecting various components such as the CPU 30.

[0030] The keyboard 39 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 40 is a type of input means for selecting and executing various instructions, selecting processing targets, moving the cursor, etc. The DVD-RW drive 41 controls the reading and writing of various data to a DVD-RW 43, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may also be DVD-R, etc. The media I / F 42 controls the reading and writing (storage) of data to a recording medium 44 such as flash memory.

[0031] Figure 4 is a block diagram showing an example of the functional configuration of server 10. Server 10 implements each function by having one or more processing circuits execute the main software 20 and various software 21-24. The processing circuit can be the CPU 30 shown in Figure 3, but it may also be an ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or conventional circuit module or other device designed to execute each function.

[0032] Server 10 comprises, as functional units, a receiving unit 50, an association unit 51, a discrimination unit 52, an extraction unit 53, an execution unit 54, a storage unit 55, a control unit 56, and a transmission unit 57. The receiving unit 50 also functions as an acquisition unit, receiving and acquiring data from the capturing device. The receiving unit 50 receives the user ID and password as user identification information entered by the user when logging in. The receiving unit 50 also receives instructions from the user. The user ID and password may be entered using an input device such as a keyboard on the capturing device, or they may be stored on an IC card or the like and read by a reader. Furthermore, the user ID and password are not limited to a combination of letters, numbers, symbols, etc., but may also be biometric information such as the user's fingerprints or vein patterns.

[0033] The association unit 51 assigns identification information to the acquired data to identify the data. The identification information can be a sequence of numbers (data numbers). However, the identification information is not limited to sequence of numbers; it may also be a randomly generated number or a combination of two or more numbers, letters, symbols, etc. For example, "10000" can be assigned to data 1, and a different identification information "10001" can be assigned to other data 2.

[0034] The discrimination unit 52 determines the type of data based on the text data contained in the acquired data. The extraction unit 53 extracts specific information from the text data based on the extraction content associated with the type of data. The type determination by the discrimination unit 52 and the extraction by the extraction unit 53 can be performed by natural language processing using the AI ​​described above. However, methods other than natural language processing may be used as long as the type and specific information can be extracted from the text data. The execution unit 54 performs one or more processes on the data according to the determined type of data. One or more software programs are set for each type of data to implement one or more processes.

[0035] The association unit 51 associates the identified type and extracted information as tags with the data number and user ID. The association unit 51 also assigns identification information to the processed data executed by the execution unit 54. Since the data before and after processing by the execution unit 54 are not the same data, the identification information is not the same. However, to distinguish it from other data, it is possible to assign identification information with the same data number but with different characters added. For example, if the original data 1 before processing is assigned the identification information "10000", then the processed data 1 can be assigned the identification information "10000a", which has the character "a" added to it.

[0036] The storage unit 55 stores the original data and the processed data in association. The storage unit 55 stores the data number and the user ID in association, and stores the extraction content set for the data type, the software to be executed, and data tagged with the type and extracted content.

[0037] The transmission unit 57 transmits search screen information and a list of data. The transmission unit 57 also transmits at least one of the following: the original data, a list of processed data, or a list of other processes. The list of other processes may include a list of software for the user to select.

[0038] The control unit 56 receives user instructions from the operating device 11, such as inputting search information, selecting data from a data list, selecting processed data from a processed data list, and selecting software from a software list. When the control unit 56 receives the selection of original data or processed data, it retrieves the data from the storage unit 55, transmits it to the operating device 11 via the transmission unit 57, and instructs the display unit to display it as output. Note that the output is not limited to display on the display unit, but may also be displayed by projection or printed on a recording medium such as paper. If software is selected, the control unit 56 instructs the execution unit 54 to start the software and execute processing using the processed data.

[0039] Figure 5 is a flowchart showing the processing flow from data acquisition to data storage performed by the server 10. Processing starts from step 100 after one of the capturing devices acquires data and sends it to the server 10. In step 101, the receiving unit 50 receives and acquires the data sent from the capturing device. The data may be sent to the server 10 from a PC or smartphone via the network 12, or it may be image data converted into text by a device with OCR (Optical Character Recognition) functionality, such as the MFP 13, and then sent to the server 10 via the network 12. Alternatively, the data may be audio data converted into text by speech recognition and then sent to the server 10. The server 10 may also be equipped with OCR functionality and speech recognition functionality, receive image data and audio data, and convert them into text data within the server 10.

[0040] When a user sends data from a capturing device, they can enter login information and log in to the system. In step 102, the receiving unit 50 receives and obtains the user ID entered during login. This explanation describes the case where the user logs in on the capturing device and the user intentionally sends data, but it is not limited to this. Therefore, this process may be executed when the data is saved to a specific folder on the server 10, or it may be triggered when printing or scanning is performed on the MFP 13. If the user has not logged in on the capturing device, the user logs in by entering their user ID and password before sending the data. This allows the server 10 to obtain the data and the user ID.

[0041] In step 103, the association unit 51 assigns identification information to the acquired data. Figure 6 shows an example of identification information assigned to the acquired data. The identification information is a data number consisting of digits, and each time data is acquired, a consecutive number is assigned by adding 1. Note that the identification information is not limited to consecutive digits as described above, but may include random numbers, letters, symbols, etc., as long as it can identify the data.

[0042] Referring again to Figure 5, in step 104, the discrimination unit 52 determines the type of data from the text data contained in the data. The type of data is determined by reading the content of the text contained in the data using AI. The classification of the type changes depending on what classification function is given to the AI, and can be, for example, the type of document format (invoice, contract, purchase order, etc.). This is just an example, and it could also be the type of use of customer feedback (inquiry, food safety, manufacturing and distribution, etc.). The association unit 51 uses the information about the type determined by the discrimination unit 52 as a tag and tags the data.

[0043] In step 105, the extraction unit 53 extracts specific information from the text data contained in the data. The specific information is the pre-set extraction content for the data type as shown in Figure 7. If the data type is an invoice, the extracted content includes the amount, recipient, payment due date, etc. If the data type is a contract, the extracted content includes the contract details, contracting party, period, etc. If the data type is a technical document, the extracted content includes the summary, technical field, etc. Since machine learning has been used to learn which pages and ranges of these documents generally contain this information, it is possible to extract it from the content of that page and range. If there is a subtitle such as "summary" or "technical field," the part following that subtitle can be extracted as the content of the summary or technical field. If there is no subtitle such as "summary" or "technical field," it is possible to extract from the above range, or to extract from the content itself. When extracting from the content, keywords, etc., can be extracted, and the summary or technical field can be created from these keywords, etc.

[0044] Referring again to Figure 5, in step 106, the association unit 51 tags the extracted content as tags for the data. Figure 8 shows an example of tagging data with type and content. The data is identified by a data number, and the user ID, type, and content such as amount and destination are associated with the data number.

[0045] Referring again to Figure 5, in step 107, the execution unit 54 executes one or more processes using one or more software programs depending on the type of data. These processes include saving and registering data. The software used for processing based on the data type is stored in the server 10 as a database, as shown in Figure 9. In Figure 9, software A is set for invoices, and software B and C are set for contracts. Therefore, if the data type is identified as an invoice, the execution unit 54 executes only the process using software A; if it identifies as a contract, it executes both processes using software B and C.

[0046] The system administrator or user can arbitrarily select and pre-configure which software to set for each type of data. Users can also associate and configure software they frequently use. A database like the one shown in Figure 9 can be created for each user, associated with user IDs, etc. For example, software B and C can be set for contracts for user X, and only software B can be set for contracts for user Y. Therefore, server 10 can refer to this database and, taking user IDs into consideration, determine which process to execute. A database can be created for each specific user, and a separate database can be created for all users except those specified.

[0047] Referring again to Figure 5, in step 108, the association unit 51 assigns identification information to the processed data executed by the execution unit 54. This makes it possible to associate the original data with the processed data. The association unit 51 can also assign information about the software that performed the processing to this data. This information about the software may be, for example, the software name.

[0048] In step 109, the storage unit 55 stores the processed data associated with the original data. By storing the processed data in association with the original data in this way, the user can retrieve the processed data at the time they need it. Then, the process proceeds to step 110, and the process up to data storage is completed.

[0049] If the original data contains text, the data type can be determined from the text and the content extracted. However, the original data does not necessarily contain text, such as image data or audio data. Also, the original data may be in PDF (Portable Document Format) format, where fonts and layouts do not change depending on the type of device, and therefore may not be recognized by character recognition.

[0050] In the case of such data, it is necessary to extract or convert the data into text data. Therefore, between steps 101 and 102 shown in Figure 5, it is possible to determine whether the data contains text data, and if it does not, to perform a process to convert it into text data.

[0051] Figure 10 is a flowchart showing an example of the process of converting data to text data. After acquiring data in step 101 shown in Figure 5, this conversion process starts from step 200. Server 10 may further include a determination unit, a text data acquisition unit, and a conversion unit to execute the conversion process. In step 201, the determination unit determines whether the acquired data contains text data. If it determines that the data contains text data, the process proceeds to step 207 and the conversion process ends.

[0052] If it is determined in step 201 that there is no text data, the process proceeds to step 202, where the determination unit determines whether the data is image data or PDF data (PDF data). Whether the data is image data or PDF data can be determined from the attribute information attached to the data. The attribute information includes information about the file type when the data is recorded as a file. Note that this method is just one example, and other methods may be used for determination.

[0053] If the data is determined to be image data or PDF data in step 202, the process proceeds to step 203, where the text data acquisition unit acquires the text data contained in the image data or PDF data. As a method for acquiring text data from image data or PDF data, a method of acquiring text data using character recognition processing such as OCR can be used. After acquiring the text data, the process proceeds to step 207, and the conversion process ends.

[0054] If it is determined in step 202 that the data is neither image data nor PDF data, the process proceeds to step 204, where the determination unit determines whether the acquired data is audio data or not. Whether or not the data is audio can also be determined from attribute information, similar to image data. Note that this method is just one example, and other methods may be used to determine whether or not the data is audio. If it is determined in step 204 that the data is audio, the conversion unit converts the audio data into text data. One method for converting audio data into text is to use an AI-powered speech recognition engine. Speech recognition processes noise and static, identifies phonemes from sound waves, identifies sequences of phonemes and converts them into words, generates sentences from the sequence of words, and outputs them as text. A phoneme is the smallest constituent unit of sound. After converting to text data, the process proceeds to step 207, and the conversion process ends.

[0055] If it is determined in step 204 that the data is not audio data, the process proceeds to step 206, where the determination unit classifies the acquired data as other data. Then, the process proceeds to step 207, and the conversion process ends. Therefore, data classified as other data is not converted into text data, and it is not possible to determine the type of data or extract its content from it.

[0056] Incidentally, software for implementing various processes can be broadly classified into two types, as shown in Figure 11. The first type is complete-process software, where the process is completed in a single sequence. Figure 11(a) shows an example of processing performed by complete-process software. In Figure 11(a), the process is completed by importing data and executing the process, and the processed data is generated. An example of complete-process software is software that imports data and converts it to PDF. In this case, the processed data is PDF data.

[0057] The second type is processing-selection software, which has multiple processing flows. With processing-selection software, the user is responsible for deciding which processing flow to select in their usage scenario. Figure 11(b) shows an example of processing performed by processing-selection software. In Figure 11(b), data is imported, registered and managed, the user selects the subsequent processing, and the selected processing is executed. An example of processing-selection software is software that imports and registers invoices and manages them with tags, etc. Subsequent processing involves other processes such as making decisions on payment execution and approval of reviews, and these are executed by the user using the corresponding software.

[0058] In processing using selection-type software, data is automatically input based on the content extracted from the data and registered as data for subsequent processing in the software. By launching the software, the user can perform automatic output, analysis, and other operations on the registered data. Therefore, the user can execute the necessary processing with minimal effort at the time required.

[0059] Furthermore, the main software may possess some or all of the functions of processing-complete software or processing-selection software.

[0060] Figure 12 is a flowchart showing an example of processing performed by the server 10 after storing the data and the processed data. After storing the data and the processed data, the server 10 starts processing from step 300 in response to an operation of the operation device 11 at any time. The user enters a user ID and password, and the server 10 accepts these inputs in step 301.

[0061] In step 302, the control unit 56 of the server 10 transmits the information on the search screen to the user-operated device 11 via the transmission unit 57, and displays it on the display unit of the user-operated device 11. The user refers to the displayed search screen and enters search information according to the instructions on the screen. In step 303, the receiving unit 50 receives the search information input from the user. In step 304, the control unit 56 searches for data according to the search information and transmits a list of the searched data to the user-operated device 11 via the transmission unit 57, and displays it.

[0062] Figure 13 shows an example of a search screen displayed on the display unit of the operating device 11. The search screen has checkboxes 60 for selecting the search target and search type as search information. The search target is either "Show all members of the group to which you belong" or "Show items associated with your ID," and the user selects one of them.

[0063] The search options are "Word Search" and "Category Search," and the user selects one of them. If "Word Search" is selected, an input field will appear, allowing the user to enter words or phrases contained in the text data as keywords. If "Category Search" is selected, categories such as the type of text data will be displayed. The types of text data are "Invoices," "Contracts," and "Internal Documents." A time period will also be displayed. The time periods are "Last Week," "Last Month," and "All Time." These are subdivided categories, and the user can select the option that corresponds to the data they want to retrieve. Note that these options are just examples, and the user is not limited to these.

[0064] When a user selects a data type and time period, server 10 searches for data stored for the selected type and time period, and sends a list of the retrieved data to the user's operating device 11. The operating device 11 displays the data list, and the user can select one data item from the list. Each data item displayed in the data list is shown as a thumbnail, indicating the type of data identified and the extracted content.

[0065] Referring again to Figure 12, in step 305, the control unit 56 transmits and displays the processed data associated with the data selected by the user via the transmission unit 57.

[0066] Figure 14 shows a first example of the screen displayed when data is selected on the operating device 11. The screen displays a thumbnail 61 of the original data, a thumbnail 62 of the processed data processed by the processing completion software, and icons 63 and 64 of the processing selection software.

[0067] Below thumbnail 61, content information such as the data type, number of pages, file size, and extracted content is displayed. Below thumbnail 62, a button 65 is displayed that allows access to the software. The user selects one of thumbnails 61, 62, or icons 63 or 64 by clicking or tapping, etc., to request the display of the original data, the display of data processed by processing-type software, or the execution of processing by processing-selection type software.

[0068] Referring again to Figure 12, in step 306, the determination unit determines whether the thumbnail 61 of the original data was selected from the thumbnails 61, 62 and icons 63, 64 selected by the user on the operating device 11. If the original data is selected in step 306, the process proceeds to step 307, where the control unit 56 transmits the original data via the transmission unit 57 and displays it on the display unit of the operating device 11.

[0069] If data other than the original data is selected in step 306, the process proceeds to step 308, where the determination unit determines whether or not a thumbnail 62 of the processed data executed by the processing-complete software has been selected. If processed data is selected in step 308, the process proceeds to step 309, where the control unit 56 transmits the processed data via the transmission unit 57 and displays it on the display unit of the operation device 11. If there are multiple thumbnails 62 of the processed data, the control unit 56 transmits and displays the processed data corresponding to the selected thumbnail.

[0070] If it is determined in step 308 that no processed data has been selected, it indicates that one of the processing selection software programs has been selected, and the process proceeds to step 310. The control unit 56 starts the selected processing selection software, transmits the registered data via the transmission unit 57, and displays the page of that data on the display unit of the operation device 11. Then, it performs a predetermined process on the displayed page.

[0071] After displaying the data on the operating device 11 in steps 307, 309, and 310, the process proceeds to step 311, and the server 10 terminates its processing. The registered data is the data of the page associated with the data selected by the user. If the original data is audio data, the audio data cannot be displayed on the screen. Therefore, the original data can be text data converted from the audio data.

[0072] Figure 15 shows an example of a screen that appears when software D is selected in the screen shown in Figure 14. Software D is software that performs predetermined processing on, for example, invoices received from other companies. When software D is selected by clicking or tapping in the screen shown in Figure 14, software D is launched.

[0073] Generally, to open an invoice page, it is necessary to launch the software and then select the data. However, in this process, as shown in Figure 15, upon launching software D, information extracted by pre-language processing is automatically entered into the input fields of a designated form, and the registered data is displayed.

[0074] The screen displays an input / edit button 66 and execution buttons 67-69. The input / edit button 66 allows the user to directly input information for items that are not listed, and to correct items that are incorrectly entered. The execution button 67 is for requesting supervisor approval, sending the automatically entered data, such as an invoice or created slip, to the supervisor for approval. Sending to the supervisor can be done via email or other means. The execution button 68 is for forwarding the approved slip and requesting payment. The execution button 69 is for creating a slip that includes the payee and amount. Note that these buttons are just examples of buttons for executing various processes on data registered in software D, and are not limited to these processes.

[0075] By using data with pre-extracted information registered in this way, users can open the software at the time they want to execute a process and have the server 10 execute the process without having to input any data.

[0076] Figure 16 shows a second example of the screen displayed when data is selected on the operating device 11. The screen displays a thumbnail 61 of the original data and a thumbnail 62 of the processed data processed by the processing-complete software. In the example shown in Figure 15, the processing selection software is represented not by an icon, but by execution buttons 70 and 71 for executing each process.

[0077] In the case of an icon, the software corresponding to the icon is launched, the registered data is displayed, and then the prescribed software is launched to perform processes such as requesting supervisor approval. On the other hand, with an execute button, the prescribed software is launched simply by pressing the button, thus reducing the effort required from the user.

[0078] The Execute button 70, like the Execute button 67 shown in Figure 15, is a button for requesting supervisor approval and automatically sends documents such as invoices with data entered to the supervisor. Sending to the supervisor can be done via email or other means. The Execute button 71 is a button that launches proprietary document analysis software. The document analysis software analyzes the content, checks for extensions of document submission deadlines, and enables the submission of documents while taking deadlines into consideration. Note that these processes are just examples, and the processes that should be executed are not limited to these.

[0079] Server 10 can perform pre-configured processes on data acquired from the capturing device and store the data after each process. It can also perform two or more consecutive processes from a pre-configured workflow on the data, store the data after those processes, and then, upon instruction from the user, execute the remaining processes of that workflow.

[0080] Up until now, we have explained the example of classifying data by document format, but data can also be classified by other criteria. One example of such other criteria is classifying customer feedback by its intended use. Customer feedback data can be automatically analyzed and classified from three perspectives of intended use: "inquiries," "food safety," and "manufacturing and distribution." The accuracy of the information can be visualized, and the importance of the information can be linked and displayed. This allows anyone to easily extract classified customer feedback and makes it easier to perform data analysis that aligns with the purpose. It should be noted that classification is not limited to these criteria, and other criteria may also be used.

[0081] An example of classification by application is explained in detail below. Figure 17 is a diagram illustrating the overview of a quality analysis service in the food industry using a data processing system. The device used by the operator who receives customer feedback (PC, headset, etc.) is the capturing device 80, and the device that analyzes and classifies the data from the capturing device 80 and performs processing such as associating importance is the server 10. The devices installed in each department that handles the classified data are the operation devices 11.

[0082] Customers who purchase food products manufactured and sold by a company may respond to surveys, post on social media, or make complaints by phone regarding their purchases. This customer feedback is valuable information for product development, quality assurance, manufacturing, and delivery. A company has a marketing department and product planning department that handle information useful for product development, a quality assurance department and customer service department that handle information useful for product assurance, and a manufacturing department and logistics department that handle information useful for manufacturing and delivery.

[0083] Customer feedback is received centrally by operators, so it needs to be categorized into data corresponding to each department. Server 10 analyzes and categorizes the data transmitted from the capturing device 80 using AI, lists it with its importance level linked to it, and transmits it to the operation devices 11 installed in each department for display. Because the data is listed with its importance level linked to it, it is possible to immediately grasp what kind of important food safety issues exist, and this can be used to improve quality, etc.

[0084] The capturing device 80 uploads customer feedback to the server 10. The data to be uploaded is a text file such as a CSV (Comma Separated Values) file, in which values ​​and items are separated by commas. The AI ​​is a pre-trained AI that has been trained using training data, and can be used immediately without keyword registration. The AI ​​understands the context of the text data contained in the data and classifies it into which type and category it belongs. The AI ​​receives keyword input and predicts and suggests related words of interest. Therefore, the server 10 can organize the important information, list it, send it to the operation device 11, and display the classification results.

[0085] The types of data include "inquiries" related to product development, "food safety" related to quality assurance, and "manufacturing and distribution" related to product manufacturing and delivery. Within "inquiries," there are categories such as "praise / encouragement," "requests / suggestions," "inquiries / consultations," and "criticisms / complaints." For example, inquiries such as "I want to be able to take it out quickly," "I want to prevent drinking too much," and "I want to finish drinking it" can be classified as "requests / suggestions" because they contain terms related to requests such as "I want to..." in their context. The marketing and product planning departments can then plan and propose offering the product in small packs based on these requests. This allows for understanding major customer trends, extracting requests and suggestions, and utilizing them in strategic marketing activities.

[0086] Within the category of "food safety," there are various categories such as "digestive symptoms," "headache / fever," "rash / skin symptoms," "constipation," "oral / throat symptoms," "vague symptoms," "symptoms in other areas," and "no health hazards." For example, comments such as "My stomach rumbles after drinking it," "My stomach has been upset lately," and "I've been getting diarrhea more easily" can be analyzed and classified under the "digestive symptoms" category because they indicate a state of incomplete digestion. The quality assurance department and customer service department can detect such conditions and verify the quality of the product. In this way, quality problems can be identified comprehensively, and appropriate feedback can be provided.

[0087] Within the "Manufacturing and Distribution" category, there are categories such as "Foreign object contamination," "Unusual odor / altered taste," "Damaged container / deformed packaging," "Defective expiration date / allergy labeling," and "Other." For example, complaints such as "The box was badly dented," "It's covered in scratches, I want to return it," and "The packaging is terrible. Is the contents okay?" can be analyzed and classified under the "Damaged container / deformed packaging" category because they relate to the container or packaging. The manufacturing and logistics departments can then check the condition of the containers and packaging upon receiving such complaints. In this way, quality issues can be fed back comprehensively without subjective bias.

[0088] Figure 18 shows an example of a screen displaying the classification results on the operating device 11. The screen consists of a field 90 that shows the type of data and a field 91 that shows categories, importance, etc. Field 90 shows three types: "Inquiry," "Food Safety," and "Manufacturing and Distribution," and currently "Food Safety" is selected.

[0089] Column 91 includes the data content 92, the data analysis date 93, the category 94, the accuracy 95, and the importance 96. The data content 92 may be the entire text content included in the data, only the text portion related to food safety, or a summarized version. The data content 92 may be displayed in an identifiable manner by changing the color or inverting the display of specific information extracted when classifying into categories.

[0090] If the data is image data or audio data, the content of the transcribed data will be displayed. However, address information such as the path name or URL (Uniform Resource Locator) indicating the storage location of the image data before transcribing, a thumbnail of the image data before transcribing, and an audio mark 97 may also be displayed along with the data content. This allows users to compare the text or audio in the image with the transcribed data and verify whether the transcription is accurate. In this case, an audio mark 97 is displayed, and the audio data can be played by clicking the audio mark 97 with a mouse or other device.

[0091] Accuracy 95 is an index that indicates how accurately an AI performing natural language processing can classify a given item based on its contextual analysis. Accuracy 95 can be expressed as a degree of certainty, with a value closer to 1 indicating higher accuracy.

[0092] The Importance 96 is an index that indicates the degree of health damage as perceived from the data. The Importance 96 can be expressed in three levels, for example. Level 1 is a level with no health damage at all, such as a broken container, where there is no need to see a doctor. Level 2 is a level with some health damage, such as a chipped tooth from hard food, where a doctor may or may not be needed. Level 3 is a level with health damage, such as diarrhea or vomiting, where a doctor is needed. Here, the Importance 96 is expressed in three levels, but the levels are not limited to three; there may be two levels, or four or more levels.

[0093] Even if "Manufacturing and Distribution" is selected in column 90, the importance level of 96 is used as an indicator of the degree of health damage and can be expressed in three stages. However, if "Inquiry" is selected, the importance level of 96 is not displayed because it does not relate to health damage.

[0094] Each data item in column 91 (data content 92, etc.) is associated with a number 98, such as a data number. Therefore, each item in column 92 can be identified by its number 98. In the example shown in Figure 18, the data is displayed in columns 90 and 91, but only the classification results may be displayed. Also, while column 91 displays all of the data content 92, data analysis date 93, category 94, accuracy 95, and importance 96, it is not limited to this. Therefore, column 91 may display only the data content 92, or only the data content 92 and the data storage location, or only one of the accuracy 95 or importance 96 may be displayed along with the data content 92.

[0095] Figure 19 is a flowchart showing the processing flow from data acquisition to classification result display performed by the server 10. After the capturing device 80 acquires data and sends it to the server 10, processing starts from step 400. In step 401, the receiving unit 50 receives and acquires the data sent from the capturing device 80. The data may be image data or audio data, and is converted to text using OCR or speech recognition.

[0096] In step 402, the association unit 51 assigns identification information to the acquired data. In step 403, the discrimination unit 52 determines the type of data from the text data contained in the data. The type of data is determined by reading the content of the text contained in the data using AI. The association unit 51 uses the information about the type determined by the discrimination unit 52 as a tag and tags the data. The tags include "inquiry," "food safety," and "manufacturing and distribution."

[0097] In step 404, the extraction unit 53 extracts specific information from the text data contained in the data. The specific information is the content of the identifiable portion of the text data shown in Figure 18. This content has been trained through machine learning to determine which words and phrases should be extracted, so it is possible to extract such words and phrases.

[0098] In step 405, the discrimination unit 52 determines the category of the data from the extracted content. The discrimination unit 52 uses tags related to the category it has determined to tag the data. For data classified as type "inquiry," the tags are, for example, "praise / encouragement," "request / suggestion," "inquiry / consultation," or "criticism / complaint." For data classified as type "food safety," the tags are, for example, "digestive symptoms," "headache / fever," "rash / skin symptoms," "constipation," "oral / throat symptoms," "ambiguous symptoms," "symptoms in other parts of the body," or "no health damage." For data classified as type "manufacturing / distribution," the tags are, for example, "foreign object contamination," "odor / alteration of taste," "damaged container / deformed packaging," "defective expiration date / allergy labeling," or "other."

[0099] In step 406, the execution unit 54 performs one or more processes using one or more software programs, depending on the type of data. These processes include classifying the data and automatically determining its accuracy and importance. The data is classified based on identified tags. The data is analyzed using AI to determine the accuracy and importance of the information. The processed data will have the data content, analysis date, accuracy, and importance associated with it.

[0100] In step 407, the association unit 51 assigns identification information to the processed data executed by the execution unit 54. The identification information is a number such as 97. This makes it possible to associate the original data with the processed data. The association unit 51 can also assign information about the software that performed the processing to this data. The information about the software may be, for example, the software name.

[0101] In step 408, the storage unit 55 stores the processed data associated with the original data. By storing the processed data in association with the original data in this way, the processed data can be retrieved at the time the user needs it. In step 409, the control unit 56 receives a request from the operating device 11, retrieves the processed data from the storage unit 55 based on its type, and lists it. The transmission unit 57 transmits the classification results of the listed data to the operating device 11. The operating device 11 receives and displays the classification results of the listed data. The classification results include the analysis date, accuracy, importance, image data or audio data, etc. Note that the operating device 11 may perform some of the processing performed by the control unit of the server 10. For example, the operating device 11 may generate the analysis date, category, accuracy, importance, and at least one of the image data and audio data, generate screen data using all or part of the received data, and display the classification results of the listed data based on the generated screen data. Then, the process proceeds to step 410, and the processing up to the display of the classification results is completed. Image data is displayed as address information and thumbnails, while audio data is displayed as audio markers, etc.

[0102] As described above, according to the present invention, when data is acquired, the associated software is executed, the processed data is stored, and the processed data can be displayed by selecting the data at any time. Furthermore, by selecting the software, processing such as displaying, analyzing, and interpreting the registered data can be automatically performed. This makes it possible to display processed data without specifying and executing processing on the data, and it can also be incorporated into additional workflows, simplifying the processing that users perform on the data.

[0103] Although one embodiment of the present invention has been described so far, the present invention is not limited to the embodiments described above. The components of this embodiment can be changed or deleted, or other components can be added to the components of this embodiment, to the extent that a person skilled in the art can conceive of such modifications. Any embodiment that achieves the effects of the present invention is included within the scope of the present invention. [Explanation of Symbols]

[0104] 10… Server 11…Operating device 12…Network 13…MFP 14…Printer 15…Webcam 16…360-degree camera 17… Microphone 18…Hearable devices 19…IWB 20…Main Software 21-24…Software 30…CPU 31…ROM 32...RAM 33…HD 34…HDD controller 35…Display 36…External device connection interface 37…Network I / F 38...Data bus 39... Keyboard 40…Pointing device 41…DVD-RW drive 42…Media I / F 43…DVD-RW 44…Recording media 50... Receiver 51…Relationship section 52...Discrimination section 53...Extraction part 54…Executive Department 55...Storage section 56... Control Unit 57...Transmitter 60... Checkbox 61, 62... Thumbnails 63, 64… Icons 65... button 66... ​​Input / Edit button 67-71...Execute button 80…Capture device Columns 90, 91… 92…Contents 93…Analysis date 94...Category 95... Accuracy 96…Importance 97…Audio mark 98... number [Prior art documents] [Patent Documents]

[0105] [Patent Document 1] Japanese Patent Publication No. 2002-55985

Claims

1. A data processing device that processes data, A means of acquiring data, A determination means for determining the type of data based on the text information contained in the acquired data or the text information converted from said data, An execution means that performs one or more processes on the data according to the type that has been identified, A control means that controls the output of one or more processed data as processing results, which have been processed by the execution means. Includes, The determination means further determines the category in the type from the content of the text information, The execution means performs a process to analyze the importance of the content of the data based on the type and category determined by the determination means and the text information. The control means controls the data processing device to output the analysis results regarding importance in association with the data as a processing result.

2. Includes an extraction means for extracting the content of a specified item from the data based on the text information, The data processing device according to claim 1, wherein the determination means determines the category from the content of the item extracted by the extraction means as the content of the text information.

3. Includes a storage means for storing the acquired data and the one or more processed data in association with each other, The data processing apparatus according to claim 1 or 2, wherein the control means instructs an operating device operated by a user to output a list of the data stored in the storage means.

4. A data processing apparatus according to any one of claims 1 to 3, comprising a conversion means for converting the acquired data into text data.

5. The data processing apparatus according to claim 4, wherein the conversion means converts the data into text data by speech recognition when the data is audio data, and converts the data into text data by character recognition when the data is image data.

6. The data processing device according to any one of claims 1 to 5, wherein the execution means classifies the data based on the type and category determined by the determination means, performs a process to analyze the accuracy of the data classification result, and outputs the classification result, the accuracy and the importance as processing results.

7. The data processing apparatus according to claim 6, wherein, if the data is image data or audio data, the output processing result includes the image data or audio data.

8. The acquisition means acquires user identification information for identifying a user, The data processing apparatus according to any one of claims 1 to 5, wherein the execution means performs one or more processes on the data according to the determined type and the acquired user identification information.

9. The data processing apparatus according to claim 8, which, as an instruction from the user, receives the selection of one data from the user and controls the output of one or more processed data associated with the selected data.

10. The data processing apparatus according to claim 9, wherein the control means controls the output of a list of processed data, a list of other processes, or both, depending on the type of processing set for the selected data type.

11. The data processing apparatus according to claim 10, wherein the control means receives a user instruction to select one processed data from the list of processed data, and controls the operating device operated by the user to output the selected processed data.

12. The data processing device according to claim 10, wherein the control means accepts the selection of one process from the list of other processes as an instruction from the user, and controls the operating device operated by the user to output data to be used for the selected process.

13. The data processing apparatus according to claim 11 or 12, wherein the operating device displays the classification result based on specific information contained in the text information as category information for the data type.

14. The data processing apparatus according to claim 13, wherein the operating device displays the contents of data including the specific information.

15. The data processing apparatus according to claim 14, wherein the operating device displays specific information contained in the text information in an identifiable manner.

16. The data processing apparatus according to claim 14 or 15, wherein the operating device displays at least one of information indicating the accuracy of the category information and information indicating the importance of the content of the data.

17. The data processing apparatus according to any one of claims 14 to 16, wherein the operating device displays together the content of the data and information indicating the storage location of the data acquired by the acquisition means.

18. A data processing system comprising a data processing device and an operating device whose output is controlled by the data processing device, The data processing device, A means of acquiring data, A determination means for determining the type of data based on the text information contained in the acquired data or the text information converted from said data, An execution means that performs one or more processes on the data according to the type that has been identified, A control means that controls the output of one or more processed data as processing results, which have been processed by the execution means. Includes, The determination means further determines the category of the data type from the content of the text information, Based on the type and category determined by the determination means and the text information, a process is performed to analyze the importance of the content of the data. The control means is a data processing system that controls the output of the analysis results regarding importance in association with the data as a processing result.

19. The data processing system according to claim 18, wherein the operating device displays the classification result based on specific information contained in the text information as category information for the data type.

20. The data processing system according to claim 19, wherein the operating device displays the contents of data including the specific information.

21. The data processing system according to claim 20, wherein the operating device displays specific information contained in the text information in an identifiable manner.

22. The data processing system according to claim 20 or 21, wherein the operating device displays at least one of information indicating the accuracy of the category information and information indicating the importance of the content of the data.

23. The data processing system according to any one of claims 20 to 22, wherein the operating device displays together the content of the data and information indicating the storage location of the data acquired by the acquisition means.

24. A method for processing data using a data processing device, Steps to acquire data, A step of determining the type of data based on the text information contained in the acquired data or the text information converted from said data, The steps include: performing one or more processes on the data according to the type identified; A step of controlling the system so that one or more processed data items are output as one or more processing results. Includes, In the aforementioned determination step, the category of the data type is further determined from the content of the text information, In the steps described above, a process is performed to analyze the importance of the data content based on the determined type and category and the text information. A data processing method in which, in the control step, the processing result is controlled to output the analysis result regarding importance in association with the data.

25. A program for causing a computer to perform each step included in the method according to claim 24.

Images