How to assist with data entry in forms

Abstract

To provide a method for supporting data input for a form for efficiently reporting on performance at a manufacturing site without imposing a burden on a person in charge of reporting performance at the site. [Solution] A method in system 1 that supports data entry of forms via user terminals 200A, 200B operated by field staff users, in which a control unit of a server terminal 100 references form template data stored in a memory unit, detects item boxes, item names, and input boxes in the form template data, determines the correspondence between the detected item boxes and input boxes, and converts the data into electronic form template data that can be entered by hand.

Description

[Technical field]

[0001] The present invention relates to a method for supporting data entry into reports for performance reports, etc., primarily at manufacturing sites. [Background technology]

[0002] Conventionally, at manufacturing and other workplaces, on-site staff would hand-enter necessary information onto paper forms, and another staff member would then manually enter the information written on the collected paper forms into a digitally formatted form.

[0003] In recent years, there has been an increasing trend to digitize reports on worksite performance. For example, Patent Document 1 discloses that a worksite worker can input questions via an electronic chat screen displayed on a terminal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6396561 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology disclosed in Patent Document 1 imposes a burden of input work and implementation on field workers. In particular, field workers who are accustomed to writing by hand on existing paper will face a large burden and resistance to "keyboard input," "typing input," and "flick input," which are required as input methods on tablet terminals and smartphones.

[0006] Therefore, an object of the present invention is to provide a data entry support method that enables efficient reporting of on-site performance without imposing a burden on on-site personnel who report on performance at the manufacturing site. [Means for solving the problem]

[0007] In one aspect of the present invention, there is provided a method for assisting field staff in entering data into forms, in which a control unit of a server terminal references form template data stored in a memory unit, detects item boxes, item names, and input boxes in the form template data, determines correspondences between the detected item boxes and input boxes, and converts the data into electronic form template data that can be entered by handwriting. Effect of the Invention

[0008] According to the present invention, it is possible to provide a data entry support method that enables efficient reporting of on-site performance without imposing a burden on the on-site personnel who report performance at the manufacturing site. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a data entry support system according to a first embodiment of the present invention; [Diagram 2] 2 is a functional block diagram showing the server terminal 100 of FIG. 1. [Diagram 3] 2 is a functional block diagram showing a user terminal 200 of FIG. 1. [Figure 4] FIG. 2 is a diagram showing an example of user data stored in the server 100. [Diagram 5] FIG. 2 is a diagram showing an example of form data stored in the server 100. [Figure 6] 4 is a flowchart showing an example of a process of a data entry assistance method according to the first embodiment of the present invention. [Figure 7] 6 is a flowchart showing another example of the process of the data entry support method according to the first embodiment of the present invention. [Figure 8] 5 is a diagram illustrating an example of form template data according to the first embodiment of the present invention. FIG. [Figure 9] 4 is a diagram illustrating an example of a user interface screen of an electronic form template displayed on a user terminal according to the first embodiment of the present invention. FIG. [Figure 10]13 is a diagram illustrating another example of a user interface screen of an electronic form template displayed on a user terminal according to the first embodiment of the present invention. FIG. [Figure 11] 13 is a diagram illustrating another example of a user interface screen of an electronic form template displayed on a user terminal according to the first embodiment of the present invention. FIG. [Figure 12] FIG. 2 is a diagram illustrating a method for generating structured data from handwritten input data according to the first embodiment of the present invention. [Figure 13] FIG. 4 is another diagram illustrating the method for generating structured data from handwritten input data according to the first embodiment of the present invention. [Figure 14] FIG. 2 is a diagram illustrating a method for visualizing structured data according to the first embodiment of the present invention. [Figure 15] 3A to 3C are diagrams illustrating a method for character recognition of handwritten input data according to the first embodiment of the present invention. [Figure 16] 10 is a flowchart showing yet another example of the process of the data entry support method according to the first embodiment of the present invention. [Figure 17] FIG. 2 is a diagram illustrating one process for creating an electronic form template according to the first embodiment of the present invention. [Figure 18] FIG. 11 is a diagram illustrating another process for creating an electronic form template according to the first embodiment of the present invention. [Figure 19] FIG. 11 is a diagram illustrating yet another process for creating an electronic form template according to the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment described below does not unduly limit the contents of the present invention described in the claims. Furthermore, not all of the components shown in the embodiment are necessarily essential components of the present invention.

[0011] <Configuration> 1 is a block diagram showing a data input support system according to a first embodiment of the present invention. The system 1 includes a server terminal 100 that generates electronic form template data for handwritten input from pre-stored form template data, receives handwritten input data by a field staff user at a factory or the like via a user terminal, generates structured data based on the handwritten input data using a method such as character recognition, and stores the structured data as final data, and user terminals 200A and 200B operated by the field staff users. For convenience of explanation, each terminal is described as a single terminal or a specific number of terminals, but the number of each terminal is not limited.

[0012] The server terminal 100 and the user terminals 200A and 200B are connected to each other via a network NW1. The network NW is configured by the Internet, an intranet, a wireless LAN (Local Area Network), a WAN (Wide Area Network), or the like.

[0013] The server terminal 100 may be, for example, a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] The user terminals 200A and 200B are, for example, information processing devices such as tablet terminals, but may also be configured as smartphones, mobile phones, PDAs, personal computers, or the like.

[0015] In this embodiment, the system 1 is described as having a configuration including a server terminal 100 and user terminals 200A, 200B, and users of each terminal use their respective terminals to operate the server terminal 100, but the server terminal 100 may be configured as a stand-alone system, and the server terminal itself may have a function for each user to directly operate it. Hereinafter, the user terminals 200A, 200B will be collectively referred to as user terminal 200, as necessary.

[0016] Fig. 2 is a functional block configuration diagram of the server terminal 100 of Fig. 1. The server terminal 100 includes a communication unit 110, a storage unit 120, and a control unit .

[0017] The communication unit 110 is a communication interface for communicating with the user terminal 200 via the network NW1, and communication is performed according to a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol).

[0018] The storage unit 120 stores programs for executing various control processes and various functions in the control unit 130, input data, etc., and is composed of a RAM (Random Access Memory), a ROM (Read Only Memory), etc. The storage unit 120 also has a user data storage unit 121 that stores various data related to users who are non-desk workers in companies engaged in, for example, manufacturing, building maintenance, construction, food and beverage, field sales, etc. (hereinafter, manufacturing, etc.), and a form data storage unit 122 that stores various data related to forms such as work records at factory sites. A database (not shown) storing various data may be constructed outside the storage unit 120 or the server terminal 100.

[0019] The control unit 130 controls the overall operation of the server terminal 100 by executing a program stored in the storage unit 120, and is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. Functions of the control unit 130 include an information receiving unit 131 that receives information such as handwritten input data from a user terminal of a field worker user at a factory, etc., a character recognition processing unit 132 that executes character recognition processing on the handwritten input data, and a form data processing unit 133 that stores form template data, electronic form template data for handwritten input, structured data, etc. The information receiving unit 131, the character recognition processing unit 132, and the form data processing unit 133 are started by a program stored in the storage unit 120 and executed by the server terminal 100, which is a computer (electronic calculator).

[0020] The information receiving unit 131 receives information from the user terminal 200 via the communication unit 110. For example, data handwritten on an electronic form template is received from the user terminal 200 as coordinate data and handwriting data.

[0021] The character recognition processing unit 132 executes character recognition processing on the handwritten input data received from the user terminal 200, and generates digital data.

[0022] The form data processing unit 133 performs predetermined processing, such as generating structured data based on the digital data generated as a result of character recognition and storing the generated structured data.

[0023] The control unit 130 may also have a screen generation unit (not shown), which generates screen information to be displayed via the user interface of the user terminal 200 upon request. For example, using image and text data (not shown) stored in the storage unit 120 as material, the control unit 130 generates a user interface by arranging various images and text in predetermined areas of the user interface based on predetermined layout rules. Processing related to the image generation unit may also be executed by a GPU (Graphics Processing Unit).

[0024] Fig. 3 is a functional block diagram showing the user terminal 200 of Fig. 1. The user terminal 200 includes a communication unit 210, a display operation unit 220, a storage unit 230, and a control unit 240.

[0025] The communication unit 210 is a communication interface for communicating with the server terminal 100 via the network NW, and communication is performed according to a communication protocol such as TCP / IP.

[0026] The display operation unit 220 is a user interface used for inputting instructions by the user and displaying text, images, etc. according to input data from the control unit 240, and is composed of a display, a keyboard, and a mouse when the user terminal 200 is configured as a personal computer, and is composed of a touch panel, etc. when the user terminal 200 is configured as a smartphone or tablet terminal. This display operation unit 220 is started up by a control program stored in the storage unit 230 and executed by the user terminal 200, which is a computer (electronic calculator).

[0027] The storage unit 230 stores programs for executing various control processes and functions in the control unit 240, input data, etc., and is composed of a RAM, a ROM, etc. The storage unit 230 also temporarily stores the contents of communication with the server terminal 100.

[0028] The control unit 240 controls the overall operation of the user terminal 200 by executing the programs stored in the storage unit 230, and is composed of a CPU, a GPU, and the like.

[0029] FIG. 4 is a diagram showing an example of user data stored in the server 100. As shown in FIG.

[0030] The user data 1000 shown in Fig. 4 stores various data related to a user. For convenience of explanation, Fig. 4 shows an example of one user (a user identified by a user ID "10001"), but information on multiple users can be stored. The various data related to a user can include, but is not limited to, basic information about the user (e.g., company name, address, name of person in charge, department name, contact information, etc.) and form data such as work records at a factory site (ID for identifying a form stored as form data).

[0031] FIG. 5 is a diagram showing an example of form data stored in the server 100. As shown in FIG.

[0032] The form data 2000 stores various data related to forms such as work records at a factory site. In FIG. 5, for convenience of explanation, an example of one form (a form identified by a form ID "20001") is shown, but information related to multiple forms can be stored. The form data can include, but is not limited to, form template data, which is template data that defines input items and input formats when a corporate user engaged in manufacturing transcribing data handwritten into a paper form such as a work record at a factory site into a file in a predetermined format such as Excel or CSV and recording it as structured data, electronic form template data that is generated based on the form template data, displayed on a user terminal used by a user in charge of the site, and converted into a format in which data such as work records can be input by handwriting, and structured data (text, numerical values, symbols, identifiers, time stamps, etc.) generated through character recognition based on data handwritten by a user in charge of the site into the electronic form template displayed on the user terminal, but is not limited to these. In addition, as document data, the user's handwriting data and coordinate data that are recognized as the handwritten input data necessary to perform character recognition during the character recognition process, and / or a learning model generated by learning the input items, input areas, and their coordinate information contained in the electronic document template data, as well as the text data and structured data as the character recognition results, can be stored.

[0033] 6 is a flowchart showing an example of the process of the data input support method according to the first embodiment of the present invention. A person in charge of a company operating in the manufacturing industry or the like refers to data handwritten on a paper form such as a work record at a factory site, transcribes the data into a file in a predetermined format such as Excel or CSV, in which data input items and input formats are predefined, and records the data as structured data. The person in charge transmits (uploads) the form template data in the predetermined format to the server terminal 100 via the user terminal 200. When the information receiving unit 131 of the control unit 130 of the server terminal 100 receives the form template data, the template data processing unit 133 of the control unit 130 stores the form template data in the form data storage unit 122 of the storage unit 120 in association with the user data.

[0034] As the processing of step S101, the template data processing unit 133 of the control unit 130 of the server terminal 100 refers to the report data storage unit 122 and performs a reading process of the report template data stored as report data 2000 based on a request from the user terminal 200 of the company's person in charge user.

[0035] 8, a corporate user has form template data in a predetermined format such as Excel, and while referring to handwritten data entered on a paper form, transcribes and inputs data into form template data 800. The corporate user requests the loading of a file uploaded in advance via a user interface screen displayed on the user terminal 200, and issues an instruction to load the form template data.

[0036] Next, in the processing of step S102, the document data processing unit 133 of the control unit 130 of the server terminal 100 refers to the document data storage unit 122 of the memory unit 120, reads the document template data for which a read request has been made, and executes a process of converting the document template data into electronic document template data that can be input by handwriting.

[0037] For example, as shown in Fig. 9, electronic document template data 900 is characterized by being template data that has input items and input areas common to the document template data 800 shown in Fig. 8 and allows handwritten input. The data format of the electronic document template data 900 can be the same format as the document template data 800 (in this example, Excel, CSV, etc.), or can be any format (PDF, etc.). In this way, by making the electronic document template data template data allowing handwritten input, on-site personnel at a factory or the like can input data while working, and compared to digital data input, data can be input intuitively as if writing on a paper document, so there is no barrier to introducing the tool and the introduction cost can be saved.

[0038] 7 is a flowchart showing another example of the process of the data input support method according to the first embodiment of the present invention. In this example, a user in charge of a company operating in the manufacturing industry or the like installs a form input support application in a user terminal 200 such as a tablet, starts the application, and inputs a record of processing work as a manufacturing journal via the electronic form template shown in FIG. 6 and displayed on the user terminal 200.

[0039] In step S201, the person in charge inputs data by handwriting using a stylus pen or the like into the electronic form template displayed on the user terminal 200, and the information receiving unit 131 of the control unit 130 of the server terminal 100 receives the handwritten input data from the user terminal 200. Here, the handwritten input data includes the user's handwriting data and coordinate data, and / or input items, input areas, and their coordinate information included in the electronic form template data. For example, as shown in FIG. 9, the user can input by handwriting into predetermined input items of the electronic form template 900, such as the processing time and the number of defective products. Here, the person in charge can also transmit media data, such as images (including videos) captured by the user terminal 200 or the like at the site and / or recorded voice, to the server terminal 100 along with the handwritten character input.

[0040] In the process of step S202, the character recognition processing unit 132 of the control unit 130 of the server terminal 100 generates digital text data based on the handwritten input data. The character recognition processing unit 132 recognizes the input characters by a method such as pattern matching based on the handwriting data and coordinate data as the handwritten input data, and performs a process of inferring which characters have been input into which input field and / or input area based on the input coordinates in the electronic document template displayed on the user terminal 200. In addition, when recognizing characters, a learning model generated by performing machine learning based on the input handwriting data, coordinate data, and the output text data and structured data can also be used. The process performed by the character recognition processing unit 132 can also be executed by a program installed in the user terminal 200.

[0041] Here, when generating digital text data based on handwritten input data, the character recognition processing unit 132 can also generate digital text data by executing character recognition processing based on OCR processing on image data generated based on an electronic form template including the handwritten input data. Then, the character recognition processing unit 132 outputs the generated text data to the input box of the electronic form template.

[0042] Here, in the character recognition process, if the data handwritten by the user is different from data previously entered by the user or is different from a specified range, the character recognition processing unit 132 can notify the user terminal 200 of the inputter or administrator of an error in real time or present predictive conversion candidates. Alternatively, when there is a possibility of an input error as described above, the character recognition processing unit 132 can convert based on the handwritten input data and distinguishably display the area in which the output text data is displayed by color coding.

[0043] In addition, in character recognition processing, when handwritten input is made outside the input area (frame) on the electronic document template, the character recognition processing unit 132 recognizes the coordinates on the electronic document template of the start point where the input was made and the end point where the first character input is completed, determines that the input was made in the input area (frame) that includes these coordinates, and outputs characters (numbers) converted from the handwritten input data so that they fit into the determined input frame.

[0044] In addition, in the character recognition process, characters handwritten by a user may be converted into similar characters (for example, the number "2" and the English letter "z", or the number "0" and the English letter "o"). In such a case, as shown in FIG. 15, the input data can be predicted based on data previously input in the same form type and in the same input area (frame) and converted into text. For example, when "123.4" is handwritten, in order to prevent the handwritten input data from being recognized as "1z314", the character recognition processing unit 132 can predict that the accepted handwritten input data is a number including a decimal point based on the structured data previously input into the same input area of ​​the same form and stored in the form data storage unit 122 of the storage unit 120. In addition, the input data can be predicted based on data previously registered by the user together with or instead of the past input data. Here, the user can pre-register input rules for a given entry box, such as that only numbers and "." can be entered in the entry box, that a choice of characters can be specified for the entry box (for example, allowing the selection of part name A, part name B, or part name C), that only the date can be entered in the entry box, or that only the worker's name or a unique ID that manages the worker can be entered.

[0045] In addition, after the character recognition process, the responsible user can check the displayed text data, such as characters and numbers, that is output to the input box of the electronic document template, and manually edit the text data for each input box, thereby correcting, adding, or deleting the text data.

[0046] Next, in step S203, the character recognition processing unit 132 performs a process of storing the recognized characters as structured data in the form data storage unit 122 of the storage unit 120. As an example of the process of recognizing as structured data, in an electronic form template 1200 displayed on the user terminal 200 shown in FIG. 12, the following areas are recognized in the template: 1) an area 1210 in which the user cannot write or select and in which information indicating the type of form is displayed (in this example, the form name "Processing Quality Check Sheet" linked to the form ID, etc.); 2) an area 1220 in which the user can write or select and in which the input information affects the entire information included in the form (in this example, "Part Number BK The input information is categorized into areas 1230 (in this example, "Laser removal," "Start and end checks," "Work date," "Worker," etc.) related to the information contained in the rows and columns of the form, and the information input for each area is recognized. When the above 2) and 3) are filled in or selected, the input data (characters, numbers, symbols, etc.) recognized in each area is converted into structured data as shown in FIG. 13 together with the timestamps based on the timestamp assigned to the time of input, the timestamp assigned to the time of application for approval of the input contents, and the timestamp assigned to the time of approval.

[0047] 13, the character recognition processing unit 132 displays the input data recognized in 1) above in an area 1310 for outputting the form ID, the input data recognized in 2) above in an area 1320 for displaying the records related to the entire form, and the input data recognized in 3) above in an area 1330 for displaying information identifying rows and columns (identifiers defined as column "A", row "1", column "B", row "3", column "A", etc.) and an area 1340 for displaying information entered in the identified rows and columns, together with timestamps indicating the time of entry, application, and approval. This allows the handwritten input data to be converted into structured data for recording and management, and also allows the input data to be recorded and managed together with the name of the person who entered the data and a timestamp, ensuring the traceability of the recorded data. As described above, each area included in the electronic document template is assigned an identifier. For example, when the area in which the input data in 2) above is displayed is designated as "1" (the area in which "BK" is displayed), and the area in which the input data in 3) above is displayed is designated as "A1" (check at the start of laser removal), "B1" (check at the end of laser removal), "A3" (check at the start of nut press-in), etc. according to row and column, by using the document ID, identifier, and time stamp, as shown in FIG. 14, the vertical axis is the entry content (numerical value, etc.), the horizontal axis is the entry date, and the identifiers ("check at the start of laser removal", "check at the end of laser removal") by each matrix can be visualized as a legend. In addition, based on the information such as the above "document ID", "identifier", and "time stamp", a user in charge of a company or an administrator user can perform a search to obtain desired information.

[0048] As shown in FIG. 10, when the handwritten input data is output as structured data to the electronic form template, the field staff user can make a data recording request to the server terminal 100 via the user terminal 200. The form data processing unit 133 of the control unit 130 of the server terminal 100 stores the structured data in the form data storage unit 122 of the memory unit 120. At the same time, the form data processing unit 133 can also give a timestamp to the time when the structured data is recorded and store it in the form data storage unit 122 together with the filler information. In addition, as shown in FIG. 11, following the recording of the structured data, the field staff user can also submit an approval application for the input contents to the manager user who is the approver. At the same time, the form data processing unit 133 can also give a timestamp to the time when the structured data is applied for and store it in the form data storage unit 122 together with the applicant information. When the administrator user receives an approval request for input data via the server terminal 100 and performs the approval process on the administrator user's user terminal 200 (for example, user terminal 200B different from the user terminal 200A of the field staff user), the server terminal 100 receives approval information from the user terminal 200B, assigns an approved status to the structured data, and simultaneously assigns a timestamp, stores this information in the report data storage unit 122 of the memory unit 120, and sends a notification to the field staff user's user terminal 200A that the data input has been approved.

[0049] As described above, when the field staff user records the handwritten input data as structured data, the structured data is stored and accumulated in the form data storage unit 122 of the memory unit 120 of the server terminal 100. The server terminal 100 can generate a response to a question request by a staff user by using the structured data thus accumulated as a data set and linking with a large-scale language model (e.g., chat GPT). For example, when the staff user inputs a question such as "For product XD-112, summarize the top three performance data with the highest number of defects and consider the common cause" via a predetermined chat user interface displayed on the user terminal 200, the server terminal 100 can refer to the data set stored in the form data storage unit 122 and generate a response such as "The top three are as follows. (Present the performance of the top three) The common cause is that worker A is working on it" based on the large-scale language model, and transmit the response to the user terminal 200. In addition, in response to a question such as "Please analyze the inventory change of part A in the inventory management form over the past 10 days and suggest the next ordering period," a response such as "200 pieces of part A have been used in 10 days. If this change continues, the next order will be required in 40 days" can be generated. Here, the form data that is the source of the data set can be managed for each company in the form data storage unit 122, and the accuracy of the response to the question can be improved while ensuring security.

[0050] 11 is a flowchart showing yet another example of the processing of the data input support method according to the first embodiment of the present invention. In this example, in the processing of converting form template data into electronic form template data described in FIG. 6, the object is to improve the detection accuracy of input items (item names) and input areas (entry frames) and further improve the input accuracy of the user.

[0051] Similarly to the above, a company's person in charge user transmits (uploads) form template data in a predetermined format such as Excel or CSV (or image data such as JPG) to the server terminal 100 via the user terminal 200. When the information receiving unit 131 of the control unit 130 of the server terminal 100 receives the form template data, the template data processing unit 133 of the control unit 130 stores the form template data in the form data storage unit 122 of the memory unit 120 in association with the user data.

[0052] In the processing of step S301, the template data processing unit 133 of the control unit 130 of the server terminal 100, based on a request from the user terminal 200, refers to the form data storage unit 122, performs a reading process of the form template data stored as the form data 2000, and detects an area to be used as an entry frame in the form template data by image recognition.

[0053] 17, for example, the template data processing unit 133 detects entry boxes and areas in the form template data by image recognition processing. Here, the information receiving unit 131 of the control unit 130 of the server terminal 100 can also determine the entry box by receiving an input from the user to adjust the size of a rectangular frame that will become the entry box in the form template data displayed on the user terminal 200. This can also improve the accuracy of the entry box in the template data.

[0054] Next, in step S302, the character recognition processing unit 132 of the control unit 130 of the server terminal 100 detects item frames and item names in the form template data by character recognition based on OCR processing. For example, in FIG. 17, item frames and item names such as "Product name", "Contact person", and "Processing content" are included.

[0055] Next, as the process of step S303, the document data processing unit 133 of the control unit 130 of the server terminal 100 executes a matching process between the detected item box and item name in the document template data and the entry box. This process is a process for determining the correspondence relationship of which input box corresponds to an item box containing an item name, and can store the correspondence relationship between the item box and the input box as a plurality of types (patterns) based on the positional relationship between the item box and the input box, or by referring to electronic document template data generated by a past user or another user, and can infer the correspondence relationship from the type. For example, as shown in FIG. 18, the document data processing unit 133 determines the entry boxes corresponding to each item of the "defect content" item, such as "short", "foreign matter", and "burr", the entry box corresponding to "abnormal content (immediate report)", and the entry box corresponding to "instructor", "enterer", etc., based on the positional relationship between each item box and the entry box. Here, when the document data processing unit 133 determines the correspondence relationship between the item box and the entry box, it can determine the optimal input method for the item. For example, when it is recognized that the item box is related to a date, "Entry Date", the corresponding input box can be set to allow the user to input a calendar. Also, when it is recognized that the item box is "Entry by", the corresponding input box can be set to allow the user name of the logged-in user to be input. These processes can be estimated by the form data processing unit 132 by referring to input information for the form template data and / or electronic form template data by past users or other users using machine learning or the like. Also, as shown in FIG. 19, the form data processing unit 133 can recognize the characters and size of the input data (text data, etc.) for the input box, define a maximum rectangle and a margin, and determine whether the input box is an entry box or an item box based on the definition. In this way, the form data processing unit 133 determines the correspondence between the item box and the input box, and stores it in the form data storage unit 122 of the storage unit 120 as electronic form template data.On the other hand, for forms whose input method is difficult to guess from item names such as "product number", "product name", "operation content", and "molding lot", input methods corresponding to each item frame (item name) are stored in advance in the form data storage unit 122 of the storage unit 120, and the input method can be guessed based on a rule base, or based on data digitized by OCR processing or the like from the contents of paper forms entered by actual workers, guessed by a learning model generated by machine learning, or guessed in combination with machine learning based on the above item frame (item name) c. This makes it possible to guess by learning based on the contents actually entered on paper forms, such as a method of connecting a number of digits with a hyphen for "product number", a method of entering "product name" in katakana for "operation content", a selection method for "operation content", and a date method for "molding lot".

[0056] Furthermore, when the document data processing unit 133 receives handwritten input from the user to an input box of electronic document template data via the user terminal 200, the document data processing unit 133 detects the trajectory of the user's input relative to the input box based on the input position of several strokes input by the user, and can estimate which input box the user is inputting data into based on the ratio of the trajectory to the input box. This makes it possible to detect which input box the user is inputting data into even if the user inputs data that protrudes from the input box. In addition, the document data processing unit 133 refers to past user input information based on text information recognized by character recognition by the character recognition processing unit 132 for the user's handwritten input, and displays candidate text information within or adjacent to the input box of the electronic document template data displayed on the user terminal 200, thereby improving the accuracy of data input by the user.

[0057] Furthermore, when the document data processing unit 133 receives handwritten input from the user to an input box of electronic document template data via the user terminal 200, the document data processing unit 133 detects the trajectory of the user's input relative to the input box based on the input position of several strokes input by the user, and can estimate which input box the user is inputting data into based on the ratio of the trajectory to the input box. This makes it possible to detect which input box the user is inputting data into even if the user inputs data that protrudes from the input box. In addition, the document data processing unit 133 refers to past user input information based on text information recognized by character recognition by the character recognition processing unit 132 for the user's handwritten input, and displays candidate text information within or adjacent to the input box of the electronic document template data displayed on the user terminal 200, thereby improving the accuracy of data input by the user.

[0058] In this way, when the worker user completes data input to the electronic document template data, a report such as a work log is created, and an input completion instruction from the user is received, the document data processing unit 133 stores the data input to the electronic document template data as report data in the document data storage unit 122 of the storage unit 120 as structured data. The reporter data can be confirmed by a user such as an administrator on a management screen or the like displayed on the user terminal 200, and can also be output as PDF data. The administrator user can edit the report data displayed on the user terminal 200 and can finally perform approval processing. The report data edited and / or approved by the administrator user is updated as needed, and the worker user or the administrator user can confirm the final report data in real time on each user terminal 200.

[0059] As described above, according to this embodiment, by inputting data into an electronic form template by handwriting, it is possible to realize a method for supporting data input that enables efficient reporting of on-site performance by structuring and recording the input data without imposing a burden on the on-site personnel who report performance at the manufacturing site. In addition, it is possible to improve the accuracy of generating an electronic form template from the original image data or form template data, and also to improve the accuracy of input by the user into the electronic form template.

[0060] Although the embodiments of the present invention have been described above, they can be embodied in various other forms and can be implemented with various omissions, substitutions, and modifications. These embodiments and modifications, as well as omissions, substitutions, and modifications, are included in the technical scope of the claims and their equivalents. [Explanation of symbols]

[0061] 1 System 100 Server terminal, 110 Communication unit, 120 Storage unit, 130 Control unit, 200 User terminal, NW1 Network

Claims

1. A method for supporting a field staff member in inputting data into a form, comprising the steps of: The control unit of the server terminal storing, in a storage unit of the server terminal, a correspondence relationship between an item box including an item name in the first form template data and an input box corresponding thereto, as a plurality of types, based on the past first form template data of the site staff or another user; receiving second form template data from the user terminal of the on-site staff; Detecting item frames, item names and input frames in the second form template data, determining a correspondence between the detected item frames and item names and input frames based on the plurality of types; converting the second form template data into electronic form template data that can be handwritten; A method according to claim 1, wherein the first form template data and the second form template data are form template data in a predetermined format for transcribing input data of a paper form that has been handwritten in advance.

2. 2. The method of claim 1 , The method includes transmitting the electronic form template data to the user terminal of the field staff.

3. 3. The method of claim 2, The control unit Accepting handwritten input data from the user terminal; generating structured data based on the input data; The method further comprises storing the structured data in the storage unit.

4. 4. The method of claim 3, The control unit is A method for converting the input data into structured data based on which input field in the electronic form template data the input data was entered into.

5. 5. The method of claim 4, The control unit is A method in which the input areas are composed of areas that cannot be filled in or selected, areas that can be filled in or selected and affect the entire electronic form template data, and areas that can be filled in or selected and affect rows and columns in the electronic form template data.

Images