A paper form handwriting automatic input and transmission system and method

Abstract

The present invention discloses a system and method for automatically inputting and transmitting handwriting on paper forms, comprising an electromagnetic pen, an electromagnetic induction board, a transmission module, and a mobile phone app. The electromagnetic pen writes text information on the electromagnetic induction board, the electromagnetic induction board collects the coordinates and pressure data of the electromagnetic pen, the output end of the electromagnetic induction board is connected to the input end of the transmission module, the transmission module is wirelessly connected to the mobile phone, and the mobile phone app is equipped with a calculation and recognition unit and is used to store handwriting information. The method of the present invention is that the handwriting information collected by the electromagnetic induction board is transmitted to the mobile phone app in real time through the transmission module for storage, and then the mobile phone app transmits the recognized electronic form information to the app through the calculation and recognition unit. The calculation and recognition unit can run in the mobile phone app or a remote server. The system automatically records the form type and handwriting information for processing, and directly enters the written information into the electronic form, greatly saving the time and energy of manual transcription.

Description

Technical Field

[0001] The present invention relates to the field of tabulation technology, and in particular to a system and method for automatically inputting and transmitting handwriting on a paper form. Background Art

[0002] In various industries, there are often various paper forms that need to be filled out. Even though information systems and mobile software are now very popular, many paper reports have been replaced by information software, but there are still a large number of paper forms that need to be filled out manually. At the same time, the paper forms must be entered into various information systems. This involves the tedious work of filling out the paper forms once and manually entering them again into the software system. The paper documents need to be archived and preserved, and the information system can analyze and count the data. For example, the civil affairs department conducts a population census, the economic department conducts a census of enterprises, the agricultural department conducts field surveys on agricultural planting and production data, the forestry department conducts surveys and records forest conditions, and so on. These industries have various paper forms that need to be filled out and entered manually at the survey site. After returning to the office, the data in the paper forms may need to be re-entered into the software system, which brings a lot of workload to office staff. Some industries, no matter how rich their level of informatization, still cannot do without paper forms. For example, when the civil affairs department conducts household surveys, the people are not very accepting of the household personnel slowly entering various information on mobile devices. They are more accepting of filling in paper forms because the respondents can see the various information written by the household personnel. For example, when the agricultural, forestry, and marine industries investigate various data on forests, farmlands, and oceans, there may generally be no mobile phone signal, and there may be water, mud, and other dirt on their hands, or they may be filling in information under the scorching sun, and the screens of mobile phones and other devices are basically invisible under the scorching sun, so the best way to record is on paper forms. After paper records have been in existence for a long time, the recording paper may age and oxidize, causing the handwriting to become blurred. There is no way to restore the previous handwriting except copying, scanning or taking photos.

[0003] In order to solve the problem of synchronization between paper reports and information systems, there are already some solutions on the market, such as: some scan paper forms and then use image recognition and manual proofreading (archives management departments often use this method to process historical paper forms in an informatized manner); some use an LCD touch screen mobile device, and various standard forms are displayed on the LCD screen, and users write and enter them on the LCD screen with their fingers or pens (banks and telecommunications offices are currently using it on a large scale, and it is an effective method of paperless office); and some directly use mobile phone software or specific hardware devices to realize the informatization of form entry (for example, traffic police and express delivery industries have special equipment to enter information, but this type of information and forms are relatively simple and basically do not change; some industries use mobile phone software to fill in various dynamic forms).

[0004] In the current technical field, there is no system and method that can synchronize original handwriting records with information records, and all kinds of handwriting written on paper can be fully synchronized in the information system without the need for manual re-entry on a computer or mobile phone, and can also allow paper record information to be directly saved in the information system. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a paper form handwriting automatic entry and transmission system and method, by placing a paper form of the same type as the form stored in the server and APP on the electromagnetic induction board for filling in the form, the electromagnetic induction board records the handwriting through the pressure of the electromagnetic pen on the electromagnetic induction board, and then transmits the handwriting data to the calculation and recognition unit of the mobile phone APP or transmits it to the server through the GPRS / 4G / 5G network. The mobile phone APP or server automatically matches the received data and fills it into the corresponding form for storage. Then the server transmits the form data information to the APP through the GPRS / 4G / 5G network, which is convenient for the mobile phone to check the information anytime and anywhere, and also saves the tedious process of manually entering the form after filling in the information on paper. At the same time, the data recognized by the server is stored, which is also convenient for big data to organize and analyze the data of previous years. When the original paper form is damaged or lost, the original handwriting data and the form can be retrieved and printed out on the same paper material as the original paper form for re-archiving and use.

[0006] The present invention solves the above technical problems through the following technical means:

[0007] The present invention provides a paper form handwriting automatic entry and transmission system, comprising an electromagnetic pen, an electromagnetic induction board, a transmission module and a mobile phone APP. The transmission module is installed in the electromagnetic induction board, the output end of the electromagnetic induction board is connected to the input end of the transmission module, the transmission module is connected to the mobile phone via Bluetooth or Wi-Fi, and the mobile phone APP is used to identify and store handwriting information.

[0008] Furthermore, the electromagnetic pen is in the form of a commonly used carbon pen or ballpoint pen combined with an electromagnetic coil. By writing text on a paper fixed on the electromagnetic induction plate, the electromagnetic induction plate can measure the position and pressure of the electromagnetic pen multiple times per second, connect the position coordinates in sequence, and record all the handwriting information of the writer.

[0009] Furthermore, a method for automatically recording and transmitting handwriting includes the following steps:

[0010] S1: The electromagnetic induction board collects the handwriting and pressure data of the electromagnetic pen on the electromagnetic board;

[0011] S2: The handwriting data is transmitted to the mobile phone APP in real time through the transmission module;

[0012] S3: The mobile app transmits the handwriting data to the computing and recognition unit in the mobile app;

[0013] S4: The computing and recognition unit can run on the mobile phone or on a remote server. When running on a remote server, the mobile phone APP transmits the handwriting data to the computing and recognition unit in the server via the mobile phone's 4G / 5G / Wi-Fi network.

[0014] S5: The computing and recognition unit in the server recognizes and processes the handwriting through the algorithm module;

[0015] S6: The recognized table ID number and the text information recognized from each cell are transmitted back to the mobile phone APP for displaying the table information on the APP.

[0016] Furthermore, the calculation and recognition unit described in step S4 identifies the concentration and value range of the handwriting, thereby identifying the corresponding form ID number of the form in the APP or backend service system. The server uses the calculation and recognition unit to identify the ID of the paper form based on the received information, and then performs detailed handwriting recognition on each cell area of the form, recognizes the text information, processes it, and then transmits it to the APP via a wireless network such as GPRS / 4G / 5G / Wi-Fi;

[0017] Furthermore, the calculation and recognition unit described in step S4 identifies the concentration and value range information of the handwriting, thereby identifying the table ID number corresponding to the table in the APP or background service system.

[0018] Furthermore, the training and learning of the computing and recognition unit specifically includes the following steps:

[0019] S1: Electromagnetic induction board records a large amount of handwriting data;

[0020] S2: Record the location of the manually marked cells, the value range of the manually marked cells, the data validity category, the writing range of the manually marked cells, and the actual content of each cell manually entered;

[0021] S3: combining the data recorded in S1 with the handwriting coordinate concentration index determination algorithm to form a handwriting focus index for each paper form;

[0022] S4: Combine the data recorded in S2 to form a manually annotated data set;

[0023] S5: Use the manually annotated data set to train the model and form unique features for each table;

[0024] S6: Determine whether the feature is a unique feature;

[0025] S7: If yes, train the recognition algorithm; otherwise, return to step S5.

[0026] Furthermore, the recognition application of the computing recognition unit includes the following steps:

[0027] S1: The mobile phone APP transmits all the handwriting data on a piece of paper to the calculation and recognition unit in the mobile phone APP;

[0028] S2: The calculation and recognition unit in the mobile phone APP processes all handwriting data into handwriting concentration indicators;

[0029] S3: The calculation and recognition unit recognizes all the handwriting data into text information;

[0030] S4: combining the processing result data of S2 and S3, using the algorithm module to identify the matching degree of known table features, directly giving a table ID number when the matching degree is higher than a specific one, and recording it as a new business table when the matching degree is lower than the specific one, reminding the management personnel to train the new business table model, and returning to the step described in claim 6;

[0031] S5: Based on the recognized table ID number, the coordinates of each cell in the table are re-segmented to all handwriting data. The algorithm module is reused to perform text information recognition and extraction based on the segmented handwriting to improve the accuracy of text information recognition. The recognized text information is returned to the mobile phone APP or stored in the server database;

[0032] Furthermore, the calculation and identification unit can be run in a mobile phone app or in a remote server. When running in a remote server, the computing power of the server can improve the calculation speed and depth of the algorithm module, thereby obtaining more accurate calculation results.

[0033] Furthermore, the mobile phone APP can automatically identify the form ID number through the calculation and recognition unit, or the user can manually select or input the form ID number; manually selecting or inputting the form ID number can also assist the algorithm module in correcting and training the recognition accuracy of the form ID number;

[0034] Furthermore, after obtaining the form ID number, the mobile phone APP can superimpose the handwriting data and the blank form image data, or superimpose the recognized text information and the blank form image data, and the user can also edit and modify the recognized text information;

[0035] Furthermore, after finishing a sheet of paper, the user only needs to switch to a new sheet of paper to continue writing, without having to notify the software on the app or device that the sheet has been changed. The computing and recognition unit can automatically determine that the user has written on a new sheet based on the degree of handwriting overlap, and will automatically segment the received handwriting, without storing the handwriting on the next sheet of paper in the previous sheet, thereby further achieving the user's writing fluency.

[0036] Furthermore, the ID number of the form can also be printed out in a specific area of the paper form, and a blank space for filling in the form ID number can be printed out. Before filling in the form, the user only needs to copy the ID number according to the printed ID number. When the algorithm program finds handwriting in this area, it can perform digital recognition on the handwriting in this area to more accurately extract the ID number of the form.

[0037] Furthermore, if all paper forms are pre-set with form IDs in the system and printed in batches from the system, and blank spaces are reserved for copying form IDs, the system's algorithm module may not require manual training and marking, or a very small amount of manual marking can achieve high accuracy. For some new industry applications, they can quickly enter the batch use period, reducing the difficulty and workload of early manual marking learning.

[0038] The beneficial effects of the present invention are as follows: by attaching a paper form of the same type as the form stored in the server and the APP to the electromagnetic induction board for filling in the form, the electromagnetic induction board records the handwriting through the pressure of the electromagnetic pen on the electromagnetic induction board, and then transmits the handwriting data to the server through the GPRS / 4G / 5G network. The server automatically matches the received data and fills it into the corresponding form for storage. The server then transmits the form data information to the APP through the GPRS / 4G / 5G network, making it convenient for the mobile phone to check the information anytime and anywhere, and also eliminating the tedious process of manually entering the form after filling in the information on paper. At the same time, the data recognized by the server is stored, which is also convenient for big data to organize and analyze the data of previous years. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a system for automatically inputting and transmitting handwriting on paper forms according to the present invention;

[0040] Figure 2 The figure is a flow chart of the calculation and recognition unit of the system and method for automatically inputting and transmitting handwriting on paper forms of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] like Figure 1-Figure 2 As shown, the present invention provides a system and method for automatically inputting and transmitting handwriting on paper forms.

[0043] In this embodiment, the present invention provides a paper form handwriting automatic entry and transmission system, including an electromagnetic pen, an electromagnetic induction board, a transmission module and a mobile phone APP. The transmission module is installed in the electromagnetic induction board, the output end of the electromagnetic induction board is connected to the input end of the transmission module, the transmission module is connected to the mobile phone via Bluetooth and Wi-Fi, and the mobile phone APP is used to identify and store handwriting information.

[0044] In this embodiment, the electromagnetic pen is in the form of a commonly used carbon pen or ballpoint pen combined with an electromagnetic coil. At the same time, by writing text on a paper fixed on the electromagnetic induction plate, the electromagnetic induction plate can measure the position and pressure of the electromagnetic pen multiple times per second, connect the position coordinates in sequence, and record all the handwriting information of the writer.

[0045] Furthermore, a method for automatically recording and transmitting handwriting includes the following steps:

[0046] S1: The electromagnetic induction board collects the handwriting and pressure data of the electromagnetic pen on the electromagnetic board;

[0047] S2: The handwriting data is transmitted to the mobile phone APP in real time through the transmission module;

[0048] S3: The mobile app transmits the handwriting data to the computing and recognition unit in the mobile app;

[0049] S4: The computing and recognition unit can run on the mobile phone or on a remote server. When running on a remote server, the mobile phone APP transmits the handwriting data to the computing and recognition unit in the server via the mobile phone's 4G / 5G / Wi-Fi network.

[0050] S5: The computing and recognition unit in the server recognizes and processes the handwriting through the algorithm module;

[0051] S6: The recognized table ID number and the text information recognized from each cell are transmitted back to the mobile phone APP for displaying the table information on the APP.

[0052] In this embodiment, the calculation and recognition unit described in step S4 is used to identify the concentration and value range of the handwriting, thereby identifying the corresponding form ID number of the form in the APP or backend service system. The server uses the calculation and recognition unit to identify the ID of the paper form based on the received information, and then performs detailed handwriting recognition on each cell area of the form, identifies the text information, processes it, and then transmits it to the APP via a wireless network such as GPRS / 4G / 5G / Wi-Fi;

[0053] In this embodiment, the calculation and recognition unit described in step S4 identifies the concentration and value range information of the handwriting, thereby identifying the table ID number corresponding to the table in the APP or background service system.

[0054] In this embodiment, the training and learning of the computing and recognition unit specifically includes the following steps:

[0055] S1: Electromagnetic induction board records a large amount of handwriting data;

[0056] S2: Record the location of the manually marked cells, the value range of the manually marked cells, the data validity category, the writing range of the manually marked cells, and the actual content of each cell manually entered;

[0057] S3: combining the data recorded in S1 with the handwriting coordinate concentration index determination algorithm to form a handwriting focus index for each paper form;

[0058] S4: Combine the data recorded in S2 to form a manually annotated data set;

[0059] S5: Use the manually annotated data set to train the model and form unique features for each table;

[0060] S6: Determine whether the feature is a unique feature;

[0061] S7: If yes, train the recognition algorithm; otherwise, return to step S5.

[0062] In this embodiment, the recognition application of the computing and recognition unit includes the following steps:

[0063] S1: The mobile phone APP transmits all the handwriting data on a piece of paper to the calculation and recognition unit in the mobile phone APP;

[0064] S2: The calculation and recognition unit in the mobile phone APP processes all handwriting data into handwriting concentration indicators;

[0065] S3: The calculation and recognition unit recognizes all the handwriting data into text information;

[0066] S4: combining the processing result data of S2 and S3, using the algorithm module to identify the matching degree of known table features, directly giving a table ID number when the matching degree is higher than a specific one, and recording it as a new business table when the matching degree is lower than the specific one, reminding the management personnel to train the new business table model, and returning to the step described in claim 6;

[0067] S5: Based on the recognized table ID number, the coordinates of each cell in the table are re-segmented to all handwriting data. The algorithm module is reused to perform text information recognition and extraction based on the segmented handwriting to improve the accuracy of text information recognition. The recognized text information is returned to the mobile phone APP or stored in the server database;

[0068] In this embodiment, the calculation and identification unit can be run in a mobile phone APP or in a remote server. When running in a remote server, the computing power of the server can improve the calculation speed and calculation depth of the algorithm module, thereby obtaining more accurate calculation results.

[0069] In this embodiment, the mobile phone APP can automatically identify the form ID number through the calculation and recognition unit, or the user can manually select or input the form ID number; manually selecting or inputting the form ID number can also assist the algorithm module in correcting and training the recognition accuracy of the form ID number;

[0070] In this embodiment, after obtaining the form ID number, the mobile phone APP can superimpose the handwriting data and the blank form image data, or superimpose the recognized text information and the blank form image data, and the user can also edit and modify the recognized text information;

[0071] In this embodiment, after the user finishes writing on a sheet of paper, they only need to switch to a new sheet of paper to continue writing. There is no need to notify the software on the app or device that the sheet has been changed. The computing and recognition unit can automatically determine that the user has written on a new sheet based on the degree of handwriting overlap, and will automatically segment the received handwriting. The handwriting written on the next sheet of paper will not be stored in the previous sheet, thereby further achieving the user's writing fluency.

[0072] In this embodiment, the ID number of the form can also be printed out in a specific area of the paper form, and a blank space for filling in the form ID number can be printed out. Before filling in the form, the user only needs to copy the ID number according to the printed ID number. When the algorithm program finds handwriting in this area, it can perform digital recognition on the handwriting in this area to more accurately extract the ID number of the form.

[0073] In this embodiment, if all paper forms are pre-set with form IDs in the system and are printed in batches from the system, and blank spaces are reserved for copying the form IDs, the system's algorithm module may not require manual training and marking, or a very small amount of manual marking can achieve high accuracy. For some new industry applications, they can quickly enter the batch use period, reducing the difficulty and workload of early manual marking learning.

[0074] In this embodiment, after the user finishes writing on a sheet of paper, they only need to switch to a new sheet of paper to continue writing. There is no need to notify the software on the app or device that the sheet has been changed. The computing and recognition unit can automatically determine that the user has written on a new sheet based on the degree of handwriting overlap, and will automatically segment the received handwriting. The handwriting written on the next sheet of paper will not be stored in the previous sheet, thereby further achieving the user's writing fluency.

[0075] In a second embodiment of the present invention, the method of ID recognition of a paper form can be implemented by other conventional methods. In this embodiment, a QR code / barcode is generated on the form, and the ID of the currently filled-in form is obtained by scanning the QR code through an APP or a barcode scanning device.

[0076] In a third embodiment of the present invention, a method for ID identification of a paper form is to print three positioning points on the four corners of the paper. Before filling out a new form, the user uses a pen to "click" or "draw a circle" on the three positioning points. This can notify the APP and server of the angle and position of the paper, and can also notify the APP and server to start filling out a new form.

[0077] In a fourth embodiment of the present invention, a method for identifying the ID of a paper form is to print a form code on the header of the paper form and print a corresponding blank space below the code. Before filling out a new form, the user copies the form code number printed on it into the blank space. The APP and the server can also obtain the ID of the paper form by identifying the number in this fixed area, and know that the user has started to fill out a new form.

[0078] The above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit the scope of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention, and such modifications or equivalents shall be encompassed by the claims of the present invention. Any techniques, shapes, and structures not described in detail herein are well known.

Claims

1. A method for automatically recording and transmitting handwriting on a paper form, comprising the following steps: S1: The electromagnetic induction board collects the handwriting and pressure data of the electromagnetic pen on the electromagnetic induction board; S2: The handwriting data is transmitted to the mobile phone APP in real time through the transmission module; S3: The mobile phone APP transmits the handwriting data to the calculation and recognition unit in the mobile phone APP. The training and learning of the calculation and recognition unit specifically includes the following steps: S3.1: The electromagnetic induction board records a large amount of handwriting data; combined with the handwriting coordinate index determination algorithm, a handwriting focus index is generated for each paper form; S3.2: Record the location of the manually annotated cells, the value range of the manually annotated cells, the data validity category, the writing range of the manually annotated cells, and the actual content of each cell manually entered to form a manually annotated data set; S3.3: Use the manually annotated data set to train the model and form unique features for each table; S3.4: Determine whether the feature is unique; S3.5: If yes, train the recognition algorithm; otherwise, return to step S3.3; S4: The calculation and recognition unit runs in the mobile phone app and the server. When the calculation and recognition unit runs on the server, the mobile phone app transmits the handwriting data to the calculation and recognition unit in the server via the mobile phone's 4G / 5G / Wi-Fi network. S5: The calculation and recognition unit performs handwriting recognition processing and automatically identifies the form ID number, including the calculation and recognition unit identifying the concentration and value range information of the handwriting, thereby identifying the form ID number corresponding to the form in the APP or backend service system, specifically including the following steps: S5.1: The mobile phone APP transmits all handwriting data on a piece of paper to the computing and recognition unit in the mobile phone APP; S5.2: The computation and recognition unit in the mobile app processes all handwriting data into a handwriting concentration index; S5.3: The computational recognition unit in the mobile phone app recognizes all handwriting data into text information; S5.4: Combine the processing results of S2 and S3 to identify the matching degree of known table features. If the matching degree is higher than a certain level, the table ID number will be directly assigned. If the matching degree is lower than the certain level, it will be recorded as a new business table, and the management personnel will be reminded to train the new business table model. S5.5: Based on the identified table ID number, re-segment all handwriting data using the coordinates of each cell in the table, and re-use the algorithm module to perform text information recognition and extraction based on the segmented handwriting. The recognized text information is returned to the mobile phone app or stored in the server database; S6: The recognized table ID number and the text information recognized from each cell are transmitted back to the mobile phone APP for displaying the table information on the APP.

2. The method for automatically recording and transmitting handwriting on a paper form according to claim 1, characterized in that: The computing and identifying unit runs in a mobile phone APP or in a remote server.

3. The method for automatically recording and transmitting handwriting on a paper form according to claim 1, characterized in that: After obtaining the form ID number, the mobile phone APP superimposes and displays the handwriting data and the blank form picture data, and superimposes and displays the recognized text information and the blank form picture data, and the user edits and modifies the recognized text information.

Images