Storage medium, information processing apparatus, and information processing method

Abstract

Embodiments provide a storage medium, an information processing apparatus, and an information processing method. The present invention makes it possible to determine an appropriate input form even if there is no expert for voice recognition. A storage medium according to an embodiment stores an information processing program. The information processing program causes a computer to implement an information acquisition unit and a form estimation unit. The information acquisition unit acquires, for each item, one or more items and information relating to values of an input field for the one or more items from a recording data list including the input field for voice input. The form estimation unit estimates the input form of the input field on the basis of the information.

Description

[0001] This application is based on Japanese Patent Application No. 2023-097228 (filing date: 6 / 13 / 2023) and benefits from priority from that application. This application incorporates the entire contents of that application by reference. Technical Field

[0002] Embodiments relate to a storage medium, an information processing device, and an information processing method. Background Art

[0003] Sometimes, at a manufacturing site or a maintenance inspection site, the measured values ​​measured by a measuring machine and the inspection results by visual inspection are input into a recording data sheet such as a bill or a table, and the inputted recording data sheet is shared between operators or with customers. In this case, the contents to be input into the recording data sheet are determined in advance, and the operators perform the work according to the work process and input the obtained data into a predetermined position in the recording data sheet.

[0004] In general accounting software, data is input in text, but inputting text takes time during operation, so there is a demand for inputting data using voice. For example, there is a voice input method in which an input object item and the content to be input to the item are set in an application independent of the accounting software, and a value can be input in the selected item when speaking, and the next item to be input is specified when setting, so that values ​​can be continuously input in the items. According to such a voice input method, the input form of the input value is predetermined by a voice recognition expert, and voice input can be performed according to the input form.

[0005] The above-mentioned voice input method generally has no particular problems. However, according to the research of the present inventors, it is difficult to determine an appropriate input format when there is no expert in voice recognition, and there is room for improvement. Summary of the invention

[0006] An object of the present invention is to provide a storage medium, an information processing device, and an information processing method that can determine an appropriate input format even when a voice recognition expert is not present.

[0007] The storage medium involved in the embodiment stores an information processing program. The information processing program enables a computer to implement an information acquisition unit and a form estimation unit. The information acquisition unit acquires, for each item, one or more items and information related to the value of the input field for the one or more items from a recording data sheet including an input field for voice input. The form estimation unit estimates the input form of the input field based on the information.

[0008] According to the storage medium having the above configuration, even when an expert in voice recognition is not available, an appropriate input format can be determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a block diagram showing the configuration of the information processing device according to the first embodiment.

[0010] Figure 2 This is a diagram showing an example of document data according to the first embodiment.

[0011] Figure 3 This is a diagram showing an example of initial setting of the input procedure storage unit according to the first embodiment.

[0012] Figure 4 This is a diagram showing an example of a word list according to the first embodiment.

[0013] Figure 5 This is a diagram showing an example of an inputted form according to the first embodiment.

[0014] Figure 6 This is a diagram showing completion of automatic setting of the input procedure storage unit according to the first embodiment.

[0015] Figure 7 This is a diagram showing an example of a guidance input form correspondence table according to the first embodiment.

[0016] Figure 8 This is a flowchart showing the input procedure creation processing according to the first embodiment.

[0017] Fig. 9 This is a diagram showing an example of range selection according to the first embodiment.

[0018] Fig.10 This is a diagram showing an example of a warning according to the first embodiment.

[0019] Fig.11 This is a diagram showing another example of the warning according to the first embodiment.

[0020] Fig.12 This is a diagram showing an example of the input format display of the form according to the first embodiment.

[0021] Fig.13 This is a diagram showing an example of display of input value examples of a form according to the first embodiment.

[0022] Fig.14 This is a diagram showing an example of the input order display of the form according to the first embodiment.

[0023] Fig.15 This is a diagram showing another example of the input order display of the form according to the first embodiment.

[0024] Fig.16 It is a diagram showing the format estimation process (no connection) according to the first embodiment.

[0025] Fig.17 This is a diagram showing an example of the format allocation rule according to the first embodiment.

[0026] Fig.18 It is a diagram showing the form estimation (no connection) according to the first embodiment.

[0027] Fig.19 It is a diagram showing the format inspection process according to the first embodiment.

[0028] Fig. 20 It is a diagram showing the voice input process according to the first embodiment.

[0029] Fig.21 This is a block diagram showing the configuration of an information processing device according to the second embodiment.

[0030] Fig. 22 This is a diagram showing an example of automatic range selection according to the second embodiment.

[0031] Fig.23 This is a flowchart showing the input procedure creation processing involved in the second embodiment.

[0032] Fig.24 This is a block diagram showing the structure of an information processing device according to the third embodiment.

[0033] Fig.25 This is a flowchart showing the input procedure creation processing involved in the third embodiment.

[0034] Fig.26 This is a flowchart showing the format estimation process (no connection) according to the third embodiment.

[0035] Fig. 27 This is a flowchart showing the format inspection process according to the third embodiment.

[0036] Fig.28 This is a block diagram showing the structure of an information processing device according to a fourth embodiment.

[0037] Fig.29 This is a flowchart showing the operation example display process according to the fourth embodiment.

[0038] Fig.30This is a diagram showing a hardware configuration example of an information processing device according to the fifth embodiment.

[0039] (Explanation of symbols)

[0040] 1: information processing device; 2: data storage unit; 2a: account data; 3: input process storage unit; 3a: input process; 4: input value example acquisition unit; 5: range selection unit; 6: form estimation unit; 7: form checking unit; 8: dictionary generation unit; 9: sound recognition unit; 10: sound synthesis unit; 11: recognition control unit; 12: display unit; 13: format setting acquisition unit; 14: action example display unit. DETAILED DESCRIPTION

[0041] Hereinafter, each embodiment will be described in detail with reference to the accompanying drawings. In the following description, the case where the input process production process and the sound input process are executed in an information processing device such as a tablet terminal with a sound input function is described as an example. In addition, as an information processing device, it is not limited to a tablet terminal, such as a PC (Personal Computer) equipped with an external microphone and an external speaker, a smart phone, etc., and any computer can be used appropriately. In addition, the information processing device does not necessarily need to perform the input process production process and the sound input process, but can be a device that at least performs the input process production process. That is, the program for the input process production process and the program for the sound input process can be installed as one information processing program or as individual information processing programs. In either case, the information processing device realizes the functions of the input process production process and the sound input process by executing the information processing program. In addition, when the information processing device only performs the input process production process in the input process production process and the sound input process, the microphone can also be omitted. In addition, in the following description, the user related to the input process production process is referred to as the input process production personnel, and the user related to the sound input process is referred to as the sound input personnel. The voice input personnel and the input process production personnel may be the same person or different persons. In addition, multiple voice input personnel and multiple input process production personnel may collaborate.

[0042] <First Embodiment>

[0043] Figure 1 1 is a block diagram showing an example of the structure of the information processing device according to the first embodiment. The information processing device 1 includes a data storage unit 2, an input process storage unit 3, an input value example acquisition unit 4, a range selection unit 5, a form estimation unit 6, a form check unit 7, a dictionary generation unit 8, a voice recognition unit 9, a voice synthesis unit 10, a recognition control unit 11, and a display unit 12. The information processing device 1 is a computer that realizes the functions of each unit by executing an information processing program installed in a memory.

[0044] Here, the data storage unit 2 is as follows Figure 2 As shown, the account data 2a stores certain data of an input column indicating voice input for each item of a account sheet, table, etc. The account data 2a is an example of a record data sheet including an input column for voice input for each item. For example, the account data 2a inputs a value in an input column associated with each item of the row and column numbers B to D respectively shown in row numbers 1 to 5, and inputs a value in an input column associated with each item of the row and column numbers B to D respectively shown in row numbers 8 to 9. In addition, the account data 2a can also set the word table described later to another table. The data storage unit 2 is an example of a memory of a computer.

[0045] Input process storage unit 3 Figure 3 As shown, an input process 3a for inputting values ​​in a plurality of input fields included in the account sheet data 2a as a record data sheet is stored. The input process 3a includes, for example, a process number, an input object item, an input form, a guide, and a reference value as an item name. However, the process number, the guide, and the reference value are arbitrary additional items and may be omitted. Such an input process 3a also functions as a list of the order in which input is performed in a plurality of items. The order of the input process 3a may be determined from top to bottom in the list, or the items of the input process may have numbers indicating the order. The input process storage unit 3 is an example of a memory of a computer.

[0046] Here, in the item name of the process number, a number indicating the order in which values ​​are input in a plurality of input fields included in the form data 2 a is shown as the value of the process number.

[0047] The name of the input object item may also be used as the value of the input object item, indicating an identifier for identifying the input field of the input object item included in the form data 2a. Figure 2 If the table is shown, the column and row numbers can also be used as shown in D2. Figure 3 In the input process 3a, an example is shown in which after B1, B2, ..., B5 are input as the first measurement result, B8, C8, D8 are input, and C1, C2, ..., C5 are continuously input as the second measurement result.

[0048] As the value of the input form, the item name of the input form indicates the input form estimated by the form estimation unit 6. In addition, the input process 3a includes an input form column in which the estimated input form can be input, and the input form estimated by the form estimation unit 6 is input in the input form column. In addition, the input form specifies the form of the value to be input into the input column of the input object item. In the recognition control unit 11 described later, a voice recognition dictionary is created based on the input form input into the input form column. It is difficult to record the voice recognition dictionary directly in the input process, so in this embodiment, the input form of simplified description is input.

[0049] As a broad classification of the input form, for example, "numeric value", "alphanumeric", "date and time", "date", "time", "word", etc. can be used appropriately.

[0050] In the case of "numeric value", you can further specify the base of the number (decimal, hexadecimal, etc.), the number of digits in the integer part in the case of a decimal number (a range specification is also possible), the number of digits in the decimal part, etc.

[0051] For example, when a decimal number is set, the range of the digits of the integer part is Mmin~Mmax, and the range of the digits of the decimal part is Nmin~Mmin, the input format is recorded as "decimal number_Mmin~Mmax_Nmin~Nmax". In addition, when the number of digits of the "numerical value" has no upper limit, the input format is recorded as "decimal number_Mmin~_Nmin~", and when the number of digits of the "numerical value" has no lower limit, it is recorded as "decimal number_~Mmax_~Nmax". In addition, when there is no range of the digits of the "numerical value" (the number of digits of the integer part is m, and the number of digits of the decimal part is N), the input format is recorded as "decimal number_M_N". In addition, in the "numerical value", when the positive and negative signs "+ (positive)" and "- (negative)" are pronounced before the number, the input format is recorded as "[+-] decimal number_M_N".

[0052] In the case of "alphanumeric", further, a pattern of English letters, numbers, and symbols can be specified. For example, as an alphanumeric pattern, it is also possible to record "A" if it is an English letter, "D" if it is a number, and "S" if it is a symbol. In addition, as "alphanumeric", it is also possible to use "$" if it is an English letter or a number, and use "!" if it is an English letter, a number, or a symbol. For example, if it is a pattern of 3 English letters, 2 numbers, 1 symbol, and 1 English letter, it is recorded as "alphanumeric_AAADDSA".

[0053] In addition, in "alphanumeric", "?" is added to optional characters (characters that can be entered or not). For example, the pattern "alphanumeric_AAA?DD" represents a pattern of 2 alphanumeric characters or 3 characters and 2 numbers. In addition, it is possible to describe a pattern of not only 1 character but also a pattern of multiple characters that can be entered.

[0054] In the case of "Date and time", "Date" and "Time", no special additional designation is required, but regarding "Date", it is also possible to record additional designations such as specifying which content to input in the date such as "Year-Month-Day", "Month-Day", "Year-Month", and "Day".

[0055] In the case of "words", as in Figure 4 As shown in an example, a word list 2b is described in another table of the account data 2a, and the group name of the word list 2b is specified. In addition to the group name, the word list 2b also includes the expression of the word, the corresponding reading (the reading spoken by the user), and the reading aloud (the content read aloud when repeating through guidance, etc.). In this embodiment, in order to automatically generate the word list 2b, a number is automatically assigned to each group name. Only the expression of the word can be obtained from the input value example, so the content of reading and reading aloud is generated according to the expression. Regarding the items of reading and reading aloud, it is also possible to prepare them as other items in a way that the input process maker can change them to other contents later, but the same content can be generated and set at the generation stage. In the generation of reading and reading aloud, any existing method can also be used. For example, morpheme analysis technology and a dictionary for morpheme analysis can be used.

[0056] In addition, the input form is not limited to the above-mentioned major categories, and any input form that can be converted into a sound recognition dictionary can be used, or a combination of various input forms can be used. For example, as an input form, a linked form such as "date_month day, word_week, (time_hour and minute)" can also be used. In this case, after the pronunciation of the date, the word group "week" (Sunday, Monday, Tuesday, ...) can be pronounced, and then the time can be pronounced or not. The () in the link indicates that the pronunciation is optional. In this case, the result of the recognition becomes space-separated. For example, if it is "date_month day, word_week, (time_hour and minute)", then as the recognition result, "3 / 10 (space) Friday (space) 9:10", "3 / 11 (space) Saturday" and the like are obtained.

[0057] In addition, regarding the input form, for example, OR of multiple input forms such as "time_hour and minute|time_hour, minute, second", "decimal number_2_1|word_whether there is an abnormality" can also be used as the input form. In this case, it is also possible to pronounce one of "time_hour and minute" and "time_hour, minute, second", and it is also possible to pronounce one of "decimal number_2_1" and "word_whether there is an abnormality". For example, if it is "time_hour and minute|time_hour, minute, second", it is possible to identify "9:10", "13:20:10", etc., and if it is "decimal number_2_1|word_whether there is an abnormality", it is possible to identify "12.3", "abnormality", etc.

[0058] In addition, the input form can also use the above-mentioned connection form and the combination OR form. In addition, the description method of the input form is not limited to the above, and any description method that can be converted into a voice recognition dictionary is sufficient. In addition, when the input form is specified as narrow as possible (if it is a numerical value, the range of digits is narrow, if it is a word, the number of word lists is reduced), although the voice recognition rate is improved, the utterance content that can be input is reduced, so when it is too detailed, the convenience of use of the voice input personnel becomes lower.

[0059] Regarding the item name of the guidance, as the value of the guidance, when the input process is actually executed and the voice input is performed, it is used as the content played at the beginning of each process to prompt the operation instructions in the corresponding process. In addition, the guidance is not limited to the item name of the input object item, and it can be a guidance related to the voice input to the input field. For example, in the case of the input process of the inspection, the guidance includes a description of the content of the guidance (guidance) inspection process. In the input form inspection of the subsequent stage, only the guidance that meets the specific pattern such as the 00 time is used among all the guidance.

[0060] The item name of the reference value is a value that represents the range of values ​​that become the reference for the value input into the input field. The reference value may also include a value that represents the range of values ​​of the input field, and an upper limit value, a lower limit value, or any value of the upper and lower limits. The reference value is used to check whether the value input in the voice input is abnormal when the input process is actually performed to input the voice.

[0061] As described above, the input process 3a is not limited to being maintained in the input process storage unit 3 as data different from the account data 2a, but can also be maintained on the account data 2a. For example, in general table calculation applications, multiple tables can be created on one file, so it can also be implemented in a way that one of them is used as account data and the other is used as input process maintenance. In this case, the input process is obtained from the process maintenance table in the identification control unit 11 described later. In addition, the input process 3a can be maintained in a storage device such as a DB (Database) or in a memory when it is maintained independently from the account data 2a. In this embodiment, it is assumed that there is a table for maintaining the input process in other tables of the file of the account data 2a. In this case, by copying the table for maintaining the input process to the file of other account data, even the account data at the copy destination can be input by voice.

[0062] In addition, the input process 3a can also be created by manually inputting an area other than the input form. For example, when creating a table for process storage in the file of the form data 2a, it can also be created by manually inputting an area other than the input form. Figure 3 The same table is manually input into the table for process storage to create an input process 3a other than the input form. Alternatively, the input process 3a can be created by clicking an input target item to display a dialog box and manually inputting information other than the input form on the dialog box.

[0063] The input value example acquisition unit 4 acquires one or more items and information related to the value of the input field for one or more items from the form data 2a including the input field for voice input for each item. As the information related to the value of the input field, for example, an input value example as an example of the value of the input field and a format setting for displaying the value of the input field can be used appropriately. In the first embodiment, the input value example is used as the information related to the value of the input field. In this case, the input value example acquisition unit 4 can acquire the value as the input value example from at least one of the form data 2a in which the value of the input field is pre-entered and the user interface that accepts the value input to the input field. As the form data 2a in which the value of the input field is pre-entered, if at least one of all the input fields of the form data 2a has been entered, the input value example can be acquired from the input field that has been entered. In addition, the input value example acquisition unit 4 can also acquire information related to the value of the input field corresponding to the input field of the range selected by the range selection unit 5. In addition, “information related to the value of the input field” may be appropriately read as “input notification to the input field”, “input notification information to the input field”, etc. The input value example acquisition unit 4 is an example of an information acquisition unit.

[0064] Figure 5An example of inputted account sheet data 2a is shown. In the case of this account sheet data 2a, only position 1 and position 2 of #1 are inputted with respect to the first and second measurement result values ​​and noise measurement. The input value example acquisition unit 4 acquires the value of the item indicated by the identifier of the input object item of each input process for the inputted account sheet data 2a, and acquires the input value example. For example, Figure 5 The bill data 2a, about Figure 3 The input value example of the input process of process number 1 only needs to obtain the value of the input object item of identifier B1, so it becomes "10:30". Regarding such inputted account data 2a, you can use the data inputted by manual input before setting the voice input process, or you can manually input the input value example for setting a new one. In addition, the inputted account data 2a may not be one but multiple. In the case of multiple account data 2a, multiple input value examples can be obtained for one input process. In addition, assuming that the process is set from a user interface such as a dialog box, a text box for accepting the input of the input value example can also be added to the dialog box, and the input value example acquisition unit 4 acquires the input value example by acquiring the text input into the text box.

[0065] The range selection unit 5 selects a range of a plurality of input fields from the form data 2a. In addition, the range selection unit 5 selects a range of a plurality of input fields as a range for estimating the input form. For example, the range selection unit 5 provides a user interface for selecting a range for setting the input form. The range selection unit 5 may use a function of an existing application for displaying a form, or may use other methods.

[0066] The form estimation unit 6 estimates the input form of the input column based on the input value example acquired by the input value example acquisition unit 4. In addition, the form estimation unit 6 may estimate the common input form in the input column of the range selected by the range selection unit 5 based on the input value example acquired corresponding to the input column of the range. In addition, the common input form is estimated based on a plurality of input value examples. In addition, the form estimation unit 6 may estimate the input form for each input column of the range, and estimate one input form among the estimated input forms as the common input form. In addition, the form estimation unit 6 may estimate the input form for each input column of the range, and estimate a plurality of input forms included in the estimated input forms as the common input form. In addition, the form estimation unit 6 may adjust the estimated input form based on the number of input value examples acquired corresponding to the input column of the selected range. In addition, the form estimation unit 6 may adjust the estimated input form based on the reference value (example, upper and lower limit values) included in the input process 3a. In any case, the form estimation unit 6 estimates the input form based on the input value example, and updates the input form column of the input process 3a with the estimated input form. Figure 6An example of the updated input process 3a is shown. In this case, the input format has been set for the input process 3a to the items other than B4, C4, and D4.

[0067] The format checking unit 7 checks the validity of the estimated input format. For example, the format checking unit 7 checks the estimated input format based on the guidance included in the input process 3a.

[0068] For example, the form checking unit 7 generates an input value example based on the inferred input form, performs sound synthesis on the generated input value example, and performs sound recognition using a sound recognition dictionary for the sound data (sound waveform data) obtained by the sound synthesis. The text of the sound recognition result represents the input value example, a command, or something else. In addition, the form checking unit 7 can also check the inferred input form by determining whether the text of the sound recognition result obtained and the generated input value example match. In addition, the form checking unit 7 can also classify the situation in which the input value example does not match by determining whether a command is detected from the text of the sound recognition result based on the result of the determination. However, the determination related to the detection of the command is not limited to the case where the determination result related to the conformity with the input value example is no. In addition, the form checking unit 7 can also check the input form by calculating the recognition rate or the false recognition rate of the sound recognition based on the number of determinations and the result of the determination. In addition, the form checking unit 7 can also notify the result obtained by checking the input form in a recognizable manner. As the result obtained by checking the input form, for example, the judgment result related to the input value example that matches, the judgment result related to the false detection of the command, the judgment result related to the false recognition other than it, the false recognition rate and other statistical information can be appropriately used. In addition, the form checking unit 7 can also delete or change the voice input command that matches the command according to the user's operation when the command is detected. That is, the form checking unit 7 can also resolve the conflict with the voice input command by deleting the voice input command or changing the language of the voice input command when a conflict occurs.

[0069] In either case, the format inspection unit 7 performs at least one of inspection based on guidance and inspection using voice synthesis.

[0070] Here, in the case of the form check based on the guidance, it is assumed that the guidance is set for the input process 3a. The form check unit 7 checks whether there is no inconsistency between the language of the guidance and the estimated input form. For example, if the language of the guidance is "○○ time" and the input form is not a time form, it can be known that there is inconsistency.

[0071] Figure 7An example of a rule table showing the conditions of the guidance language and the corresponding input form. In the guidance column, as the guidance conditions, a standard expression that matches a part of the guidance language is recorded. In the input form column, the content that the input form should satisfy when the guidance meets the conditions is recorded.

[0072] The form checking unit 7 checks whether the language of the set guidance matches the guidance in the rule table for each input process corresponding to the selected range. If there is a row of matching guidance, it checks whether the input form of the row matches the estimated input form. If not, the input process creator is notified of the fact.

[0073] In the case of form checking using speech synthesis, form checking unit 7 performs speech recognition on speech waveform data of input value examples generated based on the estimated input form to check whether an input value example whose speech is difficult to recognize is generated in the estimated input form.

[0074] In addition, the form checking unit 7 uses voice synthesis to convert the text of each input value example into voice waveform data for one input process. In addition, the form checking unit 7 can perform voice synthesis at any timing such as when a predetermined number of input value examples are generated or when a predetermined operation input by a user is accepted. In addition, the form checking unit 7 uses the voice recognition dictionary generated by the dictionary generating unit 8 for the corresponding input form to perform voice recognition on the voice waveform data obtained. At this time, the form checking unit 7 causes the voice recognition for recognizing commands used in the recognition control unit 11 described later to also operate simultaneously. In the case where the voice recognition for recognizing commands is different from the voice recognition for input values, the form checking unit 7 uses both of the two voice recognitions to recognize the generated input value examples. In the case where the voice recognition for input values ​​is used as the voice recognition for recognizing commands, the generated voice recognition dictionary also includes a dictionary for recognizing commands, and the dictionary for recognizing commands is used to perform voice recognition. In this way, the voice recognition result of the generated input value example can be obtained. Furthermore, the format checking unit 7 compares the voice recognition result with the generated input value example, and if all of them match, determines that the estimated input format is appropriate.

[0075] The dictionary generation unit 8 generates a voice recognition dictionary based on the estimated input form. For example, the dictionary generation unit 8 generates a voice recognition dictionary that can recognize the voice utterance of the input form according to the input form set in the input form column of the input process 3a. Here, the voice recognition dictionary records the grammar and can be uniquely generated according to the input form. In addition, the dictionary generation unit 8 also generates a dictionary for recognizing commands in parallel.

[0076] The sound recognition unit 9 is controlled by the form check unit 7 and the recognition control unit 11, and uses a sound recognition dictionary to convert the sound waveform data into text. For example, it is assumed that the sound recognition unit 9 uses sound recognition using a sound recognition dictionary that records a predetermined grammar in order to recognize the utterance used to input the sound. In this type of sound recognition, the sound waveform data of the utterance that complies with the grammar is recognized, and the sound waveform data of the utterance other than that is rejected. The format of the sound recognition dictionary depends on the sound recognition technology used, and can be either a grammar recorded in text or a binary dictionary. In addition, it can also be in the form of an object stored in a memory. In addition, the sound waveform data can be obtained based on the user's utterance or by synthesizing the sound of text.

[0077] In this embodiment, in addition to inputting items by voice, it is also possible to use voice to utter commands different from item input, such as restoring input, entering the next input process, and ending input, and execute the commands. At this time, in the form checking unit 7 and the recognition control unit 11, voice recognition for recognizing commands is also performed simultaneously.

[0078] Command recognition may use the same voice recognition as the above-mentioned voice recognition, or other voice recognition. For example, instead of using grammar voice recognition, voice recognition such as detecting the utterance of keywords included in the keyword list may be used. Voice recognition is not limited to these, and voice recognition for general spoken language may also be used. In this case, the voice recognition dictionary may include a word dictionary consisting of the expression and reading of the words to be recognized, and a grammar dictionary for determining whether to reject the recognition result.

[0079] Furthermore, when the voice recognition for recognizing the command is different from the voice recognition for inputting the item, the voice recognition unit 9 recognizes the generated input value example using both of the two voice recognitions. Furthermore, when the voice recognition for recognizing the command is the same as the voice recognition for inputting the item, the voice recognition unit 9 includes the dictionary for recognizing the command in the voice recognition dictionary and performs voice recognition using the dictionary.

[0080] The voice synthesis unit 10 is controlled by the form check unit 7 and the recognition control unit 11 to convert the text into voice waveform data. The voice synthesis unit 10 can play the text as voice from the speaker by providing the voice waveform data to the speaker. The voice synthesis unit 10 is used when checking the validity of the input form estimated by the form check unit 7, when the recognition control unit 11 plays the guidance at the beginning of each input process, and when repeating the input content.

[0081] The recognition control unit 11 performs voice recognition on the user's voice using a voice recognition dictionary, and inputs the text of the voice recognition result into the input field according to the identifier of the input field of the recognition input object item. For example, the recognition control unit 11 performs voice recognition on the user's voice according to the input process, and inputs the recognition result into the input field of the account data 2a. Alternatively, the recognition control unit 11 may perform voice recognition using a voice recognition dictionary generated according to the input form of the input process, and input the recognition result into the input field of the input object item of the input process.

[0082] For example, the recognition control unit 11 displays the account data on the display unit 12, and takes out an input process from the input process storage unit 3 according to the order of the input process. The recognition control unit 11 performs voice synthesis on the guidance content set in the input process through the voice synthesis unit 10. The recognition control unit 11 uses the voice recognition dictionary (and command recognition dictionary) generated according to the input form set in the input process, and performs voice recognition through the voice recognition unit 9. In the case of recognizing a command, the recognition control unit 11 executes the content of the corresponding command. In the case of recognizing something other than a command, the recognition control unit 11 performs voice synthesis on the voice recognition result and repeats it, and then compares the reference value set in the input process with the recognition result. If it is within the reference value, the recognition result is input into the input object item. If it is outside the reference value, the recognition control unit 11 notifies its meaning through voice synthesis and prompts re-input or forced input.

[0083] The display unit 12 displays data corresponding to the processing of the information processing device 1. For example, the display unit 12 displays the ledger data 2a and displays the estimated input form in the input field included in the ledger data 2a. In addition, for example, the display unit 12 may generate one or more input value examples based on the estimated input form and display the generated input value examples in the input field included in the ledger data 2a. In addition, for example, the display unit 12 may generate a range of values ​​that can be input into the input field based on the estimated input form and display the generated range in the input field included in the ledger data 2a. In addition, for example, the display unit 12 may display the ledger data 2a and, according to the input process 3a, display the order of input values ​​in a plurality of input fields included in the ledger data 2a in an overlapping manner.

[0084] Specifically, for example, the display unit 12 includes a microphone for recording the user's voice, a speaker for playing the sound, a display for displaying stored data and various information, and a control circuit for controlling the content displayed on the display. The display unit 12 displays the form data 2a on the display or the like. In addition, the display unit 12 may appropriately overlap and display the data in the input process 3a on the form data 2a. The display unit 12 may include a speaker and a display and be called an output unit, or may include a microphone and be called an input-output unit.

[0085] Next, use Figures 8 to 20 , the operation of the input procedure creation process and the operation of the voice input process performed by the information processing device configured as described above are described. In the following description, after the input procedure creator creates an input procedure list other than the input form on the partially inputted account sheet data, the information processing device 1 executes the input procedure creation process to complete the input procedure list. In addition, the information processing device 1 can perform the voice input process for the uninputted account sheet data by copying the completed input procedure list to the uninputted account sheet data.

[0086] (Input process production processing: Figure 8 )

[0087] In step ST1, the information processing device 1 creates an input procedure list as the input procedure 3a set except for the input form according to the operation of the input procedure creator. That is, regarding the input procedure 3a in step ST1, the input form is not set.

[0088] In step ST2, the range selection unit 5 determines whether there is any process in the input process 3a where the input form is not set. If not, the input process creation process is terminated. On the other hand, if the result of the determination in step ST2 is that there is any input form that is not set, the range selection unit 5 proceeds to step ST3.

[0089] In step ST3, the range selection unit 5 Fig. 9 As shown, according to the operation of the cursor cs by the input process creator, a range 5a including a plurality of input fields corresponding to one or more input target items is selected on the form data 2a, and the input process creator presses the displayed input form estimation button 5b. In addition, the range selection unit 5 may select a range including only one input field.

[0090] In step ST4, the input value example acquisition unit 4 extracts the process (O1, O2, . . . , O3) having the identifier of the input field within the selection range as the input target item from the input process list. n ). Where n represents the number of processes O in the selection range. Fig. 9 In the case of n ) are three rows (n=3) corresponding to the identifiers B1, C1, and D1 of the input fields indicated by the input target items in the input process 3a.

[0091] In step ST5, the input value example acquisition unit 4 acquires input value examples (E1, E2, . . . , E m ). Where m represents the number of input value examples in the selection range. Fig. 9 In the case of m ) are two times, "10:30" and "12:00" (m=2).

[0092] In step ST6, the form estimation unit 6 executes the process (O1, O2, ..., O n ), and its input value examples (E1, E2, ..., E m ) A form estimation process of estimating the input form. Based on the result of the form estimation process, the input form of the input process 3a is updated (set). The details of the form estimation process of step ST6 will be described later.

[0093] In step ST7, the format checking unit 7 performs format checking processing on the estimated and updated input format, determines whether notification is necessary or not, and obtains a misrecognition list of notifications if notification is necessary. Details of the format checking processing in step ST7 will be described later.

[0094] In step ST8, the format checking unit 7 determines whether notification of the format checking result is necessary, and if not, proceeds to step ST11. On the other hand, if notification is necessary as a result of the determination in step ST8, the process proceeds to step ST9.

[0095] In step ST9, the format check unit 7 notifies the display unit 12 of the incorrect recognition list as the format check result, if notification is necessary. The display unit 12 displays the format check result.

[0096] Here, use Fig.10 as well as Fig.11 , which explains an example of displaying a warning included in the form check results.

[0097] The result of the voice recognition of the input value example, in the case of a false detection of a command, becomes a situation where the command is mistakenly recognized when the input value example is spoken. In this case, for example, the input value example and the command are consistent, or the command recognition dictionary is used simultaneously for the input value example to recognize and obtain the command as the recognition result. In either case, the display unit 12 displays on the screen the meaning of "there is a possibility of false detection of the command" in correspondence with the input object item that caused the false detection, the input form (including the words in the word list in the case of "word"), and the input value example. This is particularly useful in the case of a "word" etc., because there is a possibility that the false detection of the command can be prevented by changing the input word.

[0098] Fig.10An example of a display in which a form check result indicating a command misdetection is notified in dialog box 7a is shown. In this example, words such as "no abnormality" and "abnormality" conflict with the command "above", and the command is notified that it is misdetected. In addition, the input object item that caused the misdetection and the estimated input form are also notified. The display method of the notification content is not limited to this, for example, the input object item that caused the misdetection may be highlighted.

[0099] exist Fig.10 At the bottom of the dialog box 7a, buttons bt1 to bt3 are displayed for "Change input format", "Change command", and "Keep as is". By operating any of the buttons bt1 to bt3, it is possible to choose how to deal with the possibility of command misdetection.

[0100] When the input procedure creator operates a button bt1 indicating "change input format", the display unit 12 moves the display to the corresponding input procedure portion of the procedure storage table, displays a dialog box for changing the input format, and prompts the user to change the input format.

[0101] In addition, when the input process creator operates the button bt2 indicating "change of command", the display unit 12 automatically changes the language of the erroneously detected command. For example, multiple candidates may be kept for one command, and when a false detection occurs, the candidate may be changed to a candidate selected from other candidates, or a language with similar meaning may be changed with reference to word2vec, a synonym dictionary, etc. When the input form or command is changed, the form check unit 7 performs a form check again to check whether the same problem occurs after the change. When the input process creator selects the button bt3 indicating "keep as is", the display unit 12 does nothing.

[0102] On the other hand, even if the above-described command is not misdetected, there is a possibility of misrecognition. For example, the pronunciation of the number "9" and the letter "Q" are both "きゅう", and the pronunciation cannot be distinguished. In addition, there are cases where it is difficult to recognize depending on the word.

[0103] Therefore, the form check unit 7 may also compare the sound recognition result with the generated input value example to calculate the sound recognition rate. Such an input value example may not necessarily be generated, so when generating the input value, it may be generated in a manner that includes examples such as "9" and "Q" that are likely to cause problems. In the case where the sound recognition rate is lower than a predetermined threshold, the input process creator is notified of this fact in the same manner as in the case of command misdetection. In addition, not only does it mean that the recognition rate is low, but based on the comparison of the sound recognition result and the generated input value example, the specific location that caused the misrecognition is also obtained, so it can also be notified at the same time as the details of the problem.

[0104] Fig.11 An example of a display in which a form check result indicating a reduction in recognition rate is notified in a dialog box 7b is shown. In this example, a button bt4 indicating "reselect range" is displayed instead of a button bt2 indicating "change command". In the case of a reduction in recognition rate, the input form changes by changing the selection range, and there is a possibility that the reduction in recognition rate is suppressed, so button bt4 is displayed. In this way, the form check unit 7 can save the effort of the voice input process producer to actually make a sound and confirm the action. In addition, by displaying a warning, the input process producer can be notified of the problem and prompted to make improvements.

[0105] In such a warning display, when the form check is performed once after all input forms are estimated, the selection range with problems is highlighted and the following information is displayed below. Fig.10 as well as Fig.11 The dialog boxes 7a and 7b shown are sufficient.

[0106] In addition, a form check may be performed each time the sound input process producer performs range selection and input form estimation. In this case, after the form check, if there is a problem, a warning may be displayed in the same manner. The warning notification method is not limited to the dialog box display such as the dialog boxes 7a and 7b, and any method may be used, such as displaying on the status bar, notifying with sound, etc.

[0107] Then, in step ST10, the display unit 12 processes the form check result according to the operation of the input procedure creator. In addition, the display unit 12 may also display the form check result according to the operation of the input procedure creator.

[0108] In step ST11, the display unit 12 displays the input form / input value example superimposed on the form data according to the operation of the input procedure creator. In the superimposed display, the input procedure creator confirms whether the set input form has no problem and whether there is no input process with an input form not set.

[0109] For example, the display unit 12 may display a menu or the like, and the user may select a function from the menu or the like to perform superimposed display.

[0110] Fig.12 1 is a diagram showing an example of the display of the input form in the process of superimposing input on the form data 2a. The display unit 12 displays a menu 12a on the upper part of the form data 2a. The input target items may be displayed in color, for example.

[0111] Regarding the input format, the content of the input process 3a in the input process storage unit 3 may be displayed as it is, or the content of the input process 3a may be displayed in a format that is easy for the input process creator to understand and view. Fig.12 In the example of , the input forms for time and numerical values ​​are displayed as they are, but the input form for words is displayed as a result of connecting a word list with "|" instead of a group name.

[0112] Fig.13 An example is shown in which an input value example is generated based on the input form and displayed superimposed on the form data 2a. By superimposing the input value example, it is possible to intuitively confirm what values ​​can be input compared to displaying the input form itself. In either case, the input object items of the input process that are not set in the input form are displayed in an easy-to-understand manner. Fig.13 In the example, the color of the item is changed and the icon is displayed.

[0113] In addition, by overlaying the display of guidance, input order, etc. in addition to the input form and input value examples, it is possible to confirm whether the entire input process is set correctly. Fig.14 as well as Fig.15 As shown, the sequence can be displayed either by number or by arrows.

[0114] After step ST11, the information processing device 1 returns to step ST2, and repeatedly executes steps ST2 to ST11 until all input formats are set.

[0115] Next, the form estimation process of step ST6 will be described in detail.

[0116] (Details of the form estimation process in step ST6)

[0117] The format estimation process changes depending on whether the input format is allowed or not to be linked.

[0118] (Situations where links are not allowed: Fig.16 )

[0119] Fig.16 A flowchart of the form estimation process when there is no connection is shown.

[0120] In steps ST601 to ST603, the form estimation unit 6 estimates the input value examples (E1, E2, . . . , E m ) Each guess and input value example E i The large category G of suitable input forms i (i≤m).

[0121] For example, in the input value example E i For example, if "-12.34" consists only of a numeric value, a decimal point, and a sign, it can be seen that it is a numeric value. Fig.17An example of such a rule for large-category assignment is shown. In this rule example, the standard expression is recorded in the condition. In the order of priority from high to low in such a table, in the input value example E i If the standard performance of the conditions is met, it can be regarded as the corresponding major category G i .

[0122] If none of them match, enter the value E i The large category G is assigned to represent the bottom "word" i For example, in the input value example E i In the case of "2023 / 03 / 07", the standard expression of the condition matches "\d\d\d\d / \d?\d / \d?\d", so it can be known as "date". The method of assigning to a large category is not limited to this, and the rule is not limited to this condition. In addition, the same process can be rewritten as a program without having such a rule.

[0123] In step ST604, when the input format of OR is not permitted, the format estimation unit 6 proceeds to step ST605. Whether the input format of OR is permitted is determined in advance.

[0124] In step ST605, the format estimation unit 6 selects the input format from the major categories (G1, G2, . . . , G m ) selects one major category G. The form estimation unit 6 stores the input value examples that match the selected major category G in a memory such as the data storage unit 2.

[0125] In addition, there are multiple input values ​​in the selection range (E1, E2, ..., E m ), there are cases where there are multiple input value examples corresponding to multiple items, and there are cases where there are multiple input value examples corresponding to one input object item. In either case, it is necessary to classify the input format corresponding to each input value example (G1, G2, ..., G m )Merge and select 1 large category G.

[0126] The combination of input forms may include all input forms. Alternatively, the narrowest input form may be adopted, and input value examples that do not conform to the adopted input form may be treated as illegal examples.

[0127] When the input form is determined in an all-inclusive manner, if the major categories are the same, the details of the input form are merged. When the major categories are different, the input value examples that match the most major categories and do not match the major categories match the selected major category, and the input form is re-estimated.

[0128] For example, regarding the input value example "10:30", if only this is observed, it is a "time", but it can also be regarded as an "alphanumeric", so when the most common major category is "alphanumeric", it can be regarded as a detailed form after inference as "alphanumeric". In the case where the major category does not match and cannot be re-inferred, one major category G can be selected and the input value example that does not match it can be treated as an illegal example. Alternatively, when the OR of the input form is also used as the input form, the result connected by OR can also be used as the inference result of the major category.

[0129] In step ST606, the form estimation unit 6 estimates the input value examples (E1, E2, . . . , E m ) each infers and combines the details of the input form.

[0130] In addition, if the major category G is determined, the details of the input form are estimated using the corresponding input value example for each major category G. For example, if the major category G is "numeric value (decimal number)", it is determined whether there are details such as +, - signs, the number of characters before the decimal point (i.e., the number of integer parts), and the number of characters after the decimal point (i.e., the number of decimal parts).

[0131] Furthermore, if the major category G is "alphanumeric", each character is replaced with "A" if it is an English letter, replaced with "D" if it is a number, and replaced with "S" if it is another symbol, thereby obtaining an alphanumeric pattern.

[0132] When the major category G is "word", the input value example is set as the expression of the word. Figure 4 The word list 2b shown in the figure adds a new group, sets a new name in the group name, and sets an input value example in the word expression. Through the above method, the content of reading and reading aloud is automatically generated and set. The detailed input form is set to "word_(group name)".

[0133] In step ST607, the format estimation unit 6 calculates the input value examples (E1, E2, . . . , E m ) to adjust the details of the input form. Thus, one input form can be obtained.

[0134] In addition, the details estimated for each input value example are integrated for each major category. For example, the details of the input form when the input form is determined to be all-inclusive are integrated as follows.

[0135] In the case of numerical values, for example, if the major category is "Numerical value (decimal number)", and the estimated individual input forms are "Decimal number_3_2 (3 integer digits, 2 decimal digits)" and "Decimal number_3_1 (3 integer digits, 1 decimal digit)", the decimal part can be either 1 or 2 digits. Therefore, the combined input form becomes "Decimal number_3_1~2 (3 integer digits, 1~2 decimal digits)".

[0136] In the case of alphanumeric, for example, if the major category is "alphanumeric" and the estimated individual input forms are "alphanumeric_AAADDSA" and "alphanumeric_AADDDS", the third character can be either an English letter or a number, and the last English letter can be omitted. Therefore, the combined input form becomes "alphanumeric_AA$DDSA?".

[0137] In the case of words, for example, if the major category is "words" and the inferred individual input forms are "word_1" and "word_2", a new group is created by merging the groups of the word list, and the input form of the group is used. That is, a new group "3" is created by merging the word list of group "1" and the word list of group "2", and all the words of "1" and "2" are added to the word group "3". Moreover, "word_3" is used as the merged form. At this time, in the case of no reference to the input forms of the word groups "1" and "2", the words of groups "1" and "2" are removed from word list 2b.

[0138] By combining in this way, the details of the final input form can be obtained. In addition, the combining method is not limited to this, and a majority decision can also be adopted to select one of the most input forms.

[0139] On the other hand, in step ST604 , when the OR of the input format is permitted, the format estimation unit 6 proceeds to step ST608 .

[0140] In step ST608, the form estimation unit 6 calculates the OR of each major category. For example, the form estimation unit 6 calculates the OR of each major category (G1, G2, ..., G m ) uniqueness (remove duplication), and use OR to connect the obtained large categories to obtain the OR large category G1|G2|…|G k (k≤m). In addition, the form estimation unit 6 classifies the major categories G1|G2|…|G k The major categories of G j The corresponding input value examples are stored in a memory such as the data storage unit 2 (j≤k).

[0141] After step ST608, the form estimation unit 6 calculates the value of each major category G of the element to be OR. jThe loop process is executed from step ST609 to step ST613. The loop process includes the same steps ST610 to ST611 as described above, and step ST612.

[0142] In step ST610, the form estimation unit 6 determines the major category G j The details of the input form are determined according to the corresponding input value example. In addition, step ST610 is performed in the same manner as step ST606.

[0143] In step ST611, the form estimation unit 6 adjusts the details of the input form according to the number of input value examples used. Step ST611 is performed in the same manner as step ST607.

[0144] In step ST612, the form estimation unit 6 replaces the large category G corresponding to the input value example with the details of the input form. j . Thus, we can get the j Corresponding to 1 input form. For each large category G j , execute the above steps ST609 to ST613 in a loop.

[0145] After step ST607 or ST613, in step ST614, the form estimation unit 6 calculates each process (O1, O2, ..., O n )'s input format is updated (set) to the obtained input format.

[0146] Furthermore, when the range selected by the user in the range selection unit 5 is small and the input value examples are small, there is a possibility that the input form becomes too detailed when the input form is determined only by the input value examples. As described above, there is a possibility that the convenience of the user who performs voice input is impaired by an overly detailed input form. For example, when only one input value example "12.34" is provided, the input form becomes "decimal number_2_2".

[0147] However, there is a possibility that the voice input person may omit the "0" of "12.30" and speak a numerical value such as "12.3". The input value example may include "12.30" and "12.3", but is not limited to this. In addition, on the contrary, the input value example only provides "12.3", but in fact, it is sometimes necessary to input "12.30".

[0148] In order to accommodate such examples, when there are few input values ​​within the selection range, the input format can be relaxed. That is, if it is a decimal number, it also includes numbers with one digit less and numbers with one digit more ("decimal number_M_N" → "decimal number_(M-1) to (M+1)_(N-1) to (N+1)", etc.), and if it is an alphanumeric number, the pattern part of only English letters can be replaced with a pattern of English letters or numbers ("alphanumeric_AAA" → "alphanumeric_$$$", etc.).

[0149] When the input form is estimated as described above, the input form of the currently focused input process (the input process that uses the range-selected item as the input target item) is updated.

[0150] Thus, the form estimation process of step ST6 in the case where the connection is not permitted ends.

[0151] (Links are allowed: Fig.18 )

[0152] Fig.18 A flowchart of the form estimation process when there is a connection is shown.

[0153] In steps ST621 to ST624, the form estimation unit 6 determines the input value examples (E1, E2, . . . , E m ) each performs a loop process. The loop process includes steps ST622 to ST623.

[0154] In step ST622, the format estimation unit 6 divides the input value example E by spaces. i , set as input value Example E i The parts e1, e2, ..., e il The suffix "il" indicates that the input value is included. Example E i For example, for the input value E i When the division is performed into three, il=3.

[0155] In step ST623, the form estimation unit 6 calculates the input value example E. i The parts e1, e2, ..., e il Each, select the appropriate category G1 i , G2 i , …, G il i .

[0156] In addition, when not only a simple "number" or "word" but also a combination (concatenation, OR) of these is allowed as an input format, the input values ​​are first separated by spaces. Example E i , with respect to its parts e1, e2, ..., eil , search for the matching category G1 i , G2 i , …, G il i .

[0157] For example, if the input value is E i If it is "2023 / 03 / 07 Tuesday 13:00", then the part e1 of "2023 / 03 / 07" and the large category G1 of "Date" i It matches the part e2 of "Tuesday" and the big category G2 of "Words" i In line with the "13:00" part il The big category of "moment" il i In the case where there is only one input value example, the macro-category with the highest priority among the multiple macro-categories can be selected, and then the detailed form can be estimated for each macro-category in the same manner as described above.

[0158] In step ST624, the form estimation unit 6 connects the selected major category G1 i , G2 i , …, G il i The result is added to the list of major category links. For example, "date, word, time" formed by linking "date", "word", and "time" is added to the list of major category links. i , G2 i , …, G il i Example E of the input value of the divided part i From "date", we get part e1 of "2023 / 03 / 07", from "word", we get part e2 of "Tuesday", and from "time", we get part e of "13:00" il In addition, "major category links" do not necessarily need to be linked, so it can also be called "combination of major categories."

[0159] The form estimation unit 6 repeats the above process to input the value example E i The loop processing of steps ST621 to ST624 is performed in a manner according to the amount.

[0160] Thereafter, in steps ST625 to ST629, when there are two or more links in the list of major category links, the form estimation unit 6 performs a loop process. This loop process includes steps ST626 to ST628.

[0161] For example, when there are multiple input value examples, the content of the major category link may be different between the input value examples. For example, when the input value examples are "A12C black 12.3", "A12M white 45.6", and "B23F red", the candidates for the major category link are "hexadecimal number, word, decimal number", "alphanumeric, word, decimal number", and "hexadecimal number, word". In contrast, the major category is selected so that as many as possible are matched.

[0162] In step ST626, the form estimation unit 6 extracts two major classification links from the list of major classification links. i , G2 i , …, G il i )、(G1 j , G2 j , …, G jl j ), applying the difference algorithm.

[0163] It is possible to take two of the candidates from the list of major classification connections, regard it as a calculation problem of differences including OR in the partial column, and use an existing difference algorithm to determine the arrangement of the major classification. In addition, as a difference algorithm, for example, an algorithm that finds the difference between arrangements by calculating the edit distance, LCS (Longest Common Subsequence), SES (Shortest Edit Script) used in diff, etc. can be appropriately used.

[0164] As a result of the difference algorithm, one of "same", "addition", "deletion" and "replacement" can be obtained for each character. The difference algorithm is used with each major category (or their OR) as characters, and the example with the smallest edit distance is selected.

[0165] Regarding the replacement cost between major categories, if one is included in the other, it is set to 0 (that is, it is regarded as the same, "decimal number|alphanumeric" and "alphanumeric", "hexadecimal number" and "alphanumeric", etc.), if not included, it is set to 1.

[0166] In addition, when a difference algorithm is applied to the two major category connections "hexadecimal numbers, words, decimal numbers" and "alphanumeric numbers, words, decimal numbers", "permutation of hexadecimal numbers and alphanumeric numbers, words, decimal numbers" can be obtained as difference information.

[0167] In step ST627, the form estimation unit 6 deletes the two extracted major classification links (G1 i , G2 i, …, G il i )、(G1 j , G2 j , …, G jl j ).

[0168] In step ST628, the form estimation unit 6 absorbs the difference using the difference information obtained as a result of applying the difference algorithm, and creates a link (G1 i，j , G2 i，j , …, G i，jl i，j ).

[0169] For example, the form estimation unit 6 absorbs the difference of the above-mentioned differential information "permutation of hexadecimal numbers and alphanumerics, words, decimal numbers", and converts the "permutation", "addition", and "deletion" parts into a combination of large categories. For example, in "permutation of hexadecimal numbers and alphanumerics, words, decimal numbers", when the part of the inclusion relationship is modified to the wider side with respect to the permutation part, it becomes "alphanumerics, words, decimal numbers". In this way, "permutation of ○○ and ×× (○○ is included in ××)" with an inclusion relationship can be set to "××".

[0170] When "permutation of ○○ and ××" without an inclusion relationship occurs, it may be set to OR, that is, "○○|××".

[0171] In addition, "addition of ○○" and "deletion of ××" may be set as selectable large categories, namely, "(○○)" and "(××)".

[0172] At this time, which part of which input value example corresponds to which major category is recorded in advance.

[0173] Then, the form estimation unit 6 connects the created large categories (G1 i，j , G2 i，j , …, G i，jl i，j ) to the list of major category links.

[0174] The form estimation unit 6 executes the loop processing of steps ST625 to ST629 so as to repeat the above processing until only one link of the large category remains.

[0175] Then, in steps ST630 to ST633, the form estimation unit 6 determines the connection (G1 i , G2 i , …, G il i ) in each major category G i, in order to obtain the detailed information of the input form, a loop process is performed. The loop process includes steps ST631 to ST632.

[0176] In step ST631, the form estimation unit 6 determines the i The corresponding input value example Ei is a partial set (e i1 , e i2 ,…,e im ), determine the details of the input form.

[0177] In the above example, the remaining large category G i The connection becomes "alphanumeric, word, (decimal number)". Here, the large category G with "alphanumeric" i Example E of the input value that meets the requirements i It is "A12C", "A12M", "B23F", and the large category of "words" G i Example E of the input value that meets the requirements i It is the major classification of "black", "white", and "decimal numbers" G i Example E of the input value that meets the requirements i It is "12.3" and "45.6". Later, for each major category G i , according to the input value example E that matches it i The detailed format may be estimated. In the above example, the final format is "alphanumeric_ADDA, word_1, (decimal number_2_1)", and the word list of word_1 is "black" and "white".

[0178] In step ST632, the form estimation unit 6 uses the input value example E i The set of parts (e i1 , e i2 ,…,e im ) (the input value used is E i Then, the form estimation unit 6 replaces the large classification G with the obtained details. i .

[0179] The form estimation unit 6 uses the following method to estimate the form of each major category G: i The loop processing of steps ST630 to ST633 is performed in a manner of each repetition of the above processing.

[0180] In step ST634, the form estimation unit 6 sets the input form obtained as described above as the input form of the corresponding multiple input processes. In addition, in step ST634, when there are few input value examples, the input form can be relaxed in the same way as step ST614. In any case, through the above, the form estimation processing of step ST6 in the case of allowing connection is completed.

[0181] (Formal inspection process)

[0182] In the following form check processing, form check using voice synthesis is described. In the form check processing, as described in the form check unit 7, an input value example is generated based on the input form and voice data is generated by voice synthesis, and voice recognition is performed using a voice recognition dictionary generated based on the input form, and a form check is performed based on whether misrecognition occurs.

[0183] Fig.19 A flow chart of the form checking process is shown.

[0184] In step ST701 , the form checking unit 7 generates a speech recognition dictionary through the dictionary generating unit 8 based on the input form of interest.

[0185] In step ST702, the form checking unit 7 generates a plurality of input value examples according to the inferred input form. For example, in the part of the input value example that can be generated according to the input form, a plurality of candidates can be randomly selected. For example, in the case of "numeric value", if the input form is "decimal number_2~4_1", the number of digits of the integer part of the input value example is randomly selected from 2 to 4, and the digits of each digit are randomly determined to be 0 to 9 (but the leading digits are 1 to 9).

[0186] In the case of "alphanumeric", if the input format is "alphanumeric_AA$D?", one character is selected from "A-Za-z" for the first and second characters of the input value example, one character is selected from "A-Za-z0-9" for the third character, and whether the fourth character is included or not is randomly determined. When the fourth character is included in the input value example, one character is selected from "0-9". Even for other major categories, multiple input value examples can be generated using random numbers in the same way.

[0187] In step ST703, the form check unit 7 creates an empty misrecognition list. The empty misrecognition list is a list including the type of misrecognition, input value examples, and detailed information. As the type of misrecognition, for example, command misdetection and misrecognition can be appropriately used. As detailed information, for example, when the type is command misdetection, the command can be used. In addition, for example, when the type is misrecognition, as detailed information, for example, the recognition result and the error position can be appropriately used.

[0188] Thereafter, in steps ST704 to ST711 , the format check unit 7 executes a loop process to perform format checks on each of the generated input value examples by voice recognition. This loop process includes steps ST705 to ST710 .

[0189] In step ST705 , the format check unit 7 performs speech synthesis for each of the generated input value examples, and obtains speech data as a result of the speech synthesis.

[0190] In step ST706, the form check unit 7 uses the generated voice recognition dictionary to perform voice recognition on the voice data of the voice synthesis result. At this time, the form check unit 7 uses the voice recognition dictionary to perform voice recognition of the input value example and the voice recognition of the command. As the voice recognition dictionary, a first dictionary related to the input value example and a second dictionary related to the command may be used respectively. When the first and second dictionaries are used, the form check unit 7 performs voice recognition of the input value example and the voice recognition of the command respectively. Alternatively, as the voice recognition dictionary, a dictionary related to both the input value example and the command may be used. When the dictionary related to both is used, the form check unit 7 performs voice recognition of the input value example and the voice recognition of the command at the same time. In addition, as the recognition result of step ST706, there are three cases: (1) the case where the input value example is recognized, (2) the case where the input value example is not recognized and the command is mistakenly detected, and (3) the case where the command is mistakenly detected. In addition, in the above case (1), the voice recognition is successful. In the above case (2) or (3), the voice recognition fails. In addition, when the command of (2) described above is erroneously detected, that is, when a command is obtained as a recognition result, it means that a command is recognized by mistakenly believing that an utterance corresponding to the input value example is performed.

[0191] In step ST707, the form checking unit 7 determines whether the text of the voice recognition result matches the input value example. If they match, the voice recognition is successful, so the process proceeds to step ST711. Otherwise, if the result of the determination is no, the voice recognition fails, so the process proceeds to step ST708.

[0192] In step ST708, the form checking unit 7 determines whether the reason for the failure of voice recognition is the false detection of a command, and if so, moves to step ST709, and if not, moves to step ST710. According to step ST708, when the voice recognition fails in step ST707, the situation is classified as to whether the failure is caused by the false detection of a command.

[0193] In step ST709, the form inspection unit 7 recognizes a command as a result of the voice recognition, and therefore adds information on command misdetection to the misrecognition list, and the process proceeds to step ST711.

[0194] In step ST710, since the recognition result of the voice recognition is an erroneous recognition other than command erroneous detection, the format inspection unit 7 adds information on the erroneous recognition to the erroneous recognition list, and the process proceeds to step ST711.

[0195] In step ST711 , the format checking unit 7 performs the processing of steps ST705 to ST710 on all the generated input value examples, and then ends the loop processing of steps ST704 to ST711 .

[0196] In step ST712, the format check unit 7 determines whether there is information of command misdetection in the misrecognition list, and if there is information of command misdetection, the process proceeds to step ST714, and if not, the process proceeds to step ST713.

[0197] In step ST713, the form checking unit 7 calculates the error recognition rate by dividing the number of erroneous recognitions in the erroneous recognition list by the number of input value examples, and determines whether the error recognition rate is greater than a threshold value. Based on the result of the determination, the form checking unit 7 transfers to step ST714 if the error recognition rate is greater than the threshold value, and transfers to step ST715 if not.

[0198] In step ST714, the format check unit 7 determines that notification of the format check result is necessary because there is information of command erroneous detection or the erroneous recognition rate is high, and notifies the display unit 12 of the erroneous recognition list corresponding to the format check result and ends the processing.

[0199] In step ST715 , the format checking unit 7 determines that there is no need to notify the format checking result because the erroneous recognition list is empty or the erroneous recognition rate is low, and ends the processing.

[0200] (Sound input processing)

[0201] The information processing device 1 uses the input process 3a obtained through the input process production process, together with other voice input personnel, such as Fig. 20 As shown, sound input processing is performed.

[0202] In step ST21 , the recognition control unit 11 of the information processing device 1 adds a flag indicating whether the input process 3 a has been executed (=execution flag) for each process number, and sets all execution flags to off.

[0203] In step ST22, the recognition control unit 11 determines whether the input process 3a is in which the implementation flag of the process number is turned off. If not, the voice input processing ends. On the other hand, as a result of the determination in step ST22, if the process number with the implementation flag turned off remains in the input process 3a, the recognition control unit 11 moves to step ST23.

[0204] In step ST23 , the recognition control unit 11 extracts, from the input process 3 a , a process corresponding to a row of the first process number among the process numbers whose implementation flag is turned off.

[0205] In step ST24 , the dictionary creation unit 8 creates a voice recognition dictionary based on the input format set during the extraction process under the control of the recognition control unit 11 .

[0206] In step ST25 , the voice synthesizing unit 10 performs voice synthesis on the guidance included in the fetching process under the control of the recognition control unit 11 , and reproduces the obtained voice data from the speaker.

[0207] In step ST26, the voice recognition unit 9 waits for the voice input person to utter a word under the control of the recognition control unit 11. At this time, the voice input person utters a word as appropriate.

[0208] In step ST27 , the voice recognition unit 9 performs voice recognition on the utterance of the voice input person using the voice recognition dictionary under the control of the recognition control unit 11 .

[0209] In step ST28 , the recognition control unit 11 determines whether a command is recognized based on the result of the voice recognition, and if a command is recognized, the process proceeds to step ST29 , and if not, the process proceeds to step ST30 .

[0210] In step ST29 , the control unit 11 recognizes that the recognized command has been executed, and the process returns to step ST22 .

[0211] In step ST30, the recognition control unit 11 performs voice synthesis on the text of the recognition result that does not include the command, and plays the obtained voice data through the speaker, thereby repeating the utterance of the person who input the voice through the speaker.

[0212] In step ST31 , the recognition control unit 11 inputs the value represented by the text of the recognition result into the input field of the document data 2 a based on the identifier of the input target item included in the extraction process.

[0213] In step ST32 , the recognition control unit 11 turns on the execution flag of the process and returns to step ST22 .

[0214] Thereafter, the recognition control unit 11 executes all the processes included in the list of the input process 3 a until all the execution flags are turned on.

[0215] In addition, the above voice input processing is an example, and a voice recognition dictionary is generated each time a process with a process number is implemented, but it is not limited to this. For example, a voice recognition dictionary can also be generated initially, and the necessary dictionary can be taken out and used during voice recognition. In addition, there can be no repetition. Alternatively, it can also include obtaining a confirmation such as "Is this OK?" from the system for the voice input person after repeating the recognition result, and executing the input only if it is OK, and re-executing the process from the utterance of step ST26 if it is not OK. In this case, it is possible to prevent the input of an erroneous value caused by misrecognition.

[0216] As described above, according to the first embodiment, the input value example acquisition unit 4 acquires one or more items and input value examples as information on the value of the input field for one or more items from the form data 2a including the input field for voice input for each item. The form estimation unit 6 estimates the input form of the input field based on the input value examples. In this way, by estimating the input form of the input field based on the input value examples of the input field, even in the absence of an expert in voice recognition, an appropriate input form can be determined.

[0217] In addition, in the past, the form of the value that can be input in advance was determined in advance, and the sound recognition was performed based on it. Generally speaking, by performing sound recognition based on the input form, it is possible to reject the recognition results of the sound other than the set input form and the recognition results recognized in noise, and the accuracy of sound recognition can be improved. Therefore, it is important to set an appropriate input form.

[0218] However, conventionally, it is difficult for a worker other than an expert in voice recognition to determine an appropriate input format, and setting an appropriate input format for all input items takes a lot of time.

[0219] In contrast, according to the first embodiment, as described above, even when an expert in voice recognition is not present, an appropriate input format can be determined. In addition, the information related to the value of the input field is not limited to the input value example, and can also be set to the format setting described later. Even if implemented in this way, the same effect as the above effect can be obtained.

[0220] In addition, according to the first embodiment, an input process 3a for inputting values ​​in a plurality of input fields included in the account data 2a is maintained in the input process storage unit 3. The input process 3a includes an identifier for identifying an input field for an input object item included in the account data 2a, and an inferred input form. Therefore, in addition to the above-mentioned effects, the input process 3a can be managed independently of the account data 2a. In addition, the input process 3a is not limited to the memory of the information processing device 1, and can be maintained in a database device that can communicate with the information processing device 1, or in other tables of the account data 2a. In addition, the input process 3a can also appropriately include a process number, a guide, a reference value, a process group name, etc. in addition to the identifier and the input form. Even if it is deformed in this way, the input process 3a can be managed independently of the account data 2a in the same way as described above.

[0221] According to the first embodiment, the range selection unit 5 selects a range of a plurality of input fields from the form data 2a. The input value example acquisition unit 4 acquires input value examples corresponding to the input fields of the selected range. The form estimation unit 6 estimates the input form common to the input fields of the range based on the input value examples acquired correspondingly. Therefore, in addition to the above-mentioned effects, by selecting a range of a plurality of input fields, the effort of acquiring input value examples can be reduced compared to the case of selecting each input field.

[0222] In addition, according to the first embodiment, the dictionary generation unit 8 generates a voice recognition dictionary based on the estimated input form. The recognition control unit 11 performs voice recognition for the user's utterance using the voice recognition dictionary, and inputs the text of the obtained voice recognition result into the input field according to the identifier of the input field. Therefore, in addition to the above-mentioned effects, voice input for the user's utterance can also be performed.

[0223] According to the first embodiment, the information related to the value of the input field is an input value example that is an example of the value of the input field. The input value example acquisition unit 4 acquires the value as the input value example from at least one of the form data 2a in which the value of the input field is input in advance and the user interface that accepts the input of the value to the input field. Therefore, in addition to the above-mentioned effects, the input value example can be acquired from various acquisition sources.

[0224] In addition, according to the first embodiment, the form estimation unit 6 may also estimate the input form for each input field in the range, and estimate one of the estimated input forms as a common input form. In this case, in addition to the above-mentioned effects, for example, when one of the multiple input forms includes another, the input forms can be merged by adopting the narrowest input form.

[0225] In addition, according to the first embodiment, the form estimation unit 6 may also estimate the input form for each input field in the range, and estimate a plurality of input forms included in the estimated input form as a common input form. In this case, in addition to the above-mentioned effects, for example, by determining the input form in a manner that includes all of the plurality of input forms, the input forms can be merged.

[0226] In addition, according to the first embodiment, the form estimation unit 6 adjusts the estimated input form according to the number of input value examples obtained accordingly. Therefore, in addition to the above-mentioned effect, when the range of selection (number of units) is small, the input form can be slightly relaxed by increasing the number of digits of the input form (allowing +1 digit), etc., so as to maintain the convenience of the user.

[0227] In addition, according to the first embodiment, the input process 3a may also include upper and lower limits (reference values) indicating the range of the value of the input field. The form estimation unit 6 may also adjust the estimated input form based on the upper and lower limits included in the input process 3a. In this case, it is possible to implement adjustments such as slightly modifying the input form in a manner that satisfies the conditions of the upper and lower limits, so that a more appropriate input form can be estimated.

[0228] In addition, according to the first embodiment, the input process 3a also includes guidance related to voice input to the input field. The form checking unit 7 checks the estimated input form based on the guidance. Therefore, in addition to the above-mentioned effects, by using the structure of checking the estimated input form using the guidance, the accuracy of the estimated input form can be improved. For example, it is possible to check an inappropriate input form such as a case where the language of the guidance is "○○ time" but the input form is not a time form. In addition, for example, the input form to the input field of multiple items is estimated based on the input value example, and the appropriateness of the input form is checked, so even if you are not an expert in voice recognition, you can set an appropriate input form, and the time spent on input process creation can be shortened compared to manually studying the input form.

[0229] In addition, without limitation to this, the form checking unit 7 may also check the input form by using rules for form checking. Alternatively, the form checking unit 7 may also use existing data to make the machine learning model perform machine learning and install the obtained learned model. Here, as the existing data, for example, a data set including input data including input value examples and input forms inferred from the input value examples, and output data including the check result of the inferred input form can be used. Even with such a deformation, the inferred input form can be checked, so the accuracy of the inferred input form can be improved.

[0230] In addition, according to the first embodiment, the form checking unit 7 generates an input value example based on the estimated input form, performs voice synthesis on the generated input value example, and performs voice recognition on the voice data obtained by voice synthesis using a voice recognition dictionary. The form checking unit 7 checks the estimated input form by determining whether the text of the obtained voice recognition result and the generated input value example match. Therefore, in addition to the above-mentioned effect, by using the structure of checking the estimated input form by voice recognition, the accuracy of the estimated input form can be improved.

[0231] According to the first embodiment, the form checking unit 7 may classify the situation when it does not match by judging whether a command is detected from the text of the voice recognition result based on the judgment result. In this case, the validity of the input form can be checked by confirming that there is no erroneous detection of the command.

[0232] In addition, according to the first embodiment, the form checking unit 7 checks the input form by calculating the recognition rate or the false recognition rate of the voice recognition based on the number of determinations and the result of the determination. For example, the false recognition rate can be calculated by dividing the result determined as false recognition by the number of times the number of input value examples is determined. Alternatively, the recognition rate can be calculated by dividing the result determined as successful recognition by the number of input value examples, that is, the number of determinations. In this case, in addition to the above-mentioned effects, the appropriateness of the input form can also be statistically checked.

[0233] In addition, according to the first embodiment, the form checking unit 7 can identifiably notify the result of checking the input form. Therefore, in addition to the above-mentioned effects, by notifying whether the situation of failed voice recognition is caused by misdetection of the command, it is possible to prompt the user to respond appropriately according to the situation of failed voice recognition. In addition, without limitation to this, the form checking unit 7 can notify the meaning of the possibility of reduced recognition rate, and can also notify the main reasons for the reduced recognition rate (conflict with commands, processes with low recognition rate, characters that are easy to make mistakes, etc.). For example, examples of homophones such as "9" and "Q" can also be notified. In this way, it is also possible to prompt the user to respond appropriately according to the situation of failed voice recognition.

[0234] In addition, according to the first embodiment, the form checking unit 7 may change the voice input command to match the command when a command is detected. In this case, the conflict with the command can be eliminated by changing the language of the command, so that unexpected situations caused by erroneous detection of the command can be avoided.

[0235] Furthermore, according to the first embodiment, the display unit 12 may display the form data 2a and display the estimated input form in the input field included in the form data 2a. Similarly, the display unit 12 may generate one or more input value examples based on the estimated input form and display the generated input value examples in the input field included in the form data 2a. Similarly, the display unit 12 may generate a range of values ​​that can be input into the input field based on the estimated input form and display the generated range in the input field included in the form data 2a. Therefore, in addition to the above-mentioned effects, the estimated input form and the input value examples generated based on the input form are displayed on the form data 2a, so that the input position and the input form / input value example on the form data 2a can be checked at a glance. In conjunction with this, the estimated input form can be displayed together with the form to improve the overview, and the input form can be automatically checked.

[0236] In addition, in the past, even after the input form is determined, the input form and the input object item are displayed far apart, so the overview is low and it is difficult to observe the input form to determine whether the set input form is really appropriate. Therefore, in the past, in order to check whether the input form is appropriate, it is necessary to actually try to input, which takes time.

[0237] In contrast, according to the first embodiment, for example, Fig.12 As shown, the input form and the input target items are displayed close to each other, so the overview is high and it is easy to judge whether the set input form is appropriate by looking at the input form.

[0238] In addition, according to the first embodiment, the input process 3a includes an identifier for identifying an input field of an input object item included in the account data 2a, an estimated input form, and an order (process number) of input values ​​in a plurality of input fields included in the account data 2a. The display unit 12 displays the account data 2a, and in accordance with the input process 3a, the order of input values ​​is displayed in a superimposed manner in a plurality of input fields included in the account data 2a. Therefore, in addition to the above-mentioned effects, the user can visually recognize whether the order of input is correctly set. In addition, without limitation to this, the display unit 12 can also display the information (guidance, reference value) in the input process 3a in a superimposed manner on the account data 2a. Thus, as described above, the user can visually recognize whether the information (guidance, reference value) in the input process 3a is correctly set.

[0239] <Second Embodiment>

[0240] In the first embodiment, the range selection unit 5 selects a range of estimated input forms on the form data 2a according to the operation of the input process creator. In contrast, in the second embodiment, the range selection unit 5 automatically selects a range, estimates the input form of the selected range, and automatically performs all actions set in the input process 3a. That is, unlike the first embodiment, the range selection unit 5 automatically estimates a plurality of ranges to which the same input form should be set.

[0241] Along with this, the range selection unit 5 is as follows Fig.21 As shown, in addition to the above-mentioned functions, at least one of range selection using the guidance in the input process 3a and range selection using the position of the input object item in the input process 3a is performed.

[0242] The range selection unit 5 selects the range of the input field according to the guidance, for example, by selecting the range of the input field corresponding to the guidance in the same language. Specifically, the range selection unit 5 may set the range of the input field corresponding to the same guidance included in the input process 3a as one range.

[0243] In the case of using the range selection of the item position, the range selection unit 5 selects the range of the input field according to the situation that the identifier of the input field of the input object item indicates a close position. Specifically, the range selection unit 5 may also select the range of the input field identified by the identifier as one range when the identifier of the input object item included in the input process 3a indicates an adjacent input field. However, it is preferred that an input value example is obtained from the input field of the input object item, and the input form is estimated by the form estimation unit 6 for one input value example, and the range of the input field is included only when there are more than N adjacent input fields with the same input form (N is a predetermined threshold).

[0244] exist Fig. 22 , the table data 2a, input value examples, a list of input procedures, and ranges selected based on these are shown. For example, B1-D1, B2-D2, ..., B5-D5 have the same guidance ("inspection time", "starting current", ..., "appearance"), so they are set as one range rg1, rg2, ..., rg5. As for B8-D9, B8-B9, C8-C9, D8-D9 have the same guidance ("noise, position 1", ..., "noise, position 3"), so they are set as one range rg81, ..., rg83 as shown by the dotted line. Furthermore, the item whose input format is decimal number_2_1, which is inferred from the input value example that has been entered, is adjacent to B8-D8, so when all are combined, B8-D9 becomes one range rg89 as shown by the single-dot chain line.

[0245] The other structures are the same as those of the first embodiment.

[0246] Next, use Fig.23 The flowchart illustrates the operation of the information processing device constructed as described above.

[0247] Now, in the same manner as described above, step ST1 of creating an input process other than the input form is executed.

[0248] After step ST1, in step ST3a, the range selection unit 5 selects the range of the input field, for example, based on the guidance. In addition, the range selection unit 5 selects the range of the input field, for example, based on the identifier of the input object item included in the input process 3a. Specifically, for example, when the identifier of the input object item indicates an adjacent input field, the range selection unit 5 selects the range of the adjacent input field. At this time, the input object item corresponding to the input field that does not belong to any range is set to a range consisting of a single item.

[0249] After step ST3a, a loop process in the selected range (selected range) is executed from step ST3b to step ST3d. This loop process includes steps ST4 to ST7 similar to the above, and step ST3c.

[0250] Step ST4 is a process of extracting a process within a selected range from the input process 3a.

[0251] Step ST5 is a process of acquiring an input value example of the process of extracting from the document data 2a.

[0252] Step ST6 is a process of estimating the input format based on the acquired input value example.

[0253] Step ST7 is a process of checking the estimated input form.

[0254] After step ST7, in step ST3c, even if the result of the format check indicates that notification is required, the format estimation unit 6 does not notify the user immediately, but adds the format check processing result. That is, the format estimation unit 6 adds the number of command misdetections and misidentifications in the format check processing, and merges the misidentification list.

[0255] After the loop processing for each range, the form checking unit 7 executes steps ST8 and ST9 in the same manner as described above. Thus, the form checking unit 7 refers to the addition result of the form checking processing, and notifies the input process creator from the display unit 12 according to whether there is command misdetection and whether the misrecognition rate is above the threshold value (step ST9).

[0256] Thereafter, the processing after step ST10 is executed in the same manner as described above. Thus, the input procedure creation processing is completed.

[0257] As described above, according to the second embodiment, the input process 3a also includes a guide including the item name of the input object item. The range selection unit 5 selects the range of the input field based on the guide or the identifier. Therefore, in addition to the above-mentioned effect, the range to which the same input form should be set is automatically selected, so that the time spent on setting the input process 3a can be further shortened.

[0258] <Third Embodiment>

[0259] In the first embodiment, the input format is estimated from the input value example. In contrast, in the third embodiment, the input format is estimated using the display format setting set in the form data 2a.

[0260] Along with this, the information processing device 1 is as follows Fig.24 As shown in FIG. 1 , a format setting acquisition unit 13 is provided in place of the input value example acquisition unit 4 .

[0261] The format setting acquisition unit 13 acquires the format setting of the selected range from the form data 2a. The format setting is a setting for displaying the value of the input field. As the format setting acquisition unit 13, for example, a function provided in an existing form editing application may be used. For example, in Excel (registered trademark), it is possible to select whether to display the value as a numeric value or a date and time when displaying the value as the format of the cell (input field). The format setting acquisition unit 13 is an example of an information acquisition unit that acquires the format setting included in the record data sheet as information related to the value of the input field.

[0262] On the other hand, the form estimation unit 6 estimates the input form based on the obtained format setting instead of the input value example. For example, the form estimation unit 6 can mechanically convert from one format setting to one input form. That is, if the format is "numeric value (N digits below the decimal point)", the input form is "decimal number_1~_N", and if the format is "date (2012 / 3 / 14)", the input form is "date_year-month-day", etc. For each item, there is one format setting, so one input form can be obtained for each item. Afterwards, as in the first embodiment, the merging of multiple input forms and the adjustment corresponding to the selection range are performed.

[0263] The other structures are the same as those of the first embodiment.

[0264] According to the above structure, Fig.25 , Fig.26 as well as Fig. 27 As shown, by replacing the input value example (E1, E2, ..., E m ) and use the format settings (F1, F2, ..., F m ) and performs input process creation processing, form inference processing and form checking processing in the same manner as described above.

[0265] As described above, according to the third embodiment, in the form data 2a, the information on the value of the input field for voice input is the format setting used to display the value of the input field. The format setting acquisition unit 13 acquires the format setting included in the form data 2a as the information on the value of the input field for voice input. Therefore, in addition to the above-mentioned effects, there is no need to prepare an input value example separately.

[0266] In the third embodiment, instead of the input value example as an example of the value of the input field, the format setting used to display the value of the input field is obtained from the form data 2a, but the present invention is not limited to this. For example, in the third embodiment, the format setting acquisition unit 13 may also acquire information including the input value example and the format setting from the form data 2a. In this case, the format setting acquisition unit 13 is an example of an information acquisition unit that acquires information including the input value example and the format setting (information related to the value of the input field) from the recording data sheet. According to such a modified example, the effects of the first and third embodiments can be obtained at the same time.

[0267] <Fourth Implementation Method>

[0268] In addition to the first embodiment, the fourth embodiment confirms whether the set input procedure is correctly operated, and thus presents an actual operation example to the input procedure creator.

[0269] Along with this, the information processing device 1 is as follows Fig.28 As shown, except Figure 1 In addition to the illustrated configuration, an operation example display unit 14 is further provided.

[0270] Here, the action example display unit 14 performs voice input instead of the voice input person speaking, automatically generates input values, and automatically generates voice data using voice synthesis to display action examples. In addition, the action example display unit 14 generates input value examples based on the inferred input form, performs voice synthesis on the generated input value examples, and provides the voice data obtained by voice synthesis instead of the user's voice to the recognition control unit 11, and displays the action example of the voice input controlled by the recognition control unit 11. The action example can be saved as an animation and displayed by animation playback, or it can be displayed by overlapping the ledger data 2a. After the action example is displayed by overlapping the ledger data 2a, the action example display unit 14 deletes the value input to the ledger data 2a as an action example according to the operation of the input process creator.

[0271] The other structures are the same as those of the first embodiment.

[0272] Next, use Fig.29 , the operation of the information processing device constructed as described above is described. Fig. 20In the voice input process shown, instead of the user uttering and voice recognition (steps ST26 and ST27), an input value example is automatically generated according to the input form, voice synthesis is performed on it, the synthesized voice data is played back, and voice recognition is performed. The following will explain in order.

[0273] Steps ST21 to ST25 are performed in the same manner as described above.

[0274] After step ST25 , in step ST26 a , the operation example display unit 14 generates input value examples based on the estimated input format.

[0275] In step ST26b, the operation example display unit 14 performs voice synthesis on the generated input value example to obtain voice data.

[0276] In step STS26c, the action example display unit 14 provides the obtained voice data to the recognition control unit 11. The recognition control unit 11 performs voice recognition using the voice recognition dictionary, and inputs the text of the obtained voice recognition result into the input field according to the identifier of the input field of the recognition input object item. The action example display unit 14 displays such an action example of voice input controlled by the recognition control unit 11 by providing the voice data to the recognition control unit 11.

[0277] Thereafter, the processing after step ST28 is executed in the same manner as described above.

[0278] As described above, according to the fourth embodiment, the action example display unit 14 generates an input value example based on the estimated input form, performs voice synthesis on the generated input value example, and provides the voice data obtained by voice synthesis instead of the user's voice to the recognition control unit 11, thereby displaying an action example of the voice input controlled by the recognition control unit 11. Therefore, in addition to the effect of the first embodiment, how the set input process operates is displayed and played through the actual action example, so it is possible to easily confirm whether the input process is appropriate.

[0279] <Fifth Implementation Method>

[0280] The fifth embodiment is a specific example of the above embodiments and modifications, and is a mode of realizing the above information processing device 1 by a computer. The information processing device 1 may be realized by any computer such as a general-purpose computer such as a personal computer or a dedicated computer such as an inspection device (embedded system).

[0281] Fig.30This is a block diagram illustrating the hardware structure of the information processing device 1 involved in the fifth embodiment. The information processing device 1 includes a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, a ROM (Read Only Memory) 33, a storage device 34, a display device 35, an input device 36, and a communication device 37, which are respectively connected by a bus.

[0282] The CPU 31 is a processor that performs calculation processing and control processing according to a program. The CPU 31 uses a predetermined area of ​​the RAM 362 as a working area, and cooperates with programs stored in the ROM 33 and the storage device 34 to perform processing of various parts of the above-mentioned information processing device 1. The CPU 31 and the processor can also be called a processing circuit.

[0283] The RAM 32 is a memory such as SDRAM (Synchronous Dynamic Random Access Memory) and functions as a work area of ​​the CPU 31 .

[0284] The ROM 33 is a memory that stores programs and various information in a non-rewritable manner.

[0285] The storage device 34 is a device for writing and reading data on a magnetic recording medium such as a HDD (Hard Disc Drive), a storage medium made of a semiconductor such as a flash memory, or a storage medium that can record magnetically such as a HDD, or a storage medium that can record optically. The storage device 34 writes and reads data on the storage medium according to the control from the CPU 31. The storage device 34 is an example of a memory of a computer.

[0286] The display device 35 is a display such as LCD (Liquid Crystal Display) and displays various information based on a display signal from the CPU 31 .

[0287] The input device 36 is an input device such as a mouse and a keyboard, etc. The input device 36 receives information inputted by a user operation as an instruction signal, and outputs the instruction signal to the CPU 31 .

[0288] The communication device 37 communicates with external equipment via a network in accordance with control from the CPU 31. The communication device 37 may also be referred to as a communication circuit.

[0289] The instructions shown in the processing process shown in the above embodiment can be executed according to a program as software. It is also possible for a general-purpose computer system to store the program in advance and read the program, thereby obtaining the same effect as the effect obtained by the control action of the above-mentioned information processing device 1. The instructions described in the above embodiment are recorded as a program that can be executed by a computer on a disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, Blu-ray (registered trademark) Disc (Blu-ray Disc)), a semiconductor memory, or a non-temporary recording medium similar thereto. As long as it is a recording medium that can be read by a computer or an embedded system, its storage form can be any form. The storage medium can also be referred to as a non-transitory computer-readable storage medium (non-transitory computerreadable storage medium). If the computer reads the program from the recording medium and uses the CPU to execute the instructions recorded in the program according to the program, it is possible to implement an information processing method accompanied by the same action as the control of the information processing device 1 in the above embodiment. For example, the information processing method includes: obtaining one or more items and information related to the value of the input field for one or more items from the form data 2a including the input field for voice input for each item; and inferring the input form of the input field based on the information. Of course, when the computer obtains or reads the program, it can also be obtained or read via a network.

[0290] In addition, part of each process for realizing this embodiment may be executed by an OS (operating system), database management software, network middleware, etc. running on a computer according to instructions of a program installed from a recording medium to the computer or embedded in a system.

[0291] Furthermore, the recording medium in the present embodiment is not limited to a medium independent of the computer or the embedded system, but also includes a recording medium in which a program transmitted via a LAN or the Internet is downloaded and stored or temporarily stored.

[0292] In addition, the recording medium is not limited to one, and even when the process in this embodiment is executed from a plurality of media, the recording medium in this embodiment is included, and the structure of the medium may be any structure.

[0293] In addition, the computer or embedded system in this embodiment may be a structure for executing each process in this embodiment according to a program stored in a recording medium, and may be any structure such as a device composed of a personal computer, a microcomputer, etc., a system connecting multiple devices to a network, etc.

[0294] The computer in the present embodiment is not limited to a personal computer, but also includes a processing unit, a microcomputer, etc. included in an information processing machine, and generally refers to machines and devices that can realize the functions in the present embodiment through a program.

[0295] (Variations of each embodiment)

[0296] Furthermore, each embodiment and each modification example may be expressed as an information processing method or an information processing program including each step of the above-mentioned information processing device 1 .

[0297] According to at least one embodiment described above, even when a voice recognition expert is not available, an appropriate input format can be determined. This is also the case in at least one of the above-mentioned variations.

[0298] In addition, although several embodiments of the present invention are described, these embodiments are provided as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their variations are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

[0299] Furthermore, the above-mentioned embodiments can be summarized as the following technical solutions.

[0300] Technical Solution 1

[0301] A storage medium storing an information processing program, wherein the information processing program enables a computer to implement:

[0302] an information acquisition unit that acquires, for each item, one or more items and information related to values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; and

[0303] The form estimation unit estimates the input form of the input field based on the information.

[0304] Technical Solution 2

[0305] According to the above technical solution 1, an input process for inputting values ​​into a plurality of input fields included in the recording data sheet is stored in the memory of the computer.

[0306] The input process includes identifying an identifier of an input field included in an input target item of the recording data sheet and the estimated input form.

[0307] Technical Solution 3

[0308] According to the above technical solution 2, the information processing program causes the computer to further implement a range selection unit for selecting a range of a plurality of input fields from the recording data sheet.

[0309] The information acquisition unit acquires the information corresponding to the input field in the selected range,

[0310] The form estimation unit estimates an input form common to input fields within the range based on the information acquired in the corresponding manner.

[0311] Technical Solution 4

[0312] According to the above technical solution 3, the input process further includes a guide including the project name of the input object project,

[0313] The range selection unit selects a range of the input field based on the guidance or the identifier.

[0314] Technical Solution 5

[0315] According to the above technical solution 2, the information processing program enables the computer to further implement:

[0316] a dictionary generating unit that generates a voice recognition dictionary based on the inferred input form; and

[0317] The recognition control unit performs voice recognition on the user's utterance using the voice recognition dictionary, and inputs a text of the obtained voice recognition result into the input field according to the identifier.

[0318] Technical Solution 6

[0319] According to the above technical solution 1, the information is an input value example which is an example of a value of the input field,

[0320] The information acquisition unit acquires the value as the input value example from at least one of the recording data sheet in which the value of the input field is input in advance and a user interface that receives the input of the value to the input field.

[0321] Technical Solution 7

[0322] According to the above technical solution 1, the information is used to set the format of the value of the input field.

[0323] The information acquisition unit acquires the format setting included in the recording data sheet as the information.

[0324] Technical Solution 8

[0325] According to the above technical solution 1, the information includes an input value example as an example of the value of the input field, and a format setting used to display the value of the input field.

[0326] The information acquisition unit acquires the information including the input value example and the format setting from the recording data sheet.

[0327] Technical Solution 9

[0328] According to the third aspect of the present invention, the form estimation unit estimates the input form for each input field in the range, and estimates one of the estimated input forms as the common input form.

[0329] Technical Solution 10

[0330] According to the third aspect of the present invention, the form estimation unit estimates the input form for each input field in the range, and estimates a plurality of input forms including the estimated input forms as the common input form.

[0331] Technical Solution 11

[0332] According to the above-mentioned technical solution 3, the form estimation unit adjusts the estimated input form according to the amount of the information obtained correspondingly.

[0333] Technical Solution 12

[0334] According to the above technical solution 3, the input process also includes upper and lower limits representing the range of the value of the input field,

[0335] The form estimation unit adjusts the estimated input form based on upper and lower limit values ​​included in the input process.

[0336] Technical Solution 13

[0337] According to the above technical solution 2, the computer also implements a form inspection unit,

[0338] The input process further includes guidance related to voice input to the input field,

[0339] The format checking unit checks the estimated input format based on the guidance.

[0340] Technical Solution 14

[0341] According to the above technical solution 5, the information processing program enables the computer to further implement a form checking unit.

[0342] The form checking unit generates an input value example based on the inferred input form, performs voice synthesis on the generated input value example, performs voice recognition on the voice data obtained by the voice synthesis using the voice recognition dictionary, and determines whether the text of the obtained voice recognition result matches the generated input value example, thereby checking the inferred input form.

[0343] Technical Solution 15

[0344] According to the above-mentioned technical solution 14, the format checking unit classifies the situation in the case of non-compliance by determining whether a command is detected from the text of the voice recognition result based on the result of the determination.

[0345] Technical Solution 16

[0346] According to the above-mentioned technical solution 14, the form checking unit checks the input form by calculating a recognition rate or a false recognition rate of the voice recognition based on the number of determinations and the result of the determination.

[0347] Technical Solution 17

[0348] According to the above-mentioned technical solution 14, the form checking unit can identifiably notify the result obtained by checking the input form.

[0349] Technical Solution 18

[0350] According to the fifteenth aspect of the invention, when the format checking unit detects the command, the format checking unit changes the voice input command consistent with the command according to the user's operation.

[0351] Technical Solution 19

[0352] According to the above-mentioned technical solution 1, the information processing program causes the computer to further realize a display unit that displays the recording data sheet and displays the estimated input form in an input field included in the recording data sheet.

[0353] Technical Solution 20

[0354] According to the above technical solution 1, the information processing program enables the computer to further implement a display unit.

[0355] The display unit generates one or more input value examples based on the estimated input form, and displays the generated input value examples in an input field included in the recording data sheet.

[0356] Technical Solution 21

[0357] According to the above technical solution 1, the information processing program enables the computer to further implement a display unit.

[0358] The display unit generates a range of values ​​that can be input into the input field based on the estimated input form, and displays the generated range in the input field included in the recording data sheet.

[0359] Technical Solution 22

[0360] According to the above technical solution 1, the information processing program enables the computer to further implement a display unit.

[0361] An input process including an identifier for identifying an input field of an input object item included in the recording data sheet, the estimated input form, and an order of inputting values ​​into a plurality of input fields included in the recording data sheet is stored in a memory of the computer,

[0362] The display unit displays the recording data sheet and displays the order of inputting the values ​​in a superimposed manner in a plurality of input fields included in the recording data sheet according to the input process.

[0363] Technical Solution 23

[0364] According to the above technical solution 5, the information processing program causes the computer to further implement an action example display unit.

[0365] The action example display unit generates an input value example based on the inferred input form, performs voice synthesis on the generated input value example, and provides the voice data obtained by the voice synthesis to the recognition control unit instead of the user's utterance, thereby displaying an action example of the voice input controlled by the recognition control unit.

[0366] Technical Solution 24

[0367] An information processing device comprising:

[0368] an information acquisition unit that acquires, for each item, one or more items and information related to values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; and

[0369] The form estimation unit estimates the input form of the input field based on the information.

[0370] Technical Solution 25

[0371] An information processing method, comprising:

[0372] For each item, acquiring one or more items and information on values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; and

[0373] Based on the information, the input form of the input field is inferred.

Claims

1. A storage medium storing an information processing program, wherein the information processing program enables a computer to implement: an information acquisition unit that acquires, for each item, one or more items and information related to values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; and The form estimation unit estimates the input form of the input field based on the information.

2. The storage medium according to claim 1, wherein: An input process for inputting values ​​into a plurality of input fields included in the recording data sheet is stored in the memory of the computer. The input process includes identifying an identifier of an input field included in an input target item of the recording data sheet and the estimated input form.

3. The storage medium according to claim 2, wherein: The information processing program causes the computer to further implement a range selection unit for selecting a range of a plurality of input fields from the recording data sheet. The information acquisition unit acquires the information corresponding to the input field in the selected range, The form estimation unit estimates an input form common to input fields within the range based on the information acquired in the corresponding manner.

4. The storage medium according to claim 3, wherein: The input process further includes a guide including the item name of the input object item, The range selection unit selects a range of the input field based on the guidance or the identifier.

5. The storage medium according to claim 2, wherein: The information processing program causes the computer to further implement: a dictionary generating unit, generating a voice recognition dictionary according to the inferred input form; as well as The recognition control unit performs voice recognition on the user's utterance using the voice recognition dictionary, and inputs a text of the obtained voice recognition result into the input field according to the identifier.

6. The storage medium according to claim 1, wherein: The information is an input value example which is an example of a value of the input field, The information acquisition unit acquires the value as the input value example from at least one of the recording data sheet in which the value of the input field is input in advance and a user interface that receives the input of the value to the input field.

7. The storage medium according to claim 1, wherein: The information is used to format the value of the input field. The information acquisition unit acquires the format setting included in the recording data sheet as the information.

8. The storage medium according to claim 1, wherein: the information includes an input value example which is an example of a value of the input field, and a format setting used to display the value of the input field, The information acquisition unit acquires the information including the input value example and the format setting from the recording data sheet.

9. The storage medium according to claim 2, wherein: The computer further implements a form checking unit, The input process further includes guidance related to voice input to the input field, The format checking unit checks the estimated input format based on the guidance.

10. The storage medium according to claim 5, wherein: The information processing program causes the computer to further implement a form checking unit. The form checking unit generates an input value example based on the inferred input form, performs voice synthesis on the generated input value example, performs voice recognition on the voice data obtained by the voice synthesis using the voice recognition dictionary, and determines whether the text of the obtained voice recognition result matches the generated input value example, thereby checking the inferred input form.

11. The storage medium according to claim 1, wherein: The information processing program causes the computer to further implement a display unit. The display unit displays the recording data sheet and displays the estimated input form in an input field included in the recording data sheet.

12. The storage medium according to claim 1, wherein: The information processing program causes the computer to further implement a display unit. The display unit generates one or more input value examples based on the estimated input form, and displays the generated input value examples in an input field included in the recording data sheet.

13. The storage medium according to claim 1, wherein: The information processing program causes the computer to further implement a display unit. The display unit generates a range of values ​​that can be input into the input field based on the estimated input form, and displays the generated range in the input field included in the recording data sheet.

14. The storage medium according to claim 1, wherein: The information processing program causes the computer to further implement a display unit. An input process including an identifier for identifying an input field of an input object item included in the recording data sheet, the estimated input form, and an order of inputting values ​​into a plurality of input fields included in the recording data sheet is stored in a memory of the computer, The display unit displays the recording data sheet and displays the order of inputting the values ​​in a superimposed manner in a plurality of input fields included in the recording data sheet according to the input process.

15. An information processing device comprising: an information acquisition unit that acquires, for each item, one or more items and information related to values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; and The form estimation unit estimates the input form of the input field based on the information.

16. An information processing method comprising: For each item, acquiring one or more items and information on values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; and Based on the information, the input form of the input field is inferred.

Images