Information processing program, information processing device and information processing method

Abstract

To provide an information processing program, an information processing device and an information processing method which can determine an appropriate input form, even in a case in which a specialist of sound recognition is absent.SOLUTION: An information processing program makes a computer achieve an information acquisition unit and a form estimation unit. The information acquisition unit acquires one or more terms, and information on a value of an input field for the one or more terms, for each of the terms, from a data sheet for recording including an input column of voice input. The form estimation unit estimates the input form of the input column, on the basis of the information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing program, an information processing device, and an information processing method. [Background technology]

[0002] At manufacturing sites and maintenance and inspection sites, measurements taken with measuring instruments and visual inspection results are sometimes entered into record data sheets such as forms and tables, and the record data sheets after the data is entered are sometimes shared with other workers or customers. In this case, the content to be entered into the record data sheet is determined in advance, and the worker performs the work according to the work procedure and enters the obtained data in the specified position on the record data sheet.

[0003] In general form software, data is entered as text, but since entering text while working takes time, there is a need to enter data using voice. For example, there is a voice input method that allows you to enter a value into a selected item when speaking by setting the input target item and the contents to be entered into that item in an application other than the form software, and furthermore, by specifying the next item to be entered during setting, you can enter values ​​into consecutive items. With such a voice input method, it is possible to perform voice input based on the input format by having a voice recognition expert predetermine the input format of the value to be entered.

[0004] The above-described voice input method usually presents no particular problems, but according to the inventor's investigation, there is room for improvement in that it is difficult to determine an appropriate input format when a voice recognition expert is not present. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2008-52676 A Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide an information processing program, an information processing device, and an information processing method that are capable of determining an appropriate input format even when a speech recognition expert is not available. [Means for solving the problem]

[0007] An information processing program according to an embodiment causes a computer to realize an information acquisition unit and a format estimation unit. The information acquisition unit acquires, for each item, one or more items and information on values ​​in the input fields for the one or more items from a recording data sheet including an input field for voice input. The format estimation unit estimates an input format for the input field based on the information. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to a first embodiment. [Diagram 2] FIG. 4 is a diagram showing an example of form data according to the first embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of an initial setting of an input procedure storage unit according to the first embodiment. [Figure 4] FIG. 2 is a view showing an example of a word list according to the first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of an inputted form according to the first embodiment. [Figure 6] FIG. 11 is a diagram showing an input procedure storage unit that has been automatically set according to the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of a guidance input format correspondence table according to the first embodiment. [Figure 8] 5 is a flowchart showing an input procedure creation process according to the first embodiment. [Figure 9] FIG. 4 is a view showing an example of range selection according to the first embodiment. [Figure 10] FIG. 4 is a diagram showing an example of an alert according to the first embodiment. [Figure 11]FIG. 11 is a diagram showing another example of an alert according to the first embodiment. [Figure 12] FIG. 2 is a diagram showing an example of an input format display of a form according to the first embodiment. [Figure 13] FIG. 4 is a diagram showing an example of a display of input values ​​of a form according to the first embodiment. [Figure 14] FIG. 2 is a diagram showing an example of input order display of a form according to the first embodiment. [Figure 15] FIG. 11 is a view showing another example of input order display of the form according to the first embodiment. [Figure 16] FIG. 4 is a diagram showing a format estimation process (without concatenation) according to the first embodiment. [Figure 17] FIG. 4 is a diagram showing an example of a format allocation rule according to the first embodiment. [Figure 18] FIG. 11 is a diagram showing format estimation (without concatenation) according to the first embodiment. [Figure 19] FIG. 4 is a diagram showing a format check process according to the first embodiment. [Figure 20] FIG. 4 is a diagram showing a voice input process according to the first embodiment. [Figure 21] FIG. 11 is a block diagram showing the configuration of an information processing apparatus according to a second embodiment. [Figure 22] FIG. 11 is a view showing an example of automatic range selection according to the second embodiment. [Figure 23] 10 is a flowchart showing an input procedure creation process according to the second embodiment. [Figure 24] FIG. 11 is a block diagram showing the configuration of an information processing device according to a third embodiment. [Diagram 25] 13 is a flowchart showing an input procedure creation process according to the third embodiment. [Figure 26] 13 is a flowchart showing a format estimation process (without concatenation) according to the third embodiment. [Figure 27] 13 is a flowchart showing a format check process according to the third embodiment. [Figure 28] FIG. 13 is a block diagram showing the configuration of an information processing device according to a fourth embodiment. [Figure 29] 13 is a flowchart showing an operation example display process according to the fourth embodiment. [Diagram 30] FIG. 13 is a diagram showing an example of the hardware configuration of an information processing device according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment will be described in detail below with reference to the drawings. In the following description, an example will be described in which an input procedure creation process and a voice input process are executed in an information processing device such as a tablet terminal having a voice input function. Note that the information processing device is not limited to a tablet terminal, and any computer such as a PC (Personal Computer) or a smartphone equipped with an external microphone or an external speaker can be used as appropriate. In addition, the information processing device does not necessarily have to perform the input procedure creation process and the voice input process, and it is sufficient that the information processing device is a device that at least performs the input procedure creation process. That is, the program for the input procedure creation process and the program for the voice input process may be installed as one information processing program, or may be installed as separate information processing programs. In any case, the information processing device realizes each function of the input procedure creation process and the voice input process by executing the information processing program. In addition, when the information processing device performs only the input procedure creation process of the input procedure creation process and the voice input process, the microphone may be omitted. In addition, in the following description, a user related to the input procedure creation process is called an input procedure creator, and a user related to the voice input process is called a voice input person. The voice inputter and the input procedure creator may be the same person or different people. Also, a plurality of voice inputters and a plurality of input procedure creators may work together.

[0010] <First embodiment> 1 is a block diagram showing an example of the configuration of an information processing device according to the first embodiment. The information processing device 1 includes a data storage unit 2, an input procedure storage unit 3, an input value example acquisition unit 4, a range selection unit 5, a format estimation unit 6, a format check unit 7, a dictionary generation unit 8, a voice recognition unit 9, a voice synthesis unit 10, a recognition control unit 11, and a display unit 12. The information processing device 1 is a computer that realizes the functions of each unit by executing an information processing program installed in a memory.

[0011] Here, as shown in FIG. 2, the data storage unit 2 stores form data 2a representing data having an input field for voice input for each item, such as a form or table. The form data 2a is an example of a recording data sheet including an input field for voice input for each item. For example, in the form data 2a, values ​​are input into input fields associated with rows indicated by row numbers 1 to 5 and items indicated by column numbers B to D, and values ​​are input into input fields associated with rows indicated by row numbers 8 to 9 and items indicated by column numbers B to D. The form data 2a may include a word sheet, which will be described later, as a separate sheet. The data storage unit 2 is an example of a computer memory.

[0012] As shown in FIG. 3, the input procedure storage unit 3 stores input procedures 3a for inputting values ​​into a plurality of input fields included in the form data 2a, which is a data sheet for recording. The input procedures 3a include, for example, a procedure number, an input target item, an input format, guidance, and a reference value as item names. However, the procedure number, guidance, and reference value are optional additional items and may be omitted. Such input procedures 3a also function as a list of the order for inputting into a plurality of items. The order of the input procedures 3a may be determined from top to bottom in the list, or a number indicating the order may be separately included as an item of the input procedure. The input procedure storage unit 3 is an example of a computer memory.

[0013] Here, the item name of the procedure number represents a number indicating the order in which values ​​are to be input into a plurality of input fields included in the form data 2a as the value of the procedure number.

[0014] The item name of the input target item represents an identifier that identifies the input field of the input target item contained in the form data 2a as the value of the input target item. For example, if the form data is in a table format as shown in Figure 2, the identifier may be written using the column number and row number, such as D2. In Figure 3, input procedure 3a shows an example in which the first measurement result is input into B1, B2, ..., B5, and then B8, C8, D8 are input, and the second measurement result is input into C1, C2, ..., C5.

[0015] The item name of the input format represents the input format estimated by the format estimation unit 6 as the value of the input format. The input procedure 3a includes an input format field into which the estimated input format can be input, and the input format estimated by the format estimation unit 6 is input into the input format field. In addition, the input format specifies the format of the value to be input into the input field of the input target item. The recognition control unit 11, which will be described later, creates a voice recognition dictionary based on the input format input into the input format field. Since it is difficult to directly write the voice recognition dictionary into the input procedure, in this embodiment, an input format with simplified description is input.

[0016] Input formats can be broadly categorized as appropriate, such as "numbers," "alphanumeric characters," "date and time," "date," "time," and "words."

[0017] In the case of "numbers," it may also be possible to specify the base (decimal, hexadecimal, etc.), and if it is a decimal number, the number of digits in the integer part (it may also be possible to specify a range), the number of digits in the decimal part, etc.

[0018] For example, if the input format is a decimal number, the range of digits in the integer part is Mmin~Mmax, and the range of digits in the decimal part is Nmin~Mmin, the input format is written as "Decimal number_Mmin~Mmax_Nmin~Nmax". If there is no upper limit to the number of digits in the "number", the input format is written as "Decimal number_Mmin~_Nmin~", and if there is no lower limit to the "number", the input format is written as "Decimal number_~Mmax_~Nmax". If there is no range of digits in the "number" (number of digits in the integer part is M, and in the decimal part is N), the input format is written as "Decimal number_M_N". If a positive or negative sign "+" or "-" is spoken before the number in the "number", the input format is written as "[+-]Decimal number_M_N".

[0019] In the case of "alphanumeric characters", a pattern of alphabetic characters, numbers, and symbols may be further specified. For example, as an alphanumeric character pattern, "A" may be written for alphabetic characters, "D" for numbers, and "S" for symbols. Also, as "alphanumeric characters", the character "$" may be used for alphabetic characters or numbers, and the character "!" may be used for alphabetic characters, numbers, or symbols, and for example, a pattern of three alphabetic characters, two numbers, one symbol, and one alphabetic character may be written as "alphanumeric characters_AAADDSA".

[0020] Additionally, for "alphanumeric characters," optional characters (characters that can be entered or not) are preceded by a "?". For example, the pattern "alphanumeric_AAA?DD" represents a pattern of two or three alphanumeric characters and two numeric characters. It may also be possible to write patterns that allow multiple characters to be entered, rather than just patterns of one character.

[0021] In the case of "Date and Time", "Date", and "Time", no additional specification is required, but for "Date", it may be possible to write additional specifications such as specifying which date to enter, such as "year / month / date", "month / date", "year / month", or "day".

[0022] In the case of "words", as shown in FIG. 4, a word list 2b is written on a separate sheet of the form data 2a, and a group name of the word list 2b is specified. The word list 2b includes not only the group name, but also the notation of the word, the corresponding reading (the reading spoken by the user), and the reading out (the content to be read out when repeating in the guidance, etc.). In this embodiment, a number is automatically assigned to the group name in order to automatically generate the word list 2b. Since only the notation of the word can be obtained from the input value example, the reading and reading out content are generated from the notation. The reading and reading out items are prepared as separate items so that the input procedure creator can change them to different contents later, but the same contents can be generated and set at the generation stage. Any existing method can be used to generate the reading and reading out. For example, a morphological analysis technique and a dictionary for morphological analysis can be used.

[0023] The input format is not limited to the above-mentioned major categories, and may be any input format that can be converted into a speech recognition dictionary, and may be a combination of individual input formats. For example, the input format may be a concatenated format of "date_month_date, word_day of the week, (time_hours)". In this case, after speaking the date, the word group "day of the week" (Sunday, Monday, Tuesday, ...) is spoken, and then the time can be spoken or not spoken. () in the concatenation indicates that the speech is optional. In this case, the recognition results are separated by spaces. For example, in the case of "date_month_date, word_day of the week, (time_hours)", the recognition results may be "3 / 10 (space) Friday (space) 9:10" or "3 / 11 (space) Saturday".

[0024] The input format may also be an OR of multiple input formats, such as "time_hour_minute|time_hour_minute_second" or "decimal_2_1|word_abnormality". In this case, either "time_hour_minute" or "time_hour_minute_second" may be spoken, and either "decimal_2_1" or "word_abnormality" may be spoken. For example, "time_hour_minute|time_hour_minute_second" will recognize "9:10" and "13:20:10", while "decimal_2_1|word_abnormality" will recognize "12.3" and "abnormality".

[0025] The input format may also be a combination of the above concatenated formats or ORs. The method of describing the input format is not limited to the above, and may be any description method that can be converted into a voice recognition dictionary. Note that specifying the input format as narrowly as possible (narrowing the range of digits for numbers, and reducing the number of word lists for words) will improve the voice recognition rate, but since the speech content that can be input is narrowed down, if it is too detailed, it will be difficult to use for the voice inputter.

[0026] The guidance item name is used as the guidance value to be played back at the start of each step to present the work instructions for that step when actually executing the input steps and performing voice input. Note that the guidance is not limited to the item name of the item to be input, but may be any guidance regarding voice input into the input field. For example, in the case of an inspection input procedure, the guidance includes a description that provides guidance on the contents of the inspection procedure. Of all the guidance, only guidance that matches a specific pattern, such as a certain time, is used in subsequent input format checks, etc.

[0027] The item name of the reference value indicates the range of reference values ​​for the value entered in the input field. The reference value is a value indicating the range of values ​​in the input field and may include an upper limit value, a lower limit value, or both upper and lower limits. The reference value is used to check whether the value entered by voice input is abnormal when actually executing the input procedure to enter voice input.

[0028] The input procedure 3a as described above may be stored in the form data 2a, not limited to the case where it is stored in the input procedure storage unit 3 as data separate from the form data 2a. For example, in a general spreadsheet application, since multiple spreadsheets can be created in one file, one of them may be used for form data and the other for storing input procedures. In this case, the recognition control unit 11 described later acquires the input procedure from the sheet for storing procedures. Also, when the input procedure 3a is stored separately from the form data 2a, it may be stored in a storage such as a DB (Database) or in a memory. In this embodiment, it is assumed that a sheet for storing input procedures is included in another sheet in the file of the form data 2a. In this case, by copying the sheet for storing input procedures to a file of another form data, voice input can be performed even in the form data to which it is copied.

[0029] The input procedure 3a may be created by manually inputting areas other than the input format. For example, when creating a procedure storage sheet in the file of the form data 2a, the input procedure 3a other than the input format may be created by manually inputting a table similar to that shown in Fig. 3 into the procedure storage sheet. The input procedure 3a may also be created by clicking an item to be input to display a dialog box and manually inputting information other than the input format on the dialog box.

[0030] The input value example acquisition unit 4 acquires, for each item, one or more items and information on the values ​​of the input fields for one or more items from the form data 2a including the input field for voice input. As the information on the values ​​of the input fields, for example, input value examples that are examples of the values ​​of the input fields and format settings used to display the values ​​of the input fields can be used as appropriate. In the first embodiment, the input value example acquisition unit 4 uses the input value example as the information on the values ​​of the input fields. In this case, the input value example acquisition unit 4 can acquire the values ​​as the input value example from at least one of the form data 2a in which the values ​​of the input fields have been input in advance and a user interface that accepts input of values ​​into the input fields. As the form data 2a in which the values ​​of the input fields have been input in advance, if at least one of all the input fields of the form data 2a has been input, an input value example can be acquired from the input field that has been input. In addition, the input value example acquisition unit 4 may acquire information on the values ​​of the input fields corresponding to the input fields in the range selected by the range selection unit 5. Note that the "information on the values ​​of the input fields" may be appropriately read as "input hints for the input fields", "input hint information for the input fields", etc. The input value example acquiring unit 4 is an example of an information acquiring unit.

[0031] FIG. 5 shows an example of input form data 2a. In the case of this form data 2a, only the first and second measurement result values ​​and position 1 and position 2 of #1 are input for the noise measurement. The input value example acquisition unit 4 acquires input value examples by acquiring the values ​​of the items indicated by the identifiers of the input target items of each input procedure for the input form data 2a. For example, for the form data 2a in FIG. 5, the input value example for the input procedure of procedure number 1 in FIG. 3 is "10:30" since it is sufficient to acquire the value of the input target item of identifier B1. Such input form data 2a may be manually input before the voice input procedure is set, or a new input value example may be manually input for the setting. Also, there may be more than one input form data 2a. When there are multiple form data 2a, multiple input value examples are obtained for one input procedure. Furthermore, if the procedure is set from a user interface such as a dialog, a text box for accepting example input values ​​may be added to the dialog, and the example input value acquisition unit 4 may acquire the example input values ​​by acquiring the text entered in the text box.

[0032] The range selection unit 5 selects a range of multiple input fields from the form data 2a. More specifically, the range selection unit 5 selects a range of multiple input fields as a range for estimating an input format. For example, the range selection unit 5 provides a user interface for selecting a range for setting an input format. The range selection unit 5 may use a function of an existing application that displays a form, or may use another method.

[0033] The format estimation unit 6 estimates the input format of the input field based on the input value examples acquired by the input value example acquisition unit 4. The format estimation unit 6 may also estimate an input format common to the input fields of the range selected by the range selection unit 5 based on the input value examples acquired corresponding to the input fields of the range. The common input format is estimated from a plurality of input value examples. The format estimation unit 6 may also estimate an input format for each input field of the range, and estimate one input format of the estimated input formats as the common input format. The format estimation unit 6 may also estimate an input format for each input field of the range, and estimate an input format including a plurality of the estimated input formats as the common input format. The format estimation unit 6 may also adjust the estimated input format based on the number of input value examples acquired corresponding to the input field of the selected range. The format estimation unit 6 may also adjust the estimated input format based on a reference value (e.g., upper and lower limit values) included in the input procedure 3a. In either case, the format estimation unit 6 estimates the input format from the example input value, and updates the input format field of the input procedure 3a with the estimated input format. An example of the updated input procedure 3a is shown in Figure 6. In this case, the input format has already been set for the input procedures 3a for items other than B4, C4, and D4.

[0034] The format check unit 7 checks the validity of the estimated input format. For example, the format check unit 7 checks the estimated input format based on the guidance included in the input procedure 3a.

[0035] For example, the format check unit 7 generates an input value example from the estimated input format, synthesizes the generated input value example, and executes voice recognition on the voice data (voice waveform data) obtained by the voice synthesis using a voice recognition dictionary. The text of the voice recognition result represents an input value example, a command, or something else. The format check unit 7 may check the estimated input format by determining whether or not the text of the obtained voice recognition result matches the generated input value example. The format check unit 7 may classify the situation when the input value example does not match by determining whether or not a command is detected from the text of the voice recognition result as a result of the determination. However, the determination regarding the detection of a command is performed not only when the result of the determination regarding the match with the input value example is negative. The format check unit 7 may check the input format by calculating the recognition rate or the misrecognition rate of the voice recognition based on the number of times the determination is performed and the result of the determination. The format check unit 7 may also notify the result of checking the input format in an identifiable manner. As the result of checking the input format, for example, a judgment result regarding a match with an input value example, a judgment result regarding erroneous detection of a command, a judgment result regarding other erroneous recognition, statistical information such as an erroneous recognition rate, etc., can be used as appropriate. When a command is detected, the format check unit 7 may delete or change a voice input command that matches the command in response to a user operation. That is, when a conflict with a voice input command occurs, the format check unit 7 may resolve the problem by deleting the voice input command or changing the wording of the voice input command.

[0036] In any case, the format check unit 7 executes at least one of a check based on the guidance and a check using voice synthesis.

[0037] Here, in the case of a format check based on guidance, it is assumed that guidance is set in input procedure 3a. The format check unit 7 checks whether there is any inconsistency between the wording of the guidance and the estimated input format. For example, if the wording of the guidance is "XX time" and the input format is not a time format, it is found that there is an inconsistency.

[0038] Figure 7 shows an example of a rule table of guidance wording conditions and the corresponding input format. The guidance column lists regular expressions that match parts of the guidance wording as conditions for the guidance. The input format column lists the content that the input format must satisfy when the guidance matches the conditions.

[0039] The format check unit 7 checks whether the wording of the set guidance matches the guidance in the rule table for each input procedure corresponding to the selected range. If there is a line of matching guidance, it checks whether the input format of that line matches the estimated input format. If they do not match, it notifies the creator of the input procedure.

[0040] In the case of format checking using voice synthesis, the format checking unit 7 performs voice recognition on the voice waveform data of the input value examples generated from the estimated input format, thereby checking whether or not the estimated input format generates input value examples that are difficult to voice recognize.

[0041] Supplementally, the format check unit 7 converts the text of each input value example into voice waveform data using voice synthesis for each input procedure. The format check unit 7 can execute voice synthesis at any timing, such as when a predetermined number of input value examples are generated or when a predetermined operation by the user is received. The format check unit 7 also performs voice recognition on the obtained voice waveform data using a voice recognition dictionary generated by the dictionary generation unit 8 for the corresponding input format. At this time, the format check unit 7 also simultaneously operates voice recognition for command recognition used by the recognition control unit 11 described later. When the voice recognition for command recognition is different from the voice recognition for input values, the format check unit 7 recognizes the input value examples generated using both of the two voice recognitions. When the voice recognition for input values ​​is used as the voice recognition for command recognition, the generated voice recognition dictionary also includes a dictionary for command recognition, and the voice recognition is performed using the dictionary for command recognition. As a result, a voice recognition result for the generated input value example is obtained. Furthermore, the format check unit 7 compares the speech recognition result with the generated input value examples, and if they all match, determines that the estimated input format is valid.

[0042] The dictionary generation unit 8 generates a voice recognition dictionary based on the estimated input format. For example, the dictionary generation unit 8 generates a voice recognition dictionary capable of recognizing voice utterances in an input format set in the input format field of the input procedure 3a. Here, the voice recognition dictionary describes grammar and can be generated uniquely from the input format. The dictionary generation unit 8 also generates a dictionary for command recognition in parallel.

[0043] The speech recognition unit 9 is controlled by the format check unit 7 and the recognition control unit 11, and converts the speech waveform data into text using a speech recognition dictionary. For example, it is assumed that the speech recognition unit 9 uses speech recognition that uses a speech recognition dictionary describing a predetermined grammar in order to recognize an utterance for speech input. In this type of speech recognition, speech waveform data of an utterance that conforms to the grammar is recognized, and speech waveform data of any other utterance is rejected. The format of the speech recognition dictionary depends on the speech recognition technology used, and may be a grammar described in text or a binary dictionary. It may also be in the form of an object stored in memory. The speech waveform data may be obtained from a user's utterance, or may be obtained by text-to-speech synthesis.

[0044] In this embodiment, in addition to inputting data into items by voice, commands other than the data input, such as undoing the input, proceeding to the next input step, and terminating the input, can be spoken and executed. At this time, the format check unit 7 and the recognition control unit 11 simultaneously perform voice recognition for command recognition.

[0045] The command recognition may be the same as the speech recognition described above, or a different speech recognition may be used. For example, instead of speech recognition using grammar, speech recognition that detects the utterance of keywords included in a keyword list may be used. The speech recognition is not limited to these, and speech recognition for general spoken language may be used. In this case, the speech recognition dictionary may include a word dictionary consisting of the spelling and pronunciation of words to be recognized, and a grammar dictionary for determining whether or not to reject the recognition results.

[0046] When the voice recognition for command recognition is different from the voice recognition for input to the item, the voice recognition unit 9 recognizes the input value example generated using both of the two voice recognitions. When the voice recognition for command recognition is the same as the voice recognition for input to the item, the voice recognition unit 9 includes a dictionary for command recognition in the voice recognition dictionary and performs voice recognition using the dictionary.

[0047] The speech synthesis unit 10 is controlled by the format check unit 7 and the recognition control unit 11 and converts the text into speech waveform data. The speech synthesis unit 10 can play the text as speech from the speaker by sending the speech waveform data to the speaker. The speech synthesis unit 10 is used when checking the validity of the input format estimated by the format check unit 7, when the recognition control unit 11 plays back guidance at the start of each input procedure, and when repeating the input content.

[0048] The recognition control unit 11 executes voice recognition on the user's speech using a voice recognition dictionary, and inputs the obtained text of the voice recognition result into an input field according to an identifier that identifies the input field of the input target item. For example, the recognition control unit 11 performs voice recognition on the user's voice utterance according to an input procedure, and inputs the recognition result into the input field of the form data 2a. Alternatively, the recognition control unit 11 may perform voice recognition using a voice recognition dictionary generated from the input format of the input procedure, and input the recognition result into the input field of the input target item of the input procedure.

[0049] For example, the recognition control unit 11 displays form data on the display unit 12, and retrieves one input procedure from the input procedure storage unit 3 according to the order of the input procedures. The recognition control unit 11 uses the voice synthesis unit 10 to synthesize the guidance contents set in the input procedure. The recognition control unit 11 uses the voice recognition dictionary (and the command recognition dictionary) generated from the input format set in the input procedure to perform voice recognition with the voice recognition unit 9. If a command is recognized, the recognition control unit 11 executes the contents of the corresponding command. If something other than a command is recognized, the recognition control unit 11 performs voice synthesis on the voice recognition result and repeats it, and then compares the recognition result with a reference value set in the input procedure, and inputs the recognition result into the input target item if it is within the reference value. If it is outside the reference value, the recognition control unit 11 notifies the user of this fact through voice synthesis, and prompts the user to re-enter the recognition result, or forcibly inputs the recognition result.

[0050] The display unit 12 displays data according to the processing of the information processing device 1. For example, the display unit 12 displays the form data 2a, and displays an estimated input format in an input field included in the form data 2a. For another example, the display unit 12 may generate one or more input value examples from the estimated input format, and display the generated input value examples in an input field included in the form data 2a. For another example, the display unit 12 may generate a range of values ​​that can be input into an input field from the estimated input format, and display the generated range in an input field included in the form data 2a. For another example, the display unit 12 may display the form data 2a, and superimpose an order of inputting values ​​into a plurality of input fields included in the form data 2a based on the input procedure 3a.

[0051] Specifically, for example, the display unit 12 includes a microphone for recording the user's voice utterance, a speaker for playing back the voice, a display for displaying stored data and various information, and a control circuit for controlling the contents displayed on the display. The display unit 12 displays the form data 2a on the display or the like. The display unit 12 may also appropriately superimpose the data in the input procedure 3a on the form data 2a. The display unit 12 may be called an output unit since it includes a speaker and a display, or may be called an input / output unit since it further includes a microphone.

[0052] Next, the operation of the input procedure creation process and the voice input process by the information processing device configured as above will be described with reference to Fig. 8 to Fig. 20. In the following description, after an input procedure creator creates an input procedure list other than the input format on partially filled form data, the information processing device 1 executes the input procedure creation process to complete the input procedure list. In addition, the information processing device 1 copies the completed input procedure list to unfilled form data, thereby making it possible to execute the voice input process for the unfilled form data.

[0053] (Input procedure creation process: Figure 8) In step ST1, the information processing device 1 creates an input procedure list as an input procedure 3a in which settings other than the input format have been set in response to an operation by an input procedure creator. That is, the input procedure 3a in step ST1 has no input format set.

[0054] In step ST2, the range selection unit 5 judges whether or not there remains any step among the input procedures 3a for which an input format has not been set, and if not, ends the input procedure creation process. On the other hand, if the result of the judgment in step ST2 shows that there remains any input format for which an input format has not been set, the range selection unit 5 proceeds to step ST3.

[0055] In step ST3, the range selection unit 5 selects a range 5a including multiple input fields corresponding to one or more input target items on the form data 2a in response to the operation of the cursor cs by the input procedure creator, and presses the displayed input format estimation button 5b, as shown in Fig. 9. Note that the range selection unit 5 may select a range including only one input field.

[0056] In step ST4, the input value example acquisition unit 4 performs a procedure (O 1 ,O 2 ,…,O n ) is extracted from the input procedure list. Here, n represents the number of procedures O in the selection range. In the case of FIG. 9, the extracted procedure (O 1 ,O 2 ,…,O n ) are three rows corresponding to the identifiers B1, C1, and D1 of the input fields indicated by the input target items in input procedure 3a (n=3).

[0057] In step ST5, the input value example acquisition unit 4 acquires input value examples (E 1 ,E 2 ,…,E m ) is obtained from the form data 2a. Here, m represents the number of input value examples within the selection range. In the case of FIG. 9, the extracted input value examples (E 1 ,E 2 ,…,E m) has two options: "10:30" and "12:00" (m=2).

[0058] In step ST6, the format estimation unit 6 1 ,O 2 ,…,O n ) and its input value example (E 1 ,E 2 ,…,E m ) based on the input format. The input format of the input procedure 3a is updated (set) based on the result of the format estimation process. The format estimation process of step ST6 will be described in detail later.

[0059] In step ST7, the format check unit 7 executes a format check process on the estimated and updated input format, and obtains whether or not a notification is necessary, and if a notification is necessary, obtains a list of misrecognitions to be notified. The format check process in step ST7 will be described in detail later.

[0060] In step ST8, the format check unit 7 judges whether the format check result indicates that notification is required or not, and if not, the process proceeds to step ST11. On the other hand, if the result of the judgment in step ST8 indicates that notification is required, the process proceeds to step ST9.

[0061] In step ST9, if notification is required, the format check section 7 notifies the misrecognition list as the format check result to the display section 12. The display section 12 displays the format check result.

[0062] Here, a display example in which the format check result includes an alert will be described with reference to FIGS.

[0063] When a command is erroneously detected as a result of voice recognition of an input value example, speaking the input value example results in a situation in which the command is erroneously recognized. In this case, for example, a recognition result that recognizes a command from the input value example is obtained, or the input value example is recognized simultaneously using a command recognition dictionary, and the command is obtained as a recognition result. In either case, the display unit 12 displays on the screen a message that "the command may be erroneously detected" in association with the input target item where the erroneous detection occurred, the input format (including the words in the word list in the case of a word), and the input value example. This is particularly useful in the case of a "word" or the like, because it is possible to prevent the erroneous detection of a command by changing the word to be input.

[0064] Fig. 10 shows an example of a display in which a dialog 7a is used to notify the user of the format check result indicating a command misdetection. In this example, the user is notified that the command "end" is misdetected because words such as "no abnormality" and "abnormality" conflict with the command "end". The user is also notified of the input target field in which the misdetection occurred and the estimated input format. The method of displaying the notification is not limited to this, and it is also possible to highlight the input target field in which the misdetection occurred, for example.

[0065] At the bottom of dialog 7a in Figure 10, buttons bt1 to bt3 for "Change input format," "Change command," and "Leave as is" are displayed. By operating one of buttons bt1 to bt3, you can select how to deal with the possibility of erroneous command detection.

[0066] When the creator of the input procedure operates button bt1 indicating "change input format," the display unit 12 prompts the creator to change the input format by, for example, moving the display to the corresponding input procedure section of the procedure storage sheet or displaying a dialog box for changing the input format.

[0067] Furthermore, when the input procedure creator operates the button bt2 indicating "change command", the display unit 12 automatically changes the wording of the erroneously detected command. For example, multiple candidates may be prepared for one command, and when an erroneous detection occurs, the command may be changed to one selected from the other candidates, or the wording may be changed to a similar wording by referring to word2vec or a thesaurus. When a change is made, the format check unit 7 executes a format check again to check whether the changed command will cause the same problem. When the input procedure creator selects the button bt3 indicating "as is", the display unit 12 does nothing.

[0068] On the other hand, even if the above mentioned false positives do not occur, there is a possibility of false recognition. For example, the number "9" and the alphabet "Q" are both pronounced as "kyuu" and cannot be distinguished. In addition, some words may be difficult to recognize.

[0069] Therefore, the format check unit 7 may compare the voice recognition result with the generated input value example to calculate the voice recognition rate. Since such input value example is not necessarily generated, the input value may be generated so as to always include examples that are likely to cause problems, such as "9" and "Q". If the voice recognition rate is lower than a predetermined threshold, the input procedure creator is notified of this, as in the case of a command erroneous detection. In addition to the fact that the recognition rate has decreased, the specific location where the erroneous recognition occurred can also be obtained from the comparison between the voice recognition result and the generated input value example, and these may be notified at the same time as detailed information about the problem.

[0070] FIG. 11 shows an example of a display in which a format check result indicating a drop in the recognition rate is notified in a dialog 7b. In this example, a button bt4 indicating "reselect range" is displayed instead of a button bt2 indicating "change command." In the case of a drop in the recognition rate, button bt4 is displayed because changing the selected range changes the input format and may reduce the drop in the recognition rate. In this way, the format check unit 7 can save the voice input procedure creator the trouble of actually speaking to check the operation. Also, by displaying an alert, the input procedure creator can be informed of the problem and encouraged to make improvements.

[0071] When performing format checks all at once after estimating all input formats, such an alert display can be achieved by highlighting the problematic selection range and displaying dialogs 7a and 7b below it, as shown in Figures 10 and 11.

[0072] Also, a format check may be performed each time the voice input procedure creator selects a range and estimates an input format. In this case, if there is a problem after the format check, an alert may be displayed in the same way. The method of notifying the alert is not limited to displaying the dialogs 7a and 7b, and any method may be used, such as displaying the alert in the status bar or notifying by voice.

[0073] Thereafter, in step ST10, the display unit 12 takes action on the format check result in response to an operation by the creator of the input procedure. Note that the display unit 12 may leave the format check result as it is in response to an operation by the creator of the input procedure.

[0074] In step ST11, the display unit 12 superimposes the input format and input value examples on the form data in response to the operation of the input procedure creator. During the superimposed display, the input procedure creator checks whether there is a problem with the set input format and whether there is an input procedure for which an input format has not been set.

[0075] For example, the display unit 12 may display a menu or the like, and the superimposed display may be performed by selecting a function from within the menu.

[0076] 12 is a diagram showing a display example in which an input format during an input procedure is superimposed on the form data 2a. The display unit 12 displays a menu 12a above the form data 2a. The items to be input may be displayed, for example, in color.

[0077] The input format may be displayed as is as the contents of the input procedure 3a in the input procedure storage unit 3, or the contents of the input procedure 3a may be converted and displayed so that it is easy for the input procedure creator to understand and view. For example, in the example of Fig. 12, the input format for time and numbers is displayed as is, but the input format for words is displayed as a word list linked with "|" instead of a group name.

[0078] Fig. 13 shows an example in which example input values ​​are generated from an input format and are superimposed on the form data 2a. By superimposing the example input values, it is possible to intuitively confirm what values ​​can be entered rather than just displaying the input format itself. In either case, input items for input procedures for which an input format has not been set are displayed so that this is clear. In the example in Fig. 13, the color of the item is changed and an icon is displayed.

[0079] In addition to the input format and input value examples, guidance and input sequence may be superimposed to enable confirmation of whether the entire input procedure has been set correctly. As for displaying the input sequence, the sequence number may be displayed, or the sequence may be displayed with an arrow, as shown in Figs. 14 and 15.

[0080] After step ST11, the information processing device 1 returns to step ST2, and repeatedly executes steps ST2 to ST11 until all input formats are set.

[0081] Next, the format estimation process in step ST6 will be described in detail.

[0082] (Details of the format estimation process in step ST6) The format estimation process differs depending on whether or not concatenation is allowed in the input format.

[0083] (When concatenation is not allowed: Figure 16) FIG. 16 shows a flowchart of the format estimation process when there is no connection.

[0084] In steps ST601 to ST603, the format estimation unit 6 estimates an example of an input value (E 1 ,E 2 ,…,E m ) For each of the input values, i A large classification of input formats that match G i Estimate (i≦m).

[0085] For example, input value E i If the input value E is composed of only a number, a decimal point, and a sign, such as "-12.34", then it is known to be a number. Figure 17 shows an example of such a rule for large classification. In this rule example, a regular expression is written in the condition. The priority of such a table is as follows: i If the regular expression matches the condition, the corresponding major category G i It can be regarded as:

[0086] If none of the above match, input value example E i is the bottom "word" representing the major classification G i For example, input value E i If the date is "2023 / 03 / 07", it matches the regular expression "￥d￥d￥d￥d / ￥d?￥d / ￥d?￥d" in the condition, so it is known to be a "date". The method of categorization is not limited to this, and the rules are not limited to this condition either. Also, instead of having such rules, it is possible to write a similar process as a program.

[0087] In step ST604, if the input format of OR is not permitted, the format estimation unit 6 proceeds to step ST605. Whether or not the input format of OR is permitted is determined in advance.

[0088] In step ST605, the format estimation unit 6 classifies the input format into a large category (G 1 ,G 2,…,G m ) to select one major classification G. The format estimation unit 6 stores example input values ​​that match the selected major classification G in a memory such as the data storage unit 2.

[0089] In addition, if there are multiple input values ​​in the selected range (E 1 ,E 2 ,…,E m ), there are cases where there are multiple input value examples corresponding to multiple items, and cases where there are multiple input value examples corresponding to one input target item. In either case, the major classification (G 1 ,G 2 ,…,G m ) should be combined and one major category G selected.

[0090] The input format may be unified to include all input formats, or the narrowest input format may be adopted, and input values ​​that do not match the adopted input format may be treated as invalid examples.

[0091] When deciding on an input format that is inclusive, if the major categories are the same, the details of the input format are integrated. If the major categories are different, the input format is adjusted to the most common major category, and for input value examples that do not fit that major category, the input format is re-estimated to match the selected major category.

[0092] For example, the input value example "10:30" appears to be a "time" when viewed alone, but can also be considered to be "alphanumeric characters", so if the most common major category is "alphanumeric characters", then it can be assumed to be "alphanumeric characters" and the subsequent detailed format can be inferred. If the major category does not match and re-estimation is impossible, one major category G can be selected and input values ​​that do not match it can be treated as invalid examples. Alternatively, if the OR of input formats is also used as an input format, the result connected with OR can be used as the inferred major category.

[0093] In step ST606, the format estimation unit 6 selects input value examples (E 1 ,E 2 ,…,E m) to infer and integrate details of the input format.

[0094] To supplement, once the major category G has been determined, the details of the input format are inferred using example input values ​​that correspond to each major category G. For example, if the major category G is "Numeric value (decimal)", details such as whether there is a sign such as + or -, how many characters there are before the decimal point (i.e. how many digits there are in the integer part), and how many characters there are after the decimal point (i.e. how many digits there are in the fractional part) are determined.

[0095] Also, if the major category G is "alphanumeric," an alphanumeric pattern can be obtained by replacing each character with "A" if it is an English letter, "D" if it is a number, or "S" if it is any other symbol.

[0096] If the major classification G is "word", the example input value will be used as the notation of the word. For example, add a new group to word list 2b shown in Figure 4, set a new group name, and set the example input value as the notation of the word. The pronunciation and reading content are automatically generated and set using the method described above. The detailed input format can be "word_(group name)".

[0097] In step ST607, the format estimation unit 6 estimates the input value example (E 1 ,E 2 ,…,E m ), adjust the details of the input format according to the number of arguments. This results in one input format.

[0098] To supplement, the details estimated for each input value example are integrated for each major classification. The integration of the details of the input format when the input format is determined to be all-inclusive is performed, for example, as follows.

[0099] For example, in the case of numbers, if the major category is "Number (Decimal)" and the estimated individual input formats are "Decimal_3_2 (3 digits for integer part, 2 digits for decimal part)" and "Decimal_3_1 (3 digits for integer part, 1 digit for decimal part)", the decimal part can be 1 or 2 digits. Therefore, the integrated input format will be "Decimal_3_1~2 (3 digits for integer part, 1~2 digits for decimal part)".

[0100] For example, in the case of alphanumeric characters, if the major category is "alphanumeric characters" and the estimated individual input formats are "alphanumeric characters_AAADDSA" and "alphanumeric characters_AADDDS", the third character can be either an alphabet or a number, and the last alphabet can be omitted. Therefore, the integrated input format becomes "alphanumeric characters_AA$DDSA?".

[0101] In the case of words, for example, if the major category is "word" and the estimated individual input formats are "word_1" and "word_2", a new group is created by integrating the word list groups, and this group is used as the input format. In other words, a new group "3" is created by integrating the word list of group "1" with the word list of group "2", and all words "1" and "2" are added to word group "3". The integrated format is then used as "word_3". At this time, if there are no more input formats that refer to word groups "1" or "2", the words of groups "1" and "2" are removed from word list 2b.

[0102] By integrating in this way, the final details of the input format can be obtained. Note that the integration method is not limited to this, and it is also possible to select one input format that is most popular by taking a majority vote.

[0103] On the other hand, in step ST604, if the input format OR is permitted, the format estimation unit 6 proceeds to step ST608.

[0104] In step ST608, the format estimation unit 6 performs an OR operation on each of the major classifications. For example, the format estimation unit 6 performs an OR operation on each of the major classifications (G 1 ,G 2 ,…,G m ) are taken (excluding duplicates), and the resulting classifications are linked with OR to create the OR classification G 1 |G 2 |…|G k (k≦m). The format estimation unit 6 also obtains the large classification G of ORs. 1 |G 2 |…|G k Each of the major categories G jThe input value examples corresponding to j≦k are stored in a memory such as the data storage unit 2.

[0105] After step ST608, the format estimation unit 6 calculates the large classification G that is an element of OR. j Each time, a loop process is executed from step ST609 to step ST613. The loop process includes steps ST610 to ST611 as described above, and step ST612.

[0106] In step ST610, the format estimation unit 6 j The details of the input format are determined from the corresponding input value examples. Note that step ST610 is executed in the same manner as step ST606.

[0107] In step ST611, the format estimation unit 6 adjusts the details of the input format in accordance with the number of input value examples used. Note that step ST611 is executed in the same manner as step ST607.

[0108] In step ST612, the format estimation unit 6 calculates the major classification G corresponding to the example input values. j Replace with the details of the input format. This results in the major classification G j The above loop processing of steps ST609 to ST613 is performed to obtain one input format corresponding to the large classification G j It is executed every

[0109] After step ST607 or ST613, in step ST614, the format estimation unit 6 performs the input procedure 3a. 1 ,O 2 ,…,O n )'s input format to the obtained input format.

[0110] In addition, when the range selected by the user in the range selection unit 5 is small and there are few example input values, if the input format is determined based only on the example input values, the input format may become too detailed. As mentioned above, an input format that is too detailed may impair the convenience of the user who performs voice input. For example, when only one example input value "12.34" is given, the input format becomes "Decimal_2_2".

[0111] However, depending on the voice inputter, the "0" in "12.30" may be omitted and a number such as "12.3" may be spoken. It would be fine if the input value example included "12.30" or "12.3", but this is not always the case. Conversely, there may be cases where the input value example is only "12.3", but in fact it is necessary to input "12.30".

[0112] In order to accommodate such cases, if there are few input value examples obtained within the selected range, you can relax the input format. In other words, for decimal numbers, include ones with one less or one more digit (e.g. "Decimal_M_N" → "Decimal_(M-1)~(M+1)_(N-1)~(N+1)"), and for alphanumeric characters, replace the pattern part that is only alphanumeric characters with a pattern of alphanumeric characters or numbers (e.g. "Alphanumeric_AAA" → "Alphanumeric_$$$").

[0113] Once the input format has been estimated as described above, the input format of the currently focused input procedure (the input procedure in which the range-selected item is the input target item) is updated.

[0114] This ends the format estimation process in step ST6 in which concatenation is not permitted.

[0115] (When concatenation is allowed: Figure 18) FIG. 18 shows a flowchart of the format estimation process when concatenation is present.

[0116] In steps ST621 to ST624, the format estimation unit 6 estimates the input value example (E 1 ,E 2 ,…,E m) for each of the items ST621 to ST623.

[0117] In step ST622, the format estimation unit 6 calculates an example of an input value E i Split by space and enter the example value E i Part e 1 ,e 2 ,…,e il The subscript "il" indicates the input value E i It represents the number of parts e contained in. For example, the input value E i If is divided into thirds, il=3.

[0118] In step ST623, the format estimation unit 6 calculates an example of an input value E i Part e 1 ,e 2 ,…,e il For each of the following, the corresponding major classification G 1 i ,G 2 i ,…,G il i Select .

[0119] To supplement, if you want to allow not only simple "numbers" and "words" but also combinations (concatenations and ORs) of these as input formats, first use input value example E. i Separate each part e with a space. 1 ,e 2 ,…,e il Classification G that matches 1 i ,G 2 i ,…,G il i Search for.

[0120] For example, input value example E i If "2023 / 03 / 07 Tue 13:00", then the "2023 / 03 / 07" part e 1 The major category of "dates" is G 1 i And the "fire" part e 2 The major classification of words is G 2i and the "13:00" part e il The major category of "Time" is G il i When there is only one example input value, it is sufficient to select the one with the highest priority from among the multiple major categories, and then, as described above, to infer the detailed format for each major category.

[0121] In step ST624, the format estimation unit 6 1 i ,G 2 i ,…,G il i Add the concatenation of "date", "word", and "time" to the list of major category links. For example, add "date, word, time" to the list of major category links. 1 i ,G 2 i ,…,G il i Example of input value E for the delimiter part i As a part of the "Date" to "2023 / 03 / 07" part e 1 However, the "fire" part from the word 2 However, the "13:00" part of the "Time" il Note that the "major category link" does not necessarily have to be linked, so it may also be called the "major category combination."

[0122] The format estimation unit 6 performs the above process on the input value example E i The loop process of steps ST621 to ST624 is executed so as to be repeated the same number of times.

[0123] Thereafter, in steps ST625 to ST629, the format estimation unit 6 executes a loop process if there are two or more links in the list of major classification links. The loop process includes steps ST626 to ST628.

[0124] In step ST626, the format estimation unit 6 extracts two major category links from the list of major category links. The format estimation unit 6 also extracts the two extracted major category links (G 1 i ,G 2 i ,…,G il i ),(G 1 j ,G 2 j ,…,G jl j ) and apply the differencing algorithm.

[0125] For example, when there are multiple input value examples, the contents of the major category concatenation may differ between the input value examples. For example, when the input value examples are "A12C Black 12.3", "A12M White 45.6", and "B23F Red", the candidates for the major category concatenation are "Hexadecimal, Word, Decimal", "Alphanumeric, Word, Decimal", and "Hexadecimal, Word". In response to this, the major category is selected so that there are as many matches as possible.

[0126] Two of the candidates are taken from the list of major classification links, and the problem is treated as a difference calculation problem that includes OR in the subsequence, and the major classification order can be determined using an existing difference algorithm. Note that as the difference algorithm, for example, algorithms that find differences between sequences by calculating edit distance, LCS (Longest Common Subsequence), and SES (Shortest Edit Script), which are used in diff, can be used as appropriate.

[0127] The result of the difference algorithm is either "identity", "addition", "deletion" or "substitution" for each character. The difference algorithm is run on each major category (or their OR) as a character, and the one with the smallest distance is selected.

[0128] The substitution cost between major categories is 0 if one is contained within the other (i.e., they are considered the same, such as ``Decimal | Alphanumeric'' and ``Alphanumeric'', or ``Hexadecimal'' and ``Alphanumeric''), and 1 if they are not contained within the other.

[0129] In addition, by applying the difference algorithm to the two major combinations of 'hexadecimal, word, decimal' and 'alphanumeric, word, decimal', the difference information obtained is 'substitution of hexadecimal and alphanumeric, word, decimal'.

[0130] In step ST627, the format estimation unit 6 calculates the link between the two extracted major categories (G 1 i ,G 2 i ,…,G il i ),(G 1 j ,G 2 j ,…,G jl j ) will be deleted from the list of major linked categories.

[0131] In step ST628, the format estimation unit 6 absorbs the difference using the difference information obtained as a result of applying the difference algorithm, and creates one large classification link (G 1 i,j ,G 2 i,j ,…,G i,jl i,j )

[0132] For example, the format estimation unit 6 absorbs the differences in the above-mentioned difference information "substitution of hexadecimal and alphanumeric, word, decimal" and converts the "substitution", "addition", and "deletion" parts into combinations of major categories. For example, in "substitution of hexadecimal and alphanumeric, word, decimal", if the substitution part is changed to a broader inclusion relationship, it becomes "alphanumeric, word, decimal". In this way, the inclusion relationship "substitution of XX and XX (XX is included in XX)" can be expressed as "XX".

[0133] When you come across a "substitution of ○○ and ××" that has no inclusive relationship, you can just use OR, that is, "○○|××".

[0134] Also, "Adding ○○" and "Deleting XX" can be made into optional major categories, such as "(○○)" and "(XX)".

[0135] At this time, a record is made of which portion of which input value example matches which major classification.

[0136] Thereafter, the format estimation unit 6 calculates the linkage (G 1 i,j ,G 2 i,j ,…,G i,jl i,j ) to the list of major linkages.

[0137] The format estimation unit 6 executes a loop process of steps ST625 to ST629 so as to repeat the above process until finally only one connection of the major classification remains.

[0138] Thereafter, in steps ST630 to ST633, the format estimation unit 6 calculates the remaining major classification links (G 1 i ,G 2 i ,…,G il i ) in each major classification G i A loop process is executed to obtain details of the input format for each of the inputs. The loop process includes steps ST631 to ST632.

[0139] In step ST631, the format estimation unit 6 i The set of input values ​​Ei corresponding to (e i1 ,e i2 ,…,e im ) to determine the input format details.

[0140] In the above example, the remaining major category G i The concatenation of these is "alphanumeric, word, (decimal)". Here, the major classification of "alphanumeric" is G i Example of input value E that matches i are "A12C", "A12M", and "B23F", and are the major classification of "words" G iExample of input value E that matches i is "black" and "white", and is the major classification of "Decimal" i Example of input value E that matches i The following are the major categories: i Example input value E that matches each i The detailed format can be inferred from the above. In the above example, the final result is "alphanumeric_ADDA, word_1, (decimal_2_1)", and the word list for word_1 is "black" and "white".

[0141] In step ST632, the format estimation unit 6 calculates an example of an input value E i The set of parts of (e i1 ,e i2 ,…,e im ) (Example input value used E i Then, the format estimation unit 6 adjusts the details by the obtained details. i Replace with.

[0142] The format estimation unit 6 performs the above process on each of the large classifications G i The loop process of steps ST630 to ST633 is executed so as to be repeated every time.

[0143] In step ST634, the format estimation unit 6 sets the input format obtained as above as the input format for the corresponding input procedures. In step ST634, if there are few input value examples, the input format may be relaxed as in step ST614. In any case, the format estimation process in step ST6 when concatenation is permitted is completed.

[0144] (Format check process) In the following format check process, a format check using voice synthesis will be explained. As explained in the format check unit 7, the format check process generates input value examples from the input format, generates voice data by voice synthesis, performs voice recognition using the voice recognition dictionary generated from the input format, and checks the format to see if any recognition errors occur.

[0145] FIG. 19 shows a flowchart of the format check process.

[0146] In step ST701, the format check section 7 first generates a voice recognition dictionary from the input format of interest by means of the dictionary generation section 8.

[0147] In step ST702, the format check unit 7 generates a plurality of input value examples from the estimated input format. For example, for a portion of the input value examples that can be generated from the input format from which a plurality of candidates can be generated, a candidate may be selected by a random number. For example, in the case of a "numeric value," if the input format is "Decimal_2-4_1," the number of digits in the integer part of the input value example may be selected by a random number from 2 to 4, and the number of each digit may be determined by a random number from 0 to 9 (however, the first digit is 1 to 9).

[0148] In the case of "alphanumeric characters", if the input format is "alphanumeric characters_AA$D?", one character is selected from "A-Za-z" for the first and second characters of the example input value, one character is selected from "A-Za-z0-9" for the third character, and the fourth character is determined by a random number to be included or not. If a random number is to be included as the fourth character of the example input value, one character is selected from "0-9". For other major categories, multiple example input values ​​can be generated similarly using random numbers.

[0149] In step ST703, the format check unit 7 creates an empty error recognition list. The empty error recognition list is a list including an error recognition type, an input value example, and detailed information. For example, command erroneous detection and error recognition are available as the error recognition type, as appropriate. For example, when the type is command erroneous detection, the command in question is available as detailed information. For example, when the type is error recognition, the recognition result and the location of the error are available as detailed information, as appropriate.

[0150] Thereafter, in steps ST704 to ST711, the format check unit 7 executes a loop process to perform a format check by voice recognition on each of the generated input value examples. The loop process includes steps ST705 to ST710.

[0151] In step ST705, the format check unit 7 performs voice synthesis for each of the generated input value examples, and obtains voice data as the voice synthesis result.

[0152] In step ST706, the format check unit 7 performs voice recognition on the voice data of the voice synthesis result using the generated voice recognition dictionary. At this time, the format check unit 7 performs voice recognition of the input value example and the command using the voice recognition dictionary. As the voice recognition dictionary, a first dictionary related to the input value example and a second dictionary related to the command may be used. When the first and second dictionaries are used, the format check unit 7 performs voice recognition of the input value example and the command separately. Alternatively, as the voice recognition dictionary, dictionaries related to both the input value example and the command may be used. When dictionaries related to both are used, the format check unit 7 performs voice recognition of the input value example and the command simultaneously. Note that there are three types of recognition results in step ST706: (1) when the input value example is recognized, (2) when the input value example is not recognized and the command is erroneously detected, and (3) when the command is not erroneously detected. Note that in the above case (1), the voice recognition is successful. In the above cases (2) or (3), the speech recognition has failed. Also, in the above case (2), if the command is erroneously detected, that is, if the command is obtained as the recognition result, when the input value example is spoken, the command will be erroneously recognized.

[0153] In step ST707, the format check unit 7 judges whether the text of the speech recognition result matches the input value example, and if they match, the speech recognition is successful, so the process proceeds to step ST711. If the result of the judgment is no, the speech recognition is unsuccessful, so the process proceeds to step ST708.

[0154] In step ST708, the format check unit 7 judges whether the reason for the failure of the voice recognition is erroneous detection of a command or not, and if it is erroneous detection of a command, the process proceeds to step ST709, and if not, the process proceeds to step ST710. According to step ST708, when the voice recognition fails in step ST707, the situation can be classified as to whether the failure was due to erroneous detection of a command or not.

[0155] In step ST709, since a command was recognized as the speech recognition result, the format check section 7 adds information about the erroneous command detection to the erroneous recognition list, and then proceeds to step ST711.

[0156] In step ST710, the format check section 7 adds information about the erroneous recognition to an erroneous recognition list as the recognition result of the voice recognition, since the erroneous recognition is other than the erroneous command detection, and then the process proceeds to step ST711.

[0157] In step ST711, the format check unit 7 performs the processes of steps ST705 to ST710 for all the generated input value examples, and then ends the loop process of steps ST704 to ST711.

[0158] In step ST712, the format check section 7 judges whether or not there is information on erroneous command detection in the erroneous recognition list. If there is information on erroneous command detection, the process proceeds to step ST714, and if not, the process proceeds to step ST713.

[0159] In step ST713, the format check unit 7 calculates the misrecognition rate by dividing the number of misrecognitions in the misrecognition list by the number of input value examples, and judges whether the misrecognition rate is equal to or greater than a threshold. If the misrecognition rate is equal to or greater than the threshold, the format check unit 7 proceeds to step ST714, otherwise proceeds to step ST715.

[0160] In step ST714, the format check unit 7 determines that notification of the format check results is necessary because there is information of a command misdetection or the misrecognition rate is high, and notifies the display unit 12 of the misrecognition list corresponding to the format check results, and ends the process.

[0161] In step ST715, since the misrecognition list is empty or the misrecognition rate is low, the format check section 7 determines that it is not necessary to notify the result of the format check and ends the process.

[0162] (Voice input processing) Based on the input procedure 3a obtained by the input procedure creation process, the information processing device 1 performs a voice input process together with another voice input person, as shown in FIG.

[0163] In step ST21, the recognition control unit 11 of the information processing device 1 adds a flag (=implementation flag) indicating whether or not each step number of the input procedure 3a has been implemented, and sets all implementation flags to OFF.

[0164] In step ST22, the recognition control unit 11 judges whether or not the implementation flag of the procedure number is OFF in the input procedure 3a, and if not, ends the voice input process. On the other hand, as a result of the judgment in step ST22, if the procedure number with the implementation flag OFF remains in the input procedure 3a, the recognition control unit 11 proceeds to step ST23.

[0165] In step ST23, the recognition control unit 11 extracts from the input procedure 3a the procedure corresponding to the row with the earliest procedure number among the procedure numbers whose implementation flags are OFF.

[0166] In step ST24, the dictionary generating unit 8, under the control of the recognition control unit 11, generates a voice recognition dictionary from the input format set in the extracted procedure.

[0167] In step ST25, the voice synthesis unit 10, under the control of the recognition control unit 11, synthesizes the guidance included in the extracted procedure into voice, and reproduces the obtained voice data from the speaker.

[0168] In step ST26, the voice recognition unit 9 waits for the person inputting the voice to speak under the control of the recognition control unit 11. At this time, the person inputting the voice appropriately speaks.

[0169] In step ST27, the voice recognition unit 9, under the control of the recognition control unit 11, performs voice recognition on the speech of the voice input person by using a voice recognition dictionary.

[0170] In step ST28, the recognition control unit 11 determines whether or not a command has been recognized as a result of the voice recognition, and if the command has been recognized, the process proceeds to step ST29, and if not, the process proceeds to step ST30.

[0171] In step ST29, the recognition control unit 11 executes the recognized command, and the process returns to step ST22.

[0172] In step ST30, the recognition control unit 11 performs voice synthesis on the text of the recognition result that does not include a command, and reproduces the obtained voice data from the speaker, whereby the speech input content is repeated by the speaker.

[0173] In step ST31, the recognition control unit 11 inputs a value represented by the text of the recognition result into an input field of the form data 2a, based on the identifier of the input target item included in the extracted procedure.

[0174] In step ST32, the recognition control unit 11 sets the execution flag of the procedure to ON, and returns to step ST22.

[0175] Thereafter, the recognition control unit 11 executes all the procedures included in the list of the input procedure 3a until all the execution flags are turned ON.

[0176] The above voice input process is an example, and a voice recognition dictionary is generated each time a procedure of a procedure number is performed, but the process is not limited to this. For example, a voice recognition dictionary may be generated in advance in a lump, and a dictionary required for voice recognition may be extracted and used. Also, repetition may not be required. Alternatively, after the recognition result is repeated, the system may ask the person inputting the voice a question such as "Is this OK?", and input is executed only if the result is OK, and if the result is NG, the process is executed again from the utterance of step ST26. In this case, input of an incorrect value due to erroneous recognition can be prevented.

[0177] As described above, according to the first embodiment, the input value example acquisition unit 4 acquires, for each item, one or more items and input value examples, which are information on values ​​in the input field for one or more items, from the form data 2a including input fields for voice input. The format estimation unit 6 estimates the input format of the input field based on the input value examples. In this way, by estimating the input format of the input field based on the input value examples of the input field, an appropriate input format can be determined even when a voice recognition expert is not available.

[0178] To supplement, in the conventional case, the format of values ​​that can be input is determined in advance, and speech recognition is performed based on that format. Generally, by performing speech recognition based on the input format, it is possible to reject the recognition results of speech that does not match the set input format, or to reject the recognition results that are recognized due to noise, thereby improving the accuracy of speech recognition. Therefore, it is important to set an appropriate input format.

[0179] However, in the conventional method, it is difficult for an operator other than a speech recognition expert to determine an appropriate input format, and it takes a very long time to set an appropriate input format for all input items.

[0180] In contrast, according to the first embodiment, as described above, even when a speech recognition expert is not available, an appropriate input format can be determined. Note that information regarding values ​​in the input fields is not limited to input value examples, and may be format settings, which will be described later. Even when implemented in this way, the same effects as those described above can be obtained.

[0181] According to the first embodiment, the input procedure 3a for inputting values ​​into a plurality of input fields included in the form data 2a is stored in the input procedure storage unit 3. The input procedure 3a includes an identifier for identifying the input field of the input target item included in the form data 2a and an estimated input format. Therefore, in addition to the above-mentioned effect, the input procedure 3a can be managed separately from the form data 2a. The input procedure 3a is not limited to being stored in the memory of the information processing device 1, but may be stored in a database device that can communicate with the information processing device 1, or may be stored in a separate sheet of the form data 2a. The input procedure 3a may include, as appropriate, a procedure number, guidance, a reference value, a group name of the procedure, and the like, in addition to the identifier and the input format. Even with this modification, the input procedure 3a can be managed separately from the form data 2a, as described above.

[0182] Also, according to the first embodiment, the range selection unit 5 selects a range of multiple input fields from the form data 2a. The input value example acquisition unit 4 acquires input value examples corresponding to the input fields of the selected range. The format estimation unit 6 estimates an input format common to the input fields of the range based on the corresponding acquired input value examples. Therefore, in addition to the above-mentioned effects, the configuration of selecting a range of multiple input fields can reduce the effort of acquiring input value examples compared to selecting individual input fields.

[0183] According to the first embodiment, the dictionary generation unit 8 generates a voice recognition dictionary based on the estimated input format. The recognition control unit 11 executes voice recognition for the user's speech using the voice recognition dictionary, and inputs the obtained text of the voice recognition result into an input field according to the identifier of the input field. Therefore, in addition to the above-mentioned effects, it is possible to execute voice input for the user's speech.

[0184] According to the first embodiment, the information on the value of the input field is an input value example, which is an example of the value of the input field. The input value example acquisition unit 4 acquires the value as an input value example from at least one of the form data 2a in which the value of the input field is previously input and a user interface that accepts input of a value into the input field. Therefore, in addition to the above-mentioned effects, it is possible to acquire input value examples from various sources.

[0185] According to the first embodiment, the format estimation unit 6 may estimate an input format for each input field in the range, and estimate one of the estimated input formats as a common input format. In this case, in addition to the above-mentioned effects, for example, when one of a plurality of input formats includes the other, the input formats can be integrated by adopting the narrowest input format.

[0186] According to the first embodiment, the format estimation unit 6 may estimate an input format for each input field in the range, and estimate an input format that includes multiple input formats as a common input format. In this case, in addition to the above-mentioned effects, for example, the input formats can be integrated by determining an input format that includes all of the multiple input formats.

[0187] Furthermore, according to the first embodiment, the format estimation unit 6 adjusts the estimated input format based on the number of correspondingly acquired input value examples. Therefore, in addition to the above-mentioned effects, when the selected range (number of cells) is small, the input format can be relaxed a little, such as by increasing the number of digits in the input format (allowing +1 digit), thereby maintaining user convenience.

[0188] According to the first embodiment, the input procedure 3a may further include upper and lower limit values ​​(reference values) indicating the range of values ​​in the input field. The format estimation unit 6 may adjust the estimated input format based on the upper and lower limit values ​​included in the input procedure 3a. In this case, the input format can be adjusted to be slightly looser so as to satisfy the conditions of the upper and lower limit values, so that a more appropriate input format can be estimated.

[0189] According to the first embodiment, the input procedure 3a further includes guidance regarding voice input into the input field. The format check unit 7 checks the estimated input format based on the guidance. In addition to the above-mentioned effects, the accuracy of the estimated input format can be improved by checking the estimated input format with the guidance. For example, an inappropriate input format can be checked, such as when the input format is not a time format while the wording of the guidance is "XX time". Also, for example, the input format of the input field for multiple items is estimated from the input value example, and the validity of the input format is checked, so that even if the person is not a voice recognition expert, an appropriate input format can be set, and the time required for creating the input procedure can be reduced compared to manually considering the input format.

[0190] However, the format check unit 7 may check the input format by using rules for format check. Alternatively, the format check unit 7 may be implemented as a trained model obtained by machine learning a machine learning model using existing data. Here, as the existing data, for example, a data set including input data including an input value example and an input format estimated from the input value example, and output data including a check result of the estimated input format can be used. Even if the data is modified in this way, the estimated input format can be checked, so that the accuracy of the estimated input format can be improved.

[0191] Also, according to the first embodiment, the format check unit 7 generates input value examples from the estimated input format, performs voice synthesis on the generated input value examples, and executes voice recognition on the voice data obtained by the voice synthesis using a voice recognition dictionary. The format check unit 7 checks the estimated input format by determining whether the text of the obtained voice recognition result matches the generated input value examples. Therefore, in addition to the above-mentioned effects, the configuration for checking the estimated input format by voice recognition can improve the accuracy of the estimated input format.

[0192] According to the first embodiment, the format check unit 7 may classify the situation where there is no match by determining whether or not a command is detected from the text of the speech recognition result. In this case, the validity of the input format can be checked by confirming that there is no erroneous detection of a command.

[0193] Also, according to the first embodiment, the format check unit 7 checks the input format by calculating the recognition rate or the error recognition rate of the voice recognition based on the number of judgments and the judgment result. For example, the error recognition rate may be calculated by dividing the result of erroneous recognition by the number of judgments, which is the number of input value examples. Alternatively, the recognition rate may be calculated by dividing the result of successful recognition by the number of judgments, which is the number of input value examples. In this case, in addition to the above-mentioned effects, the validity of the input format can be checked statistically.

[0194] Also, according to the first embodiment, the format check unit 7 notifies the result of checking the input format in an identifiable manner. Therefore, in addition to the above-mentioned effects, the configuration of notifying whether the situation in which the voice recognition failed is due to erroneous command detection or not can prompt the user to take an appropriate action depending on the situation in which the voice recognition failed. However, without being limited to this, the format check unit 7 may notify that there is a concern about a decrease in the recognition rate, or may notify the cause of the decrease in the recognition rate (clash with a command, a procedure with a low recognition rate, a character that is often mistaken, etc.). For example, it may notify an example of homophones such as "9" and "Q". In this way, too, it is possible to prompt the user to take an appropriate action depending on the situation in which the voice recognition failed.

[0195] According to the first embodiment, when a command is detected, the format check unit 7 may change the voice input command that matches the detected command. In this case, the conflict with the command can be resolved by changing the wording of the command, so that unexpected situations due to erroneous detection of the command can be avoided.

[0196] According to the first embodiment, the display unit 12 may display the form data 2a and display the estimated input format in the input field included in the form data 2a. Similarly, the display unit 12 may generate one or more input value examples from the estimated input format and display the generated input value examples in the input field included in the form data 2a. Similarly, the display unit 12 may generate a range of values ​​that can be input into the input field from the estimated input format and display the generated range in the input field included in the form data 2a. Therefore, in addition to the above-mentioned effects, the estimated input format and the input value examples generated from the input format are displayed on the form data 2a, so that the input position, input format, and input value examples on the form data 2a can be checked at a glance. Accordingly, the estimated input format is displayed together with the form in a highly visible manner, and the input format can be automatically checked.

[0197] To add, in the past, even after deciding on the input format, the input format and the display of the input target items were far apart, making it difficult to see at a glance, and it was difficult to judge by looking at the input format whether the set input format was really appropriate. Therefore, in the past, in order to check whether the input format was appropriate, it was necessary to actually try entering data, which took even more time.

[0198] In contrast, according to the first embodiment, as shown in FIG. 12, the input format and the items to be input are displayed in close proximity to each other, providing high visibility, and it is easy to determine by looking at the input format whether the set input format is appropriate.

[0199] According to the first embodiment, the input procedure 3a includes an identifier for identifying an input field of an input target item included in the form data 2a, an estimated input format, and an order (procedure number) for inputting values ​​into a plurality of input fields included in the form data 2a. The display unit 12 displays the form data 2a, and based on the input procedure 3a, superimposes and displays the order for inputting values ​​into a plurality of input fields included in the form data 2a. Thus, in addition to the above-mentioned effect, the user can visually confirm whether the input order has been set correctly. However, without being limited to this, the display unit 12 may superimpose and display information (guidance, reference value) in the input procedure 3a on the form data 2a. As a result, the user can visually confirm whether the information (guidance, reference value) in the input procedure 3a has been set correctly, as described above.

[0200] <Second embodiment> In the first embodiment, the range selection unit 5 selected the range for which the input format is to be estimated on the form data 2a in response to the operation of the input procedure creator. In contrast, in the second embodiment, the range selection unit 5 automatically selects a range, and automatically estimates the input format of the selected range and sets it in the input procedure 3a. That is, unlike the first embodiment, the range selection unit 5 automatically estimates multiple ranges for which the same input procedure should be set.

[0201] Accordingly, as shown in FIG. 21, in addition to the functions described above, the range selection unit 5 performs at least one of range selection using guidance in the input procedure 3a and range selection using the position of the input target item in the input procedure 3a.

[0202] The range selection unit 5 selects the range of input fields based on the guidance, such as selecting the range of input fields corresponding to the same guidance wording. Specifically, the range selection unit 5 may set the range of input fields corresponding to the same guidance included in the input procedure 3a as one range.

[0203] In the case of range selection using item positions, the range selection unit 5 selects the range of input fields based on the situation where the identifiers of the input fields of the input target items indicate nearby positions. Specifically, when the identifiers of the input target items included in the input procedure 3a indicate adjacent input fields, the range selection unit 5 may set the range of the input fields identified by the identifiers as one range. However, it is preferable to obtain input value examples from the input fields of the input target items, estimate the input format of one of the input value examples by the format estimation unit 6, and include it in the range of the input fields only when there are N or more adjacent input fields with the same input format (N is a predetermined threshold).

[0204] Figure 22 shows the form data 2a, examples of input values, a list of input procedures, and the ranges selected based on them. For example, B1-D1, B2-D2, ..., B5-D5 have the same guidance ("inspection time", "starting current", ..., "appearance"), so they are set to one range rg1, rg2, ..., rg5. For B8-D9, B8-B9, C8-C9, D8-D9 have the same guidance ("noise, position 1", ..., "noise, position 3"), so they are set to one range rg81, ..., rg83, as shown by the dashed line. Furthermore, the item with the input format of decimal_2_1 estimated from the input value example that has already been entered is adjacent to B8-D8, so when all are combined, B8-D9 becomes one range rg89, as shown by the dashed line.

[0205] The other configuration is similar to that of the first embodiment.

[0206] Next, the operation of the information processing device configured as above will be described with reference to the flowchart of FIG.

[0207] Now, in the same manner as described above, step ST1 is executed to create an input procedure other than the input format.

[0208] After step ST1, in step ST3a, the range selection unit 5 selects the range of input fields based on, for example, guidance. The range selection unit 5 also selects the range of input fields based on, for example, the identifier of the input target item included in the input procedure 3a. Specifically, for example, when the identifier of the input target item indicates an adjacent input field, the range selection unit 5 selects the range of the adjacent input field. At this time, the input target item corresponding to the input field that does not belong to any range is set to a range consisting of a single item.

[0209] After step ST3a, a loop process in the selected range (selected range) is executed from step ST3b to step ST3d. The loop process includes steps ST4 to ST7 similar to those described above, and step ST3c.

[0210] Step ST4 is a process of extracting procedures within the selected range from the input procedures 3a.

[0211] Step ST5 is a process of acquiring an example of input values ​​for the extracted procedure from the form data 2a.

[0212] Step ST6 is a process of estimating the input format from the acquired input value examples.

[0213] Step ST7 is a process for checking the estimated input format.

[0214] After step ST7, in step ST3c, the format estimation unit 6 adds up the format check processing results without notifying the user immediately even if the format check result indicates that notification is required. That is, the format estimation unit 6 adds up the number of command erroneous detections and the number of erroneous recognitions in the format check processing, and merges the erroneous recognition list.

[0215] After the loop process for each range, the format check unit 7 executes steps ST8 and ST9 as described above. As a result, the format check unit 7 refers to the addition result of the format check process, and notifies the input procedure creator from the display unit 12 whether or not there is a command erroneous detection, and whether or not the erroneous recognition rate is equal to or higher than a threshold (step ST9).

[0216] Thereafter, the processing from step ST10 onwards is executed in the same manner as described above, whereby the processing for creating the input procedure is completed.

[0217] As described above, according to the second embodiment, the input procedure 3a further includes guidance including the item name of the item to be input. The range selection unit 5 selects the range of the input field based on the guidance or the identifier. Therefore, in addition to the above-mentioned effects, the range to which the same input format should be set is automatically selected, so that the time required for setting the input procedure 3a can be further shortened.

[0218] <Third embodiment> In the first embodiment, the input format is estimated based on an example of an input value, whereas in the third embodiment, the input format is estimated using a display format setting already set in the form data 2a.

[0219] Accordingly, the information processing device 1 includes a format setting acquisition unit 13 instead of the input value example acquisition unit 4, as shown in FIG.

[0220] The format setting acquisition unit 13 acquires the format setting of the selected range from the form data 2a. The format setting is a setting used for displaying values ​​in input fields. For example, a function provided in an existing form editing application may be used as the format setting acquisition unit 13. For example, in Excel (registered trademark), it is possible to select whether to display values ​​as numbers or as date and time when displaying values ​​as the format of a cell (input field). The format setting acquisition unit 13 is an example of an information acquisition unit that acquires the format setting included in a recording data sheet as information related to values ​​in input fields.

[0221] On the other hand, the format estimation unit 6 estimates the input format based on the acquired format setting instead of the input value example. For example, the format estimation unit 6 can mechanically convert one format setting into one input format. That is, if the format is "numeric value (number of decimal places N)", the input format is "Decimal_1~_N", if the format is "Date (2012 / 3 / 14)", the input format is "Date_year / month / day", and so on. Since there is one format setting for each item, one input format is obtained for each item. After that, integration of multiple input formats and adjustment according to the selection range are performed in the same manner as in the first embodiment.

[0222] The other configurations are similar to those of the first embodiment.

[0223] According to the above configuration, as shown in FIGS. 25, 26, and 27, the input value example (E 1 ,E 2 ,…,E m ) instead of Formatting (F 1 ,F 2 ,…,F m ), the input procedure creation process, the format estimation process, and the format check process are executed in the same manner as described above.

[0224] As described above, according to the third embodiment, in the form data 2a, information on the value of the input field for voice input is the format setting used to display the value of the input field. The format setting acquisition unit 13 acquires the format setting included in the form data 2a as information on the value of the input field for voice input. Therefore, in addition to the above-mentioned effects, there is no need to separately prepare examples of input values.

[0225] In the third embodiment, instead of the input value example, which is an example of the value of the input field, the format setting used to display the value of the input field is acquired from the form data 2a, but this is not limited to this. For example, in the third embodiment, the format setting acquisition unit 13 may acquire information including the input value example and the format setting from the form data 2a. In this case, the format setting acquisition unit 13 is an example of an information acquisition unit that acquires information including the input value example and the format setting (information regarding the value of the input field) from a recording data sheet. According to such a modified example, the effects of the first and third embodiments can be obtained at the same time.

[0226] <Fourth embodiment> In addition to the first embodiment, the fourth embodiment presents an actual operation example to the creator of an input procedure to check whether the set input procedure operates correctly.

[0227] Accordingly, the information processing device 1, as shown in FIG. 28, further includes an operation example display unit 14 in addition to the configuration shown in FIG.

[0228] Here, the operation example display unit 14 displays an operation example by automatically generating input values ​​and automatically generating voice data using voice synthesis, instead of the voice input person speaking. To supplement, the operation example display unit 14 generates an input value example from the estimated input format, voice synthesizes the generated input value example, and provides the voice data obtained by voice synthesis to the recognition control unit 11 instead of the user's speech, thereby displaying an operation example of voice input by the recognition control unit 11. The operation example may be saved as a video and displayed by playing the video, or may be displayed superimposed on the form data 2a. After displaying the operation example superimposed on the form data 2a, the operation example display unit 14 deletes all the values ​​input into the form data 2a as an operation example in response to the operation of the input procedure creator.

[0229] The other configuration is similar to that of the first embodiment.

[0230] Next, the operation of the information processing device configured as above will be described with reference to Fig. 29. This operation differs from the voice input process shown in Fig. 20 in that, instead of the user's utterance and voice recognition (steps ST26 and ST27), an input value example is automatically generated from the input format, voice-synthesized, and the synthesized voice data is played back and voice-recognized. The following will explain each step in order.

[0231] Steps ST21 to ST25 are executed in the same manner as described above.

[0232] After step ST25, in step ST26a, the operation example display unit 14 generates input value examples from the estimated input format.

[0233] In step ST26b, the operation example display unit 14 performs voice synthesis on the generated input value examples to obtain voice data.

[0234] In step STS26c, the operation example display unit 14 provides the obtained voice data to the recognition control unit 11. The recognition control unit 11 executes voice recognition using a voice recognition dictionary, and inputs the obtained text of the voice recognition result into an input field according to an identifier that identifies the input field of the input target item. By providing the voice data to the recognition control unit 11, the operation example display unit 14 displays an operation example of voice input by the recognition control unit 11.

[0235] Thereafter, the processes from step ST28 onwards are executed in the same manner as described above.

[0236] As described above, according to the fourth embodiment, the operation example display unit 14 generates input value examples from the estimated input format, voice-synthesizes the generated input value examples, and provides the voice data obtained by the voice synthesis to the recognition control unit 11 in place of the user's speech, thereby displaying an operation example of voice input by the recognition control unit 11. Therefore, in addition to the effects of the first embodiment, an actual operation example is displayed and played back to show how the set input procedure works, so that it is easy to check whether the input procedure is appropriate.

[0237] <Fifth embodiment> The fifth embodiment is a specific example of each of the above embodiments and each of the modified examples, and is a form in which the above-mentioned information processing device 1 is realized by a computer. The information processing device 1 may be realized by either a general-purpose computer such as a personal computer, or a dedicated computer such as an inspection device (embedded system).

[0238] 30 is a block diagram illustrating a hardware configuration of an information processing device 1 according to a fifth embodiment. The information processing device 1 includes a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, a ROM (Read Only Memory) 33, a storage 34, a display device 35, an input device 36, and a communication device 37, which are connected to each other via a bus.

[0239] The CPU 31 is a processor that executes arithmetic processing and control processing according to a program. The CPU 31 executes the processing of each part of the information processing device 1 described above in cooperation with the programs stored in the ROM 33 and the storage 34, etc., using a predetermined area of ​​the RAM 362 as a working area. Each of the CPU 31 and the processor may be called a processing circuit.

[0240] The RAM 32 is a memory such as a Synchronous Dynamic Random Access Memory (SDRAM), and functions as a working area for the CPU 31.

[0241] The ROM 33 is a memory that stores programs and various information in a non-rewritable manner.

[0242] The storage 34 is a device that writes and reads data to a magnetic recording medium such as a hard disk drive (HDD), a semiconductor storage medium such as a flash memory, a magnetically recordable storage medium such as a HDD, an optically recordable storage medium, etc. The storage 34 writes and reads data to and from the storage medium in response to control from the CPU 31. The storage 34 is an example of a computer memory.

[0243] The display device 35 is a display such as an LCD (Liquid Crystal Display), etc. The display device 35 displays various information based on a display signal from the CPU 361.

[0244] The input device 36 is an input device such as a mouse, a keyboard, etc. The input device 36 receives information input by a user as an instruction signal, and outputs the instruction signal to the CPU 31.

[0245] The communication device 37 communicates with external devices via a network under the control of the CPU 31. The communication device 37 may be called a communication circuit.

[0246] The instructions shown in the processing procedures shown in the above-mentioned embodiments can be executed based on a program, which is software. A general-purpose computer system can store this program in advance and obtain the same effect as the effect of the control operation of the information processing device 1 described above by reading this program. The instructions described in the above-mentioned embodiments are recorded as a program that can be executed by a computer on a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, Blu-ray (registered trademark) Disc, etc.), a semiconductor memory, or a non-transitory recording medium similar thereto. The recording medium may be in any storage format as long as it is a recording medium that can be read by a computer or an embedded system. The storage medium may be called a non-transitory computer readable storage medium. The computer can realize an information processing method involving an operation similar to the control of the information processing device 1 of the above-mentioned embodiment by reading the program from this recording medium and executing the instructions described in the program based on this program with a CPU. For example, the information processing method includes obtaining, for each item, information about one or more items and values ​​of the input fields for the one or more items from form data 2a including input fields for voice input, and estimating an input format of the input field based on the information. Of course, when the computer obtains or reads the program, it may obtain or read it through a network.

[0247] In addition, an OS (operating system), database management software, network, or other MW (middleware) running on a computer may execute some of the processes required to realize this embodiment based on instructions from a program installed on the computer or embedded system from a recording medium.

[0248] Furthermore, the recording medium in this embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like.

[0249] Furthermore, the number of recording media is not limited to one, and cases in which the processing in this embodiment is executed from multiple media are also included in the recording media in this embodiment, and the media may have any configuration.

[0250] The computer or embedded system in this embodiment is for executing each process in this embodiment based on a program stored in a recording medium, and may be configured as any one of a device such as a personal computer or a microcomputer, or a system in which multiple devices are connected to a network.

[0251] In addition, the computer in this embodiment is not limited to a personal computer but also includes an arithmetic processing device, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions in this embodiment by a program.

[0252] (Modifications of each embodiment) Each of the embodiments and modifications may be expressed as an information processing method or an information processing program including each step of the information processing device 1 described above.

[0253] According to at least one of the embodiments described above, even if a speech recognition expert is not available, it is possible to determine an appropriate input format. This also applies to at least one of the modified examples described above.

[0254] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0255] 1...information processing device, 2...data storage unit, 2a...report data, 3...input procedure storage unit, 3a...input procedure, 4...input value example acquisition unit, 5...range selection unit, 6...format estimation unit, 7...format check unit, 8...dictionary generation unit, 9...speech recognition unit, 10...speech synthesis unit, 11...recognition control unit, 12...display unit, 13...formatting acquisition unit, 14...operation example display unit.

Claims

1. an information acquisition unit that acquires, for each item, information about one or more items and values ​​of the input fields for the one or more items from a recording data sheet that includes an input field for voice input; a format estimation unit that estimates an input format of the input field based on the information; An information processing program that enables a computer to realize the above.

2. an input procedure for inputting values ​​into a plurality of input fields included in the recording data sheet is stored in a memory of the computer; the input procedure includes an identifier for identifying an input field of an input target item included in the recording data sheet, and the estimated input format. The information processing program according to claim 1 .

3. a range selection unit for selecting ranges of a plurality of input fields from the recording data sheet, the information acquisition unit acquires the information corresponding to the input field in the selected range, The format estimation unit estimates an input format common to the input fields in the range based on the corresponding acquired information. The information processing program according to claim 2.

4. The input procedure further includes guidance including the name of the item to be input, The range selection unit selects a range of the input field based on the guidance or the identifier. The information processing program according to claim 3.

5. a dictionary generation unit that generates a speech recognition dictionary based on the estimated input format; a recognition control unit that executes voice recognition on a user's speech using the voice recognition dictionary and inputs a text of the obtained voice recognition result into the input field according to the identifier; The information processing program according to claim 2 , further causing the computer to execute the following:

6. the information is example input values ​​that are examples of values ​​in the input field, the information acquiring unit acquires, as the example input value, the value from at least one of the recording data sheet in which the value of the input field has been input in advance and a user interface that accepts input of the value into the input field; The information processing program according to claim 1 .

7. the information being a format used to display values ​​in the field; the information acquisition unit acquires the format setting included in the recording data sheet as the information; The information processing program according to claim 1 .

8. the information includes example input values ​​that are examples of values ​​for the input field and formatting used to display the values ​​for the input field; the information acquisition unit acquires the information including the input value examples and the format setting from the recording data sheet; The information processing program according to claim 1 .

9. the format estimation unit estimates an input format for each input field in the range, and estimates one input format among the estimated input formats as the common input format. The information processing program according to claim 3.

10. the format estimation unit estimates an input format for each input field in the range, and estimates an input format including a plurality of the estimated input formats as the common input format. The information processing program according to claim 3.

11. The format estimation unit adjusts the estimated input format based on the number of the corresponding acquired pieces of information. The information processing program according to claim 3.

12. the input step further includes upper and lower limit values ​​indicating a range of values ​​in the input field, the format estimation unit adjusts the estimated input format based on upper and lower limit values ​​included in the input procedure. The information processing program according to claim 3.

13. A format check unit is further implemented in the computer, The input procedure further includes guidance regarding voice input into the input field, The format check unit checks the estimated input format based on the guidance. The information processing program according to claim 2.

14. a format checking unit that checks the estimated input format by generating example input values ​​from the estimated input format, performing speech synthesis on the generated example input values, executing speech recognition on speech data obtained by the speech synthesis using the speech recognition dictionary, and determining whether or not a text of the obtained speech recognition result matches the generated example input values; The information processing program according to claim 5 , further causing the computer to execute the following:

15. the format check unit classifies the situation of no match by determining whether a command is detected from the text of the speech recognition result as a result of the determination. The information processing program according to claim 14.

16. the format check unit checks the input format by calculating a recognition rate or a misrecognition rate of the voice recognition based on the number of times the determination is made and a result of the determination. The information processing program according to claim 14.

17. The format check unit notifies a result of checking the input format in an identifiable manner. The information processing program according to claim 14.

18. when the command is detected, the format check unit changes the voice input command to match the command in response to a user operation. The information processing program according to claim 15.

19. a display unit that displays the recording data sheet and displays the estimated input format in an input field included in the recording data sheet; The information processing program according to claim 1 , further comprising:

20. a display unit that generates one or more input value examples from the estimated input format and displays the generated input value examples in an input field included in the recording data sheet; The information processing program according to claim 1 , further comprising:

21. a display unit that generates a range of values ​​that can be input into the input field from the estimated input format and displays the generated range in the input field included in the recording data sheet; The information processing program according to claim 1 , further comprising:

22. A display unit is further implemented in the computer, an input procedure including an identifier for identifying an input field of an input target item included in the recording data sheet, the estimated input format, and an order for inputting values ​​into a plurality of input fields included in the recording data sheet is stored in a memory of the computer; the display unit displays the recording data sheet, and superimposes and displays an order in which the values ​​are to be input in a plurality of input fields included in the recording data sheet based on the input procedure. The information processing program according to claim 1 .

23. an operation example display unit that generates an input value example from the estimated input format, voice-synthesizes the generated input value example, and provides the voice data obtained by the voice synthesis to the recognition control unit in place of the user's speech, thereby displaying an operation example of voice input by the recognition control unit; The information processing program according to claim 5 , further causing the computer to execute the following:

24. an information acquisition unit that acquires, for each item, information about one or more items and values ​​of the input fields for the one or more items from a recording data sheet that includes an input field for voice input; a format estimation unit that estimates an input format of the input field based on the information; An information processing device comprising:

25. acquiring, for each item, information on one or more items and values ​​of the input fields for the one or more items from a recording data sheet including an input field for voice input; estimating an input format of the input field based on the information; An information processing method comprising:

Images