Voice control method and device for simulator teacher platform software and electronic equipment

Abstract

The invention provides a voice control method and device for simulator teacher table software and electronic equipment. The method comprises the steps that a configuration file set comprising interface control information and voice instruction information is acquired; constructing a voice instruction knowledge base including control elements and instruction action elements based on the configuration file set; a to-be-recognized voice instruction is acquired and converted into an instruction text; and based on the voice instruction knowledge base, analyzing the instruction text to determine a target control and a target instruction action, and generating a control instruction for controlling simulation parameters of the flight simulator according to the target control and the target instruction action. According to the method, a voice instruction knowledge base is constructed in advance, natural voice instructions of a user are directly analyzed and mapped into exact target controls and target instruction actions in teacher platform software, then control instructions including specific parameters can be automatically generated and executed, the bottleneck problem of operation efficiency is solved, the interaction load is remarkably reduced, and the user experience is improved. And the accuracy and reliability of control instruction issuing are ensured.

Description

Technical Field

[0001] This invention relates to the field of voice control technology, and in particular to a voice control method, device, and electronic device for simulator instructor console software. Background Technology

[0002] The instructor control console system is the main control console of the flight simulator, enabling functions such as overall simulator operation control, simulation status management, and real-time monitoring of simulation data. Instructor control systems are equipped on different levels of flight simulators, including IPT (Integrated Procedures Trainer), FTD (Flight Training Device), and FFS (Full Flight Simulator).

[0003] While the current instructor control console software has categorized and integrated the functional logic-related pages, the large number of settings pages still presents challenges such as operational efficiency bottlenecks, excessive interactive load, and the risk of accidental touches. Specifically, during multi-parameter configuration, instructors need to frequently switch pages and input various numerical or string parameters via keyboard, mouse, or touch devices. The operation process is executed sequentially in a single thread, unable to support multiple external parallel inputs, thus limiting setup efficiency. Simultaneously, instructors must frequently switch their attention between monitoring the software interface and observing student operations, making it difficult to respond quickly to student malfunctions or abnormal situations, resulting in a high cognitive load. Furthermore, during full-motion simulator operation, in environments with drastic changes in flight attitude or strong turbulence, keyboard, mouse, and touch operations are easily affected by aircraft movement, posing a risk of accidental triggering and impacting the accuracy and safety of operations. Summary of the Invention

[0004] This invention provides a voice control method, device, and electronic device for simulator instructor console software, which solves the defects in the current instructor console software, such as operational efficiency bottlenecks, excessive interactive load, and risk of accidental touches, which still exist due to the large number of setting pages, even though the functional logic related pages have been categorized and integrated.

[0005] This invention provides a voice control method for simulator instructor console software, comprising the following steps: Obtain a configuration file set that includes UI control information and voice command information; Based on the interface control information, control elements are determined; based on the voice command information, command action elements are determined; and based on the control elements and the command action elements, a voice command knowledge base is constructed. Obtain the voice command to be recognized and convert it into command text; Based on the voice command knowledge base, the command text is parsed to obtain the target control and target command action corresponding to the command text. Based on the target control and the target command action, the control command used to control the simulation parameters of the flight simulator is determined.

[0006] According to the present invention, a voice control method for simulator instructor console software is provided, wherein the voice command information includes compound command delimiters obtained from a compound command delimiter library; The step of parsing the command text based on the voice command knowledge base to obtain the target control and target command action corresponding to the command text includes: Based on the compound instruction delimiter, the instruction text is divided into sub-instruction texts; Based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

[0007] According to a voice control method for simulator instructor console software provided by the present invention, the method further includes: If no compound instruction separator word is detected in the instruction text, and if the instruction text is found to include at least two of the instruction action elements, mark the position information corresponding to each of the control element and the instruction action element in the instruction text; Based on the location information and the preset instruction boundary recognition rules, the instruction boundaries in the instruction text are determined; The instruction text is segmented based on the instruction boundary to obtain sub-instruction text; Based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

[0008] According to the voice control method of the simulator instructor console software provided by the present invention, the preset command boundary recognition rule includes at least one of the following: The end position of the unit word associated with the control element is determined as the instruction boundary; The end position of the numeric type instruction parameter is determined as the instruction boundary; wherein, the end position of the numeric type instruction parameter is not adjacent to the unit word in the instruction text; The starting position of the control element that serves as the new instruction body in the instruction text is determined as the instruction boundary; wherein, the control element of the new instruction body is adjacent to at least one instruction action element in the instruction text.

[0009] According to a voice control method for simulator instructor console software provided by the present invention, the step of determining control commands for controlling flight simulator simulation parameters based on the target control and the target command action includes: Determine the instruction type of the instruction text; the instruction type includes numeric type instructions, Boolean type instructions, and string type instructions; When the instruction type is a numeric type instruction or a string type instruction, extract the instruction parameters associated with the target instruction action from the instruction text; The control command is determined based on the target control, the target instruction action, and the instruction parameters.

[0010] According to a voice control method for simulator instructor console software provided by the present invention, the step of determining the control command based on the target control, the target command action, and the command parameters includes: When the instruction type is the numerical type instruction, the parameter value range corresponding to the target control is obtained from the voice instruction knowledge base, and the parameter value range includes a minimum value and a maximum value; If the instruction parameter is less than the minimum value, the minimum value will be used as the output parameter; If the instruction parameter is greater than the maximum value, the maximum value is used as the output parameter; If the instruction parameter is not less than the minimum value and not greater than the maximum value, the instruction parameter will be used as the output parameter; The control command is determined based on the target control, the target instruction action, and the output parameters.

[0011] According to a voice control method for simulator instructor console software provided by the present invention, the method further includes determining control commands for controlling flight simulator simulation parameters based on the target control and the target command action, and then further comprising: Send the control commands to the background simulation system of the flight simulator; Obtain the real-time status data corresponding to the target control in the background simulation system of the flight simulator; Based on the voice command knowledge base, a feedback text including the real-time status data is constructed; The feedback text is processed by speech synthesis to obtain synthesized speech, which is then broadcast.

[0012] According to a voice control method for simulator instructor console software provided by the present invention, the step of acquiring the voice command to be recognized includes: Listen to the preset voice control activation command; When the voice control activation command is detected, the voice command to be recognized is obtained.

[0013] The present invention also provides a voice control device for simulator instructor console software, comprising the following units: The acquisition unit is used to acquire a configuration file set that includes interface control information and voice command information; A construction unit is used to determine control elements based on the interface control information, determine command action elements based on the voice command information, and construct a voice command knowledge base based on the control elements and the command action elements. A conversion unit is used to acquire the voice command to be recognized and convert the voice command to be recognized into command text; The determining unit is used to parse the instruction text based on the voice instruction knowledge base to obtain the target control and target instruction action corresponding to the instruction text, and to determine the control instruction for controlling the simulation parameters of the flight simulator based on the target control and the target instruction action.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a voice control method for the simulator instructor console software as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the voice control method of the simulator instructor console software as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a voice control method for the simulator instructor console software as described above.

[0017] The present invention provides a voice control method, device, and electronic device for simulator instructor console software. The method involves acquiring a configuration file set including interface control information and voice command information; constructing a voice command knowledge base including control elements and command action elements based on the configuration file set; acquiring the voice command to be recognized and converting it into command text; and finally, parsing the command text based on the voice command knowledge base to determine the target control and target command action, and generating control commands for controlling the simulation parameters of the flight simulator accordingly. This method, by pre-constructing a voice command knowledge base including control elements and command action elements, directly parses and maps the user's natural voice commands into precise target controls and target command actions in the instructor console software. This enables the automatic generation and execution of control commands including specific parameters, replacing the cumbersome page switching and parameter input process of traditional keyboard, mouse, or touch control methods, thus solving the operational efficiency bottleneck problem. Simultaneously, it frees the instructor from continuous attention to the software interface, allowing them to focus on student responses, significantly reducing the interaction load. Furthermore, this non-contact interaction method fundamentally avoids the risk of accidental touches caused by physical contact in the turbulent environment of the simulator, ensuring the accuracy and reliability of control command issuance. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the voice control method of the simulator instructor console software provided by this invention.

[0020] Figure 2 This is the second flowchart illustrating the voice control method of the simulator instructor console software provided by this invention.

[0021] Figure 3 This is a schematic diagram of the voice control device of the simulator instructor console software provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] In related technologies, to meet the functional requirements of flight simulators, instructor console software is generally equipped with functional pages such as operating status settings, aircraft settings, position settings, environment settings (atmospheric environment settings, wind shear settings, microstorm settings, airport lighting settings), training subject settings, fault settings, communication settings (including communication settings, navigation station settings, communication station settings, and data link settings), and parameter overview. To facilitate quick parameter settings, the instructor console software design incorporates a page search bar and a multi-level structure, placing logically similar functional pages under the same main search page, making it easy to find the required parameter settings page among numerous options.

[0025] Simulators can be categorized into fixed-wing / rotor-wing simulators based on aircraft type, and into engineering / training flight simulators based on their intended use. The instructor control console software for different types of simulators exhibits significant differences in its functional pages. To adapt to the application needs of instructor control console software across different aircraft models and usage scenarios, and without requiring underlying program modifications or code compilation, the development of instructor control console software page functionalities for different flight simulators can be quickly completed through configuration file modifications. A configurable instructor control software was initially proposed. This software adopts the MVVM (Model-View-ViewModel) paradigm, separating the front-end interface from the back-end logic. The display interface is configured using front-end languages ​​such as XAML or QML, and the data interface is configured using XML. The back-end program uniformly calls the front-end XAML / QML and XML files to complete the interface display and back-end data interface interconnection. The development of front-end interface updates and interface adaptation functions for instructor console software can be quickly achieved without modifying or compiling the back-end program, meeting the adaptability development needs of instructor console software applications in different usage scenarios and on different models.

[0026] However, although the current instructor console software has categorized and integrated the functional logic related pages, there are still problems such as operational efficiency bottlenecks, excessive interactive load, and the risk of accidental touches due to the large number of settings pages.

[0027] To address the above problems, this invention provides a voice control method for simulator instructor console software. Figure 1 This is one of the flowcharts illustrating the voice control method of the simulator instructor console software provided by this invention, such as... Figure 1 As shown, the method includes steps 110, 120, 130 and 140.

[0028] Step 110: Obtain the configuration file set, which includes interface control information and voice command information.

[0029] Specifically, firstly, a configuration file set can be obtained, which includes interface control information and voice command information.

[0030] Here, the configuration file set refers to a set of data files used to define and configure the instructor console software interface, functions, and voice interaction logic. The configuration file set can be loaded when the instructor console software system starts up, or dynamically read at runtime. The format of the configuration file can be, but is not limited to, XML, JSON (JavaScript Object Notation), YAML, or INI formats, etc., and this embodiment of the invention does not impose specific limitations on them.

[0031] It should be understood that using configuration files allows for the addition or modification of voice commands or interface controls, requiring only the updating of the corresponding configuration files without recompiling the entire software, thus greatly improving the maintainability and scalability of the system.

[0032] The interface control information refers to the data describing the attributes and associated logic of each interactive control in the graphical user interface of the instructor's console software. In a specific embodiment, the instructor's console software can be developed based on the MVVM (Model-View-ViewModel) architecture. The interface control information can be stored in front-end description files such as XAML or QML, and its detailed attributes can be defined through back-end XML configuration files, such as Content.xml or Culture-cn.xml.

[0033] Here, the interface control information may include the control's unique identifier (ContentID), the control's display text (DisplayText), etc., and this embodiment of the invention does not impose specific limitations on this.

[0034] The UI control information is stored in the control information library. A Content object is created, and its members are consistent with the Content.xml configuration item elements, including at least the following:

[0035] Each Content configuration item in Content.xml is read into a Content object, and the collection of all Content objects forms a control information library.

[0036] The unique identifier of a control is used to uniquely identify it within the software. The display text of a control is the text the user sees on the interface, such as landing gear, sea conditions, or left engine failure. This text is crucial for subsequent voice command parsing. The associated command information of a control includes, for example, the setting command ID associated with the control, used to link the control to specific control actions. The associated status reporting information of a control includes, for example, the reporting parameter ID associated with the control, used to obtain the control's real-time status.

[0037] Here, voice command information refers to the data used to define the action portion of a voice command. Voice command information is typically predefined and stored in one or more configuration files, such as Action.xml.

[0038] The voice command information may include numerical type command action words, Boolean type command action words, and domain-specific command action words, etc., and the embodiments of the present invention do not specifically limit them. Numerical type command action words, such as "set to", "adjust to", "set to", "adjust to", etc., are used to control a numerical parameter. Boolean type command action words, such as "activate", "open", "start", etc., are used to set the state to true; and, for example, "close", "cancel", "stop", etc., are used to set the state to false.

[0039] Domain-specific command action words, such as relocation, airport, and runway, are used to trigger specific functional modules, such as relocation in a flight simulator.

[0040] Step 120: Determine control elements based on the interface control information, determine command action elements based on the voice command information, and construct a voice command knowledge base based on the control elements and the command action elements.

[0041] Specifically, after obtaining the configuration file set, control elements can be determined based on interface control information, command action elements can be determined based on voice command information, and a voice command knowledge base can be constructed based on control elements and command action elements.

[0042] The voice command knowledge base is a structured, in-memory dataset. It can be organized into various data structures, such as hash tables, dictionaries, object lists, or a small database, for efficient querying and matching.

[0043] After obtaining the configuration file set, it can be parsed. For example, files such as Content.xml can be traversed to extract the display text (DisplayText) of each control, and these texts can be stored in the knowledge base as control elements. At the same time, files such as Action.xml can be parsed to extract all preset command action words, and these can be stored in the knowledge base as command action elements.

[0044] In this context, control elements refer to keywords extracted from the interface control information that reflect a specific interface control. In most cases, this is the control's display text. For example, sea conditions, wind direction, and left engine malfunction are all control elements.

[0045] Here, the instruction action element is used to reflect the operational intent of the voice instruction information toward the target control. For example, setting, activating, turning off, etc., and this embodiment of the invention does not specifically limit this.

[0046] Through this step, the system integrates the scattered information originally located in different configuration files into a unified voice command knowledge base that can be used for natural language understanding.

[0047] Step 130: Obtain the voice command to be recognized and convert the voice command to be recognized into command text.

[0048] Specifically, the process involves acquiring the voice command to be recognized, which refers to the voice signal emitted by the instructor captured through an audio input device, such as a microphone.

[0049] To avoid false responses to ambient sounds, in a preferred embodiment, the method can first listen for a preset voice control activation command, also known as a wake-up word, such as "Hello, instructor's desk." Only after the system detects this activation command does it begin to formally capture subsequent voice content as the voice command to be recognized. Of course, other methods can also be used to activate voice capture, such as clicking a button on the interface or pressing a shortcut key on the physical keyboard, i.e., the Push-to-Talk method. This embodiment of the invention does not specifically limit this method.

[0050] After obtaining the voice command to be recognized, the system will call an Automatic Speech Recognition (ASR) engine to convert the voice command to be recognized into a command text. For example, when the instructor says "Set the sea state to level 3", the system will convert it into the command text "Set the sea state to level 3".

[0051] In a specific embodiment, the conversion process may further include standardizing the text, converting Chinese numerals to Arabic numerals, and obtaining the final text string InstructionDataString for output. For example, converting the Chinese numeral "thirty-five" to the Arabic numeral "35", and converting "twenty-five point five" to "25.5" to facilitate subsequent parameter extraction.

[0052] Step 140, based on the voice command knowledge base, parse the command text to obtain the target control and target command action corresponding to the command text, and determine a control command for controlling the simulation parameters of the flight simulator based on the target control and the target command action.

[0053] Specifically, based on the voice command knowledge base, the command text can be parsed to obtain the target control and target command action corresponding to the command text.

[0054] That is to say, the system searches in the command text to see if there are control elements in the knowledge base. For example, in the command text "Set the sea state to level 3", the system can find the control element "sea state" through matching. At the same time, the system searches in the command text to see if there are command action elements in the knowledge base. For example, in the same command text, the system can find the command action element "set". Through the above matching, the system obtains the target control and target command action corresponding to the command text. Among them, the target control is the software control object associated with the control element "sea state", and the target command action is the action "set".

[0055] After determining the target control and target command action, a control command for controlling the simulation parameters of the flight simulator can be determined based on the target control and the target command action.

[0056] After determining the target control and target command action, the system parses the components in the command text other than the control element and command action element through an instruction parameter extraction process to obtain specific parameter assignments. For example, the parameter value "3" is extracted from the command text "Set the sea state to level 3".

[0057] The next step is the control command construction stage, which integrates and encapsulates the internal identifiers corresponding to the target control, such as ContentID or RequestId, the type of the target command action, and the extracted command parameters to form a structured control command that conforms to the communication protocol between the instructor console and the flight simulator host. Its data format can be represented as {RequestId: 'REQ_SST', Parameter: 3}.

[0058] The flight simulator simulation parameters controlled here refer to the variables inside the simulation engine used to describe the state of the simulated world, such as integers representing sea state levels, floating-point numbers representing temperature, or Boolean values ​​indicating the switching state of the system. The final control commands are sent to the flight simulator, which will directly or indirectly modify the values ​​of the corresponding simulation parameters, thereby achieving precise control of the simulated environment or aircraft state.

[0059] The method provided in this invention involves: acquiring a configuration file set including interface control information and voice command information; constructing a voice command knowledge base including control elements and command action elements based on the configuration file set; acquiring the voice command to be recognized and converting it into command text; and finally, parsing the command text based on the voice command knowledge base to determine the target control and target command action, and generating control commands for controlling the simulation parameters of the flight simulator. This method, by pre-constructing a voice command knowledge base including control elements and command action elements, enables the direct parsing and mapping of the user's natural voice commands into precise target controls and target command actions in the instructor's console software. This allows for the automatic generation and execution of control commands including specific parameters, replacing the cumbersome page switching and parameter input process of traditional keyboard and mouse or touch control methods, thus solving the operational efficiency bottleneck problem. Simultaneously, it frees the instructor from continuous attention to the software interface, allowing them to focus on student responses, significantly reducing the interaction load. Furthermore, this non-contact interaction method fundamentally avoids the risk of accidental touches caused by physical contact in the turbulent environment of the simulator, ensuring the accuracy and reliability of control command issuance.

[0060] Based on the above embodiments, the voice command information includes compound command delimiters obtained from the compound command delimiter library; Step 140, which involves parsing the command text based on the voice command knowledge base to obtain the target control and target command action corresponding to the command text, includes: Step 141: Based on the compound instruction delimiter, the instruction text is segmented into sub-instruction texts; Step 142: Based on the control elements and command action elements in the voice command knowledge base, parse the sub-command text to obtain the target control and target command action corresponding to each sub-command text.

[0061] Specifically, the voice command information includes compound command delimiter words obtained from a compound command delimiter library. This means that when obtaining the configuration file set, in addition to obtaining interface control information and command action information, a configuration file defining compound command delimiter words will also be obtained additionally. For example, it can be named Delimiters.xml.

[0062] Among them, the compound command delimiter words are conjunctions, punctuation marks or phrases used to connect multiple independent commands in natural language. For example, these words can be "and", "simultaneously", "as well as", "then", "next", or they can be Chinese or English commas ",", semicolons ";" etc. The embodiments of the present invention do not make specific limitations on this. After these delimiter words are loaded, they are also stored in the voice command knowledge base.

[0063] Correspondingly, the instruction text can be segmented into sub-instruction texts based on the compound command delimiter words. That is, the system will use the compound command delimiter words in the voice command knowledge base as delimiters to segment the obtained complete instruction text. For example, the method of regular expressions can be adopted to find all occurrences of the delimiter words in the instruction text, and the original text string is segmented into an array containing multiple strings with these delimiter words as boundaries. For example, for the instruction text "Set the sea state to level 3, and then activate the left engine failure fault", the system will identify the compound command delimiter words ", and then", and segment it into two sub-instruction texts: "Set the sea state to level 3" and "Activate the left engine failure fault".

[0064] Then, based on the control element and command action element in the voice command knowledge base, the sub-instruction texts are parsed to obtain the target controls and target command actions corresponding to the sub-instruction texts respectively. The system will independently and sequentially execute the parsing process for each sub-instruction text generated in the previous step. That is, determine its target control (sea state) and target command action (set) for "Set the sea state to level 3", and determine its target control (left engine failure fault) and target command action (activate) for "Activate the left engine failure fault". Finally, generate the corresponding control commands for each sub-instruction text.

[0065] The method provided in this invention includes composite command delimiters obtained from a composite command delimiter library. Based on the composite command delimiters, the command text is segmented into sub-command texts. Then, based on control elements and the command action elements, the sub-command texts are parsed to obtain the target control and target command action corresponding to each sub-command text. In other words, a complex command is automatically segmented into multiple sub-commands using composite command delimiters, and the operation object and action corresponding to each sub-command are parsed out. This allows users to complete multiple related operations with a single voice input, avoiding the cumbersome process of repeated wake-up and step-by-step commands, significantly improving operational efficiency and the natural fluency of interaction.

[0066] Based on the above embodiments, the method further includes: Step 210: If no compound instruction separator word is detected in the instruction text, and the instruction text is found to include at least two instruction action elements, mark the position information corresponding to the control element and the instruction action element in the instruction text. Step 220: Based on the location information and the preset instruction boundary recognition rules, determine the instruction boundaries in the instruction text; Step 230: Segment the instruction text based on the instruction boundary to obtain sub-instruction text; Step 240: Based on the control elements and command action elements in the voice command knowledge base, parse the sub-command text to obtain the target control and target command action corresponding to each sub-command text.

[0067] Specifically, in one implementation, the system performs a compound instruction check before parsing the instruction text. This check can include: first, the system can construct a regular expression using a predefined instruction delimiter dictionary (lib_delimiters), search for matches in the instruction text, and if any delimiter is found, it is determined to be a compound instruction.

[0068] The method of this invention is mainly applicable to cases where no matching compound instruction delimiter word is found in the aforementioned checks. In this case, the system initiates a second-stage check: counting the number of action words appearing in the instruction text. Specifically, the system can use various instruction action libraries from the voice instruction knowledge base, such as lib_numeric_actions (numeric type), lib_true_actions (boolean truth value), lib_false_actions (boolean false value), lib_reposition (relocation type), and lib_datalink (data link type), to construct a combined regular expression, search for matches in the instruction text, and count the total number of matched instruction action elements. If the number of action words is greater than or equal to 2, it is also determined to be a compound instruction.

[0069] The above method involves marking the positional information of control elements and command action elements within the command text when no compound command separators are detected and the command text contains at least two command action elements. For example, for the command text "Set altitude 10,000 feet, lower landing gear," the system first checks and finds no separators such as "and" or "then." Next, the system scans the command text and uses a voice command knowledge base to identify the command action elements "set" and "lower," as well as the control elements "altitude" and "landing gear." Subsequently, the system records the positional information of these elements within the original string, including start and end position information.

[0070] Secondly, based on location information and preset instruction boundary recognition rules, the instruction boundaries in the instruction text are determined. Here, the preset instruction boundary recognition rules are a set of heuristic rules used to infer the instruction boundaries between instructions based on the positional relationships of marked elements.

[0071] Next, the command text is segmented based on command boundaries to obtain sub-command texts. Once the location of the command boundary is determined, the system performs a segmentation operation on the original command text at that location, thereby obtaining multiple independent sub-command texts. For example, the system can determine a command boundary between "ten thousand feet" and "landing gear" based on preset command boundary recognition rules, thus segmenting the original text into two sub-command texts: "set altitude to ten thousand feet" and "landing gear down".

[0072] Finally, based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

[0073] The method provided by this invention can accurately split compound instructions by analyzing the internal structure and semantic patterns of the text, even if the user does not use standard delimiters when expressing themselves. This makes human-computer voice interaction more natural, fluent, and more tolerant of errors, and reduces the requirements for teachers to memorize and use specific sentence structures.

[0074] Based on the above embodiments, the preset instruction boundary recognition rule includes at least one of the following: The end position of the unit word associated with the control element is determined as the instruction boundary; The end position of the numeric type instruction parameter is determined as the instruction boundary; wherein, the end position of the numeric type instruction parameter is not adjacent to the unit word in the instruction text; The starting position of the control element that serves as the new instruction body in the instruction text is determined as the instruction boundary; wherein, the control element of the new instruction body is adjacent to at least one instruction action element in the instruction text.

[0075] Specifically, the preset instruction boundary recognition rules include at least one of the following: The end position of the unit word associated with the control element is determined as the instruction boundary. When building the voice command knowledge base, the system can store the associations between control elements and their regular unit words; for example, altitude is associated with meters and feet; speed with sections. During parsing, when the system recognizes a unit word, such as feet, it has a high probability that an instruction regarding that control element, such as altitude, has been fully expressed. Therefore, the end position of this unit word can be considered a reliable instruction boundary. For example, in the command "Set altitude to 10,000 feet, landing gear down," the end position of "feet" is determined as the first instruction boundary.

[0076] The end position of a numeric command parameter is defined as the command boundary; specifically, the end position of a numeric command parameter must not be adjacent to a unit word in the command text. This rule applies to numeric settings without units. When the numeric parameter of a numeric command, such as "Set Sea State 3", is followed immediately by another control element or command action element instead of a unit word, the end position of that numeric parameter can be defined as the command boundary. For example, in "Set Sea State 3 Turn on Surface Power", the end position of "3" is defined as the command boundary.

[0077] The starting position of the control element that forms the new body of the instruction in the instruction text is determined as the instruction boundary; wherein, the control element of the new body of the instruction is adjacent to at least one instruction action element in the instruction text. When a new control element appears in the text and is immediately preceding or following an instruction action element, this usually marks the beginning of a new instruction. Therefore, the starting position of this new control element can be determined as the instruction boundary. For example, in "Set altitude to 10,000 feet, lander gear down," "lander gear" is a new control element, and it is immediately followed by the instruction action element "down," so the starting position of "lander gear" is determined as the instruction boundary.

[0078] In practical applications, priorities can be set for these rules. For example, the unit word rule has the highest priority to improve the accuracy of boundary recognition. This embodiment of the invention does not impose specific limitations on this.

[0079] In one specific embodiment, when no delimiter is detected in lib_delimiters and delimiter splitting fails, a method based on multi-lexicon collaboration for instruction boundary recognition and splitting is adopted. Based on the Content control information library and action lexicons such as lib_true_actions, lib_false_actions, lib_numeric_actions, lib_reposition, and lib_datalink, the positions of key elements in the text are marked. The end position of the Unit word in the Content class is used as the most reliable boundary, the appearance position of the new instruction body display name in the DisplayText class of the Content class is used as the second most reliable boundary, and the appearance position of action words in the action lexicon is used as the auxiliary boundary. Regular expressions are used to search for content matching the instruction, marking the key positions of various types of content in the text. Then, instruction boundaries are intelligently identified based on positional relationship rules (Rule 1: When a unit word is encountered, it is likely the end of an instruction; Rule 2: When a numeric type is encountered and there is no unit following it, it is considered the end of an instruction; Rule 3: When a new instruction body is encountered and there is an action word before or after it, it is considered the start of a new instruction). Finally, segmentation is performed according to the identified boundaries, splitting the matching instruction into individual instructions.

[0080] The method provided in this embodiment of the invention determines the end position of the unit word associated with the control element as the instruction boundary, and determines the end position of the numeric type instruction parameter as the instruction boundary; wherein the end position of the numeric type instruction parameter is not adjacent to the unit word in the instruction text; the starting position of the control element in the instruction text that serves as the new instruction body is determined as the instruction boundary; wherein the control element is adjacent to at least one instruction action element in the instruction text, thereby improving the accuracy and reliability of the instruction boundary.

[0081] Based on the above embodiments, step 140, which involves determining the control command for controlling the flight simulator simulation parameters based on the target control and the target command action, includes: Step 140-1: Determine the instruction type of the instruction text; the instruction type includes numeric type instructions, Boolean type instructions, and string type instructions; Step 140-2: When the instruction type is the numeric type instruction or the string type instruction, extract the instruction parameters associated with the target instruction action from the instruction text; Step 140-3: Determine the control instruction based on the target control, the target instruction action, and the instruction parameters.

[0082] Specifically, first, determine the instruction type of the instruction text. Instruction types include numeric type instructions, Boolean type instructions, and string type instructions.

[0083] Here, numerical type commands are used to set a parameter with continuous or discrete values. Their typical structure is "control element + command action element + command parameter + [unit]", for example, "set sea state to level 3". Boolean type commands are used to control a switch or activate / deactivate a state. Their typical structure is "command action element + control element", for example, "activate left engine failure", "start ground power", "shut down left engine failure", "shut down ground power", etc.

[0084] Here, string-type instructions are typically used to set text-based information, such as airport four-letter codes or data link messages. Their typical structure is "control element + instruction action element + instruction parameter", for example, "relocate to XX airport 35 right".

[0085] The system can help determine the command type by checking which preset dictionary, such as lib_numeric_actions or lib_true_actions, the command action element belongs to.

[0086] Secondly, when the instruction type is a numeric or string type instruction, the instruction parameters associated with the target instruction action are extracted from the instruction text. For Boolean type instructions, the parameter (true / false) is usually implicitly determined by the instruction action element itself, such as "start" / "stop", and does not require additional extraction. However, for numeric and string type instructions, the system needs to locate and extract the specific parameter value from the instruction text. For example, extract "3" from "set sea state to level 3"; extract "XX airport 35 right" from "relocate to XX airport 35 right".

[0087] Finally, the control command is determined based on the target control, the target command action, and the command parameters.

[0088] The method provided in this invention classifies instruction text by type, making the subsequent parameter extraction and instruction construction process more standardized and modular. Different processing strategies are adopted for different types of instructions, thereby improving the accuracy of instruction parsing.

[0089] Based on the above embodiments, step 140-3 includes: Steps 140-31: When the instruction type is the numerical type instruction, obtain the parameter value range corresponding to the target control from the voice instruction knowledge base. The parameter value range includes a minimum value and a maximum value. Steps 140-32: If the instruction parameter is less than the minimum value, the minimum value is used as the output parameter; If the instruction parameter is greater than the maximum value, the maximum value is used as the output parameter; If the instruction parameter is not less than the minimum value and not greater than the maximum value, the instruction parameter will be used as the output parameter; Steps 140-33: Determine the control instruction based on the target control, the target instruction action, and the output parameters.

[0090] Specifically, first, the parameter value range corresponding to the target control is obtained from the voice command knowledge base. This range includes a minimum and a maximum value. It should be noted that the parameter value range is read from a configuration file set, such as the aforementioned Request.xml, and associated with the corresponding control element during the construction of the voice command knowledge base. For example, for the "height" control, its parameter range might be a minimum of 0 and a maximum of 50,000 feet.

[0091] If the command parameter is less than the minimum value, the minimum value will be used as the output parameter. For example, if the command text is "set altitude to -100 meters", and the extracted parameter "-100" is less than the minimum value of 0, then the output parameter used to generate the control command will be corrected to 0.

[0092] If the command parameter is greater than the maximum value, the maximum value will be used as the output parameter. For example, if the command text is "set the height to 60,000 feet", and the extracted parameter "60000" is greater than the maximum value of 50000, then the output parameter will be corrected to 50000.

[0093] If the instruction parameter is not less than the minimum value and not greater than the maximum value, the instruction parameter will be used as the output parameter. That is, if the parameter is within the valid range, it can be used directly.

[0094] The instruction parameters are obtained from the instruction parameter library, and the steps for creating the instruction parameter library are as follows: Create a Request object whose members match the Request.xml configuration elements. It must include at least the following:

[0095] Each Request configuration item in Request.xml is read into a Request object, and the collection of all Request objects forms the control instruction library.

[0096] The method provided in this invention, by introducing a parameter value range verification and automatic correction mechanism, can effectively prevent invalid or dangerous values ​​from being input due to instructors' slips of the tongue or unfamiliarity with the parameter range, thus ensuring the stability and safety of the flight simulator simulation system and improving the robustness of the entire voice control system.

[0097] In related technologies, after the instructor completes the setup, the set value is displayed on the settings button. To confirm whether the set parameter status matches expectations, the feedback value needs to be checked. This further increases the instructor's workload, preventing them from focusing on the trainees' operations during training.

[0098] Based on the above embodiments, step 140, which involves determining the control command for controlling the flight simulator simulation parameters based on the target control and the target command action, further includes: Step 1401: Send the control command to the background simulation system of the flight simulator; Step 1402: Obtain the real-time status data corresponding to the target control in the background simulation system of the flight simulator; Step 1403: Based on the voice command knowledge base, construct feedback text including the real-time status data; Step 1404: Perform speech synthesis on the feedback text to obtain synthesized speech, and then broadcast the synthesized speech.

[0099] Specifically, first, control commands are sent to the flight simulator's background simulation system. Second, the system retrieves real-time status data corresponding to the target control from the flight simulator's background simulation system. Within a very short time after the command is sent, for example, in the next simulation frame cycle, the system proactively initiates a query request to the flight simulator's background simulation system to read the current actual value of the parameter that was just set. This step is to confirm whether the command has been successfully received and correctly executed by the flight simulator's background simulation system, and to ensure that the retrieved data is accurate, real-time status data from the simulation engine.

[0100] Secondly, based on the voice command knowledge base, feedback text including real-time status data is constructed. The system uses information from the knowledge base, such as the display text of controls (DisplayText) and the unit (VdnUnit) of the feedback parameters, to concatenate the retrieved real-time status data into a complete feedback text that conforms to natural language habits. For example, if "Sea State" is set to 3 and the value 3 is successfully read back from the simulator, the system can construct the feedback text: "Sea State, Level 3".

[0101] Finally, the feedback text is processed into synthesized speech, which is then played back. The system uses a text-to-speech (TTS) engine to convert the feedback text created in the previous step into synthesized speech, which is then played back through an audio device. Once the instructor hears a voice announcement such as "Sea state, Level 3," they can confirm that the operation was successful. If the control command fails or is invalid, a similar feedback text, such as "No valid command recognized, please try again," can be created and its corresponding synthesized speech played back.

[0102] The method provided in this invention forms a complete interactive closed loop of voice input, system execution, and voice output by adding a voice feedback step after command execution. This allows instructors to complete operations and verifications solely through hearing, without needing to shift their gaze to the software interface for secondary confirmation. This allows instructors to focus entirely on observing trainees, significantly reducing their interactive load and attention switching pressure.

[0103] Based on the above embodiments, the step 130 of obtaining the voice command to be recognized includes: Step 131: Listen to the preset voice control activation command; Step 132: When the voice control activation command is detected, the voice command to be recognized is obtained.

[0104] Specifically, firstly, it listens for preset voice control activation commands. When idle, the instructor's console software's voice module is in a low-power listening state, its goal being to continuously detect voice control activation commands (wake words) from microphone input. Voice control activation commands can be phrases that are not easily encountered in daily conversation, such as "Hello, Instructor" or "Hey, Instructor," to reduce the probability of false wake-ups.

[0105] Then, when a voice control activation command is detected, the system retrieves the voice instruction to be recognized. Once the system successfully matches the voice control activation command, it immediately switches to full voice recognition mode and begins recording the user's subsequent speech. This recorded speech is considered the voice instruction intended by the user. The system will typically provide an audio or visual cue to inform the user that the system has been activated and is listening.

[0106] The method provided in this invention employs an activation command mechanism, ensuring that the voice control function only responds to subsequent instructions when explicitly activated. This effectively prevents the system from misidentifying irrelevant speech, such as normal conversations between instructors and others or verbal guidance to trainees, as control commands. This ensures the intent and accuracy of the input voice commands, improving the reliability of the simulator instructor console software's voice control and user experience.

[0107] Based on any of the above embodiments Figure 2 This is the second flowchart illustrating the voice control method of the simulator instructor console software provided by this invention, as shown below. Figure 2 As shown, the method includes: First, a multi-dimensional information base, namely a voice command knowledge base, is constructed. The configurable instructor console software uses XAML / QML front-end files and XML interface configuration files to automatically construct the front-end interface display and back-end interface under the MVVM architecture. Each control has a ContentID number in the XAML / QML front-end page file. The back-end XML configuration files include Content.xml, Request.xml, Vdn.xml, Culture-us.xml, and Culture-cn.xml, which respectively configure control information, control command information, control feedback parameter information, control English display content, and control Chinese display content. A unique association is established between the front-end ContentID and the ContentID in Content.xml. A unique association is also established between the overall control information configuration in Content.xml and the control command information, control feedback parameter information, control English display content, and control Chinese display content. The above information reuses the information base created at the bottom layer of the instructor console, and adds a voice command action word library and a compound command separator library on top of it for command information splitting.

[0108] The system then continuously listens to the user's voice input and first determines if the voice contains a wake-up word. Only after detecting a preset wake-up word will the system process the subsequent voice command. The system then performs numerical standardization on the text after voice recognition and determines if the command is a compound command. If it is a compound command, it first performs a compound command splitting step, breaking it down into multiple independent sub-command texts. If it is a single command or after splitting, the process enters the command type classification stage, categorizing the command into numerical, Boolean, or string types. Subsequently, the system extracts command parameters according to the command type and performs strict data range validation on the extracted numerical parameters to ensure their validity. After successful validation, the system constructs and executes the control command, sending the command to the simulator host. To complete the interaction loop, the system also constructs a broadcast message based on the execution result and finally provides feedback to the user via TTS voice broadcast, thus ending a single voice interaction process.

[0109] The construction of the multidimensional information database includes not only the control information database and the instruction parameter database, but also the creation of the return parameter database. Create a Vdn object whose members match the Vdn.xml configuration elements. It must include at least the following:

[0110] Each Vdn configuration item in Vdn.xml is read into a Vdn object, and all Vdn objects are combined to form a return parameter library.

[0111] English display dictionary creation: Create a WordUS object whose members match the Culture-us.xml configuration element. It must include at least the following:

[0112] Each Word configuration item in Culture-us.xml is read into a WordUS object, and all WordUS objects are combined to form an English display dictionary.

[0113] Chinese display dictionary creation: Create a WordCN object whose members match the Culture-cn.xml configuration element. It must include at least the following:

[0114] Each Word configuration item in Culture-cn.xml is read into a WordCN object, and all WordCN objects form an information database.

[0115] Action instruction library and separator library: Based on the original configuration file of the instructor's console software, a preset action command dictionary and compound command separator library have been added.

[0116] The system reads the Action.xml action command configuration file to form a numeric type action dictionary (lib_numeric_actions = ["set to", "adjust to", "set to", "set to", "adjust to", "set to"]), a bill type action dictionary (lib_true_actions = ["activate", "open", "start", "launch"] and lib_false_actions = ["close", "cancel", "stop", "release"]), a reposition dictionary (lib_reposition=["reposition", "airport", "apron", "taxiway", "runway end", "takeoff position", "nautical mile"]), and a data link dictionary (Lib_datalink=["BeiDou", "data link"].

[0117] Read the Delimiters.xml action command configuration file to form a compound command delimiter library lib_delimiters= ["and", "simultaneously", "and", "as well as", "then", "next", "、", ",", ";", " ").

[0118] Finally, based on the target control, target instruction action, and output parameters, the control instruction is determined. The final control instruction is constructed using the output parameters, which have undergone range validation and correction. Furthermore, this step may include unit conversion; if the units in the instruction text are inconsistent with the system's internal units—for example, if the instruction text is in "meters" but the system's unit is "feet"—the output parameters are calculated and converted according to a preset conversion relationship.

[0119] Here, instruction parameter extraction includes instruction body identification, Boolean type instruction extraction, numeric type instruction extraction, and string type instruction extraction.

[0120] Command subject identification: Iterate through the Content objects in the control information library to see if the DisplayText contains instruction content. found = [c for c in lib_content_DisplayText if c in InstructionDataString]. If found is not empty, then the Content to which the DisplayText belongs is confirmed to be the subject of instruction execution.

[0121] After the instruction execution body is completed, parameters for different types of instructions are extracted according to the instruction action classification rules.

[0122] Boolean type instruction extraction: The commands `for c in lib_true_actions if c in InstructionDataString –content.displayText]` and `for c in lib_false_actions if c in InstructionDataString – content.displayText]` are used to determine if the instruction action has a matching object in the `lib_true_actions` library. If so, the instruction is a boolean instruction and its output parameter `parameterOutput` is set to `true`. If the instruction parameter has a matching object in the `lib_false_actions` library, the instruction is a boolean instruction and its output parameter `parameterOutput` is set to `false`. If no match is found, the instruction is ineffective.

[0123] Numeric type instruction extraction: The code snippet `for c in lib_numeric_actions if c in InstructionDataString –content.displayText]`, `for c in lib_reposition if c in InstructionDataString –content.displayText]`, and `for c in lib_datalink if c in InstructionDataString –content.displayText]` checks if the instruction action matches in the `lib_numeric_actions` library but not in `lib_reposition` or `lib_datalink`. If so, the instruction is a numeric type. The instruction is broken down into four parts: instruction content, instruction action, instruction parameters, and instruction unit. The instruction unit is optional, such as "set rain level to heavy rain" or "set time to daytime." The extracted parameters (e.g., "heavy rain" or "daytime") are converted to their corresponding integer values. The unit parameter defaults to empty (N / A). Instruction parameters must match the data type, data unit, and maximum / minimum value range in the instruction parameter library. The acquired instruction parameters are converted to data that meets the requirements of the data type, data unit, and maximum / minimum value range as the instruction output parameter `parameterOutput`.

[0124] Extracting string type instructions: If the content contained in the voice command is located in lib_reposition, lib_datalink relocation, or a data link dictionary, then the command is a string type command. The command content is split into two parts: "command content + command action + command parameters". The output parameter `parameterOutput` directly takes the command parameters.

[0125] Data type conversion: If the Request object in the command library corresponding to the command body of a numeric type parameter obtains a DataType data type parameter, then the command parameter is forcibly cast to the corresponding DataType. For enumeration type (int) parameters, the int value corresponding to the command parameter is used as the output parameter parameterOutput through the enumeration mapping relationship. For example, if the command parameter "Rain level set to heavy rain" has an enumeration int value of 3, then 3 will be used as parameterOutput.

[0126] Data range validation: If the Request object in the instruction library corresponding to the instruction body of the numeric type parameter obtains the Min and Max configurations, then the instruction parameter will be matched for minimum / maximum value. If it is less than the minimum value, the minimum value will be taken; if it is greater than the maximum value, the maximum value will be taken; if it is in between, the instruction parameter will be used as the output parameter parameterOutput.

[0127] Unit conversion: When the data units obtained from the Request object in the instruction library corresponding to the instruction body of the numeric type are inconsistent, the conversion ratio of the units is obtained according to the unit conversion relationship, and the instruction parameter is multiplied by the conversion ratio and used as the output parameter parameterOutput.

[0128] Compound instruction support: Compound instruction check Regular expressions are constructed using the predefined delimiter library lib_delimiters to search for matches in the text. If any delimiter is found, it is determined to be a compound instruction.

[0129] If no matching delimiter is found in the delimiter dictionary, count the number of action words appearing in the text. Use command action libraries such as lib_true_actions, lib_false_actions, lib_numeric_actions, lib_reposition, and lib_datalink to construct regular expressions and search for matches in the text. If the number of action words is greater than or equal to 2, it is determined to be a compound command.

[0130] Compound instruction splitting: The delimiter is used first for instruction splitting. After loading the lib_delimiters instruction delimiter library, regular expressions are used to match all delimiters, the original text is split according to the delimiter, empty strings and pure whitespace items are removed, and the splitting result is returned to obtain a single instruction.

[0131] When no delimiter is detected in `lib_delimiters` and delimiter splitting fails, a multi-lexicon collaborative instruction boundary recognition and splitting method is adopted. Based on the Content control information library and action lexicons such as `lib_true_actions`, `lib_false_actions`, `lib_numeric_actions`, `lib_reposition`, and `lib_datalink`, the positions of key elements in the text are marked. The end position of the Unit word in the Content class is used as the most reliable boundary, the appearance position of the new instruction body display name in the DisplayText class of the Content class is used as the second most reliable boundary, and the appearance position of action words in the action lexicon is used as the auxiliary boundary. Regular expressions are used to search for content matching the instruction, marking the key positions of various types of content in the text. Then, instruction boundaries are intelligently identified based on positional relationship rules (Rule 1: When a unit word is encountered, it is likely the end of an instruction; Rule 2: When a numeric type is encountered and there is no unit following it, it is considered the end of an instruction; Rule 3: When a new instruction body is encountered and there is an action word before or after it, it is considered the start of a new instruction). Finally, segmentation is performed according to the identified boundaries, splitting the matching instruction into individual instructions.

[0132] Command sent and executed: For commands whose main body or parameters are not recognized, a speech recognition validity flag is given. For correctly recognized speech commands, the communication API interfaces of the console and simulator host are invoked to package the matching command body information and command parameters (parameterOutput) to complete the speech command transmission from the instructor's console.

[0133] Broadcast message construction: Different voice commands are constructed based on the validity status of the speech recognition flag. If the flag is invalid, the voice broadcast message is constructed as "No valid command recognized, please try again" or other prompts, and the voice broadcast is performed directly. If the flag is valid, the host returns the parameters in the next frame after the voice command is sent from the instructor's console. The broadcast message string ReturnDataString is formed by concatenating the DisplayText of the corresponding command body Content object in the control information library, the returned parameters, and the VdnUnit of the corresponding Vdn object in the returned parameter library.

[0134] Voice broadcast: The ReturnDataString, a broadcast information string constructed by calling the audio device through TTS technology, allows the person issuing the instruction to confirm the report information through hearing.

[0135] The method provided in this embodiment of the invention allows one or more parameters to be set directly by voice, eliminating the need for flight instructors to switch between multiple setting pages or perform keyboard input operations, thus improving the efficiency of instructor station settings during training.

[0136] 2. This method uses voice to set parameters, eliminating the need for instructors to view the instructor console software page or perform settings operations. This reduces the instructor's interactive load, allows for continuous monitoring of trainees' action responses, and enables better evaluation of the trainees' operational procedures.

[0137] 3. It has a voice feedback function. After the command data set by voice is transmitted back, it will be played back by voice. The instructor does not need to take his attention to check the parameter values ​​set in the software interface. He can continue to pay attention to the trainee's action response and make a better judgment on the standardization of the trainee's operation process.

[0138] 4. This method is controlled by voice commands. Even when the aircraft is in a large pitch / roll attitude or is in a state of severe turbulence, the voice commands will not cause accidental touches as with keyboard, mouse or touch screen operations.

[0139] 5. It is compatible with the original MVVM architecture of the instructor platform, realizing the adaptation of voice commands and configurable software systems. It reuses most of the original information base. When adding new commands, the configuration file set can be configured and the software can be restarted. The command base update does not require compilation, and the zero-training cost word matching mechanism avoids the introduction of complex language models. It does not require vectorization of the language model, nor does it rely on the retraining process. Furthermore, the voice command knowledge base is updated quickly and efficiently, and the voice control commands and voice feedback commands are updated quickly and efficiently.

[0140] The voice control device of the simulator instructor console software provided by the present invention will be described below. The voice control device of the simulator instructor console software described below can be referred to in correspondence with the voice control method of the simulator instructor console software described above.

[0141] Based on any of the above embodiments, the present invention provides a voice control device for simulator instructor console software. Figure 3 This is a schematic diagram of the voice control device of the simulator instructor console software provided by the present invention, as shown below. Figure 3 As shown, the device includes: The acquisition unit 310 is used to acquire a configuration file set including interface control information and voice command information; The construction unit 320 is used to determine control elements based on the interface control information, determine command action elements based on the voice command information, and construct a voice command knowledge base based on the control elements and the command action elements. The conversion unit 330 is used to acquire the voice command to be recognized and convert the voice command to be recognized into command text; The determining unit 340 is used to parse the instruction text based on the voice instruction knowledge base to obtain the target control and target instruction action corresponding to the instruction text, and to determine the control instruction for controlling the simulation parameters of the flight simulator based on the target control and the target instruction action.

[0142] The apparatus provided in this invention acquires a configuration file set including interface control information and voice command information; constructs a voice command knowledge base including control elements and command action elements based on the configuration file set; acquires the voice command to be recognized and converts it into command text; finally, parses the command text based on the voice command knowledge base to determine the target control and target command action, and generates control commands for controlling the simulation parameters of the flight simulator accordingly. This method, by pre-constructing a voice command knowledge base including control elements and command action elements, enables the direct parsing and mapping of the user's natural voice commands into precise target controls and target command actions in the instructor's console software. This allows for the automatic generation and execution of control commands including specific parameters, replacing the cumbersome page switching and parameter input process of traditional keyboard and mouse or touch control methods, solving the operational efficiency bottleneck problem. Simultaneously, it frees the instructor from continuous attention to the software interface, allowing them to focus on student responses, significantly reducing the interaction load. Furthermore, this non-contact interaction method fundamentally avoids the risk of accidental touches caused by physical contact in the turbulent environment of the simulator, ensuring the accuracy and reliability of control command issuance.

[0143] Based on any of the above embodiments, the voice command information includes compound command delimiters obtained from the compound command delimiter library; The determining unit 340 is specifically used for: Based on the compound instruction delimiter, the instruction text is divided into sub-instruction texts; Based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

[0144] Based on any of the above embodiments, a parsing unit is further included, wherein the parsing unit is specifically used for: If no compound instruction separator word is detected in the instruction text, and if the instruction text is found to include at least two of the instruction action elements, mark the position information corresponding to each of the control element and the instruction action element in the instruction text; Based on the location information and the preset instruction boundary recognition rules, the instruction boundaries in the instruction text are determined; The instruction text is segmented based on the instruction boundary to obtain sub-instruction text; Based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

[0145] Based on any of the above embodiments, the preset instruction boundary recognition rule includes at least one of the following: The end position of the unit word associated with the control element is determined as the instruction boundary; The end position of the numeric type instruction parameter is determined as the instruction boundary; wherein, the end position of the numeric type instruction parameter is not adjacent to the unit word in the instruction text; The starting position of the control element that serves as the new instruction body in the instruction text is determined as the instruction boundary; wherein, the control element of the new instruction body is adjacent to at least one instruction action element in the instruction text.

[0146] Based on any of the above embodiments, the determining unit 340 specifically includes: A type determination unit is used to determine the instruction type of the instruction text; the instruction type includes numeric type instructions, Boolean type instructions, and string type instructions; The extraction unit is used to extract instruction parameters associated with the target instruction action from the instruction text when the instruction type is the numeric type instruction or the string type instruction; A determining subunit is used to determine the control instruction based on the target control, the target instruction action, and the instruction parameters.

[0147] Based on any of the above embodiments, the determining subunit is specifically used for: When the instruction type is the numerical type instruction, the parameter value range corresponding to the target control is obtained from the voice instruction knowledge base, and the parameter value range includes a minimum value and a maximum value; If the instruction parameter is less than the minimum value, the minimum value will be used as the output parameter; If the instruction parameter is greater than the maximum value, the maximum value is used as the output parameter; If the instruction parameter is not less than the minimum value and not greater than the maximum value, the instruction parameter will be used as the output parameter; The control command is determined based on the target control, the target instruction action, and the output parameters.

[0148] Based on any of the above embodiments, a broadcasting unit is further included, wherein the broadcasting unit is specifically used for: Send the control commands to the background simulation system of the flight simulator; Obtain the real-time status data corresponding to the target control in the background simulation system of the flight simulator; Based on the voice command knowledge base, a feedback text including the real-time status data is constructed; The feedback text is processed by speech synthesis to obtain synthesized speech, which is then broadcast.

[0149] Based on any of the above embodiments, the acquisition unit 310 is specifically used for: Listen to the preset voice control activation command; When the voice control activation command is detected, the voice command to be recognized is obtained.

[0150] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute the voice control method of the simulator instructor console software. This method includes: acquiring a configuration file set including interface control information and voice command information; determining control elements based on the interface control information, determining command action elements based on the voice command information, and constructing a voice command knowledge base based on the control elements and the command action elements; acquiring a voice command to be recognized and converting the voice command to be recognized into command text; parsing the command text based on the voice command knowledge base to obtain the target control and target command action corresponding to the command text, and determining control commands for controlling the flight simulator simulation parameters based on the target control and the target command action.

[0151] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the voice control method of the simulator instructor console software provided by the above methods. The method includes: acquiring a configuration file set including interface control information and voice command information; determining control elements based on the interface control information, determining command action elements based on the voice command information, and constructing a voice command knowledge base based on the control elements and the command action elements; acquiring a voice command to be recognized and converting the voice command to be recognized into command text; parsing the command text based on the voice command knowledge base to obtain the target control and target command action corresponding to the command text, and determining control commands for controlling the simulation parameters of the flight simulator based on the target control and the target command action.

[0153] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a voice control method for executing the simulator instructor console software provided by the methods described above. This method includes: acquiring a configuration file set including interface control information and voice command information; determining control elements based on the interface control information, determining command action elements based on the voice command information, and constructing a voice command knowledge base based on the control elements and the command action elements; acquiring a voice command to be recognized and converting the voice command to be recognized into command text; parsing the command text based on the voice command knowledge base to obtain the target control and target command action corresponding to the command text, and determining control commands for controlling the simulation parameters of the flight simulator based on the target control and the target command action.

[0154] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice control method for simulator instructor console software, characterized in that, include: Obtain a configuration file set that includes UI control information and voice command information; Based on the interface control information, control elements are determined; based on the voice command information, command action elements are determined; and based on the control elements and the command action elements, a voice command knowledge base is constructed. Obtain the voice command to be recognized and convert it into command text; Based on the voice command knowledge base, the command text is parsed to obtain the target control and target command action corresponding to the command text. Based on the target control and the target command action, the control command used to control the simulation parameters of the flight simulator is determined.

2. The voice control method for the simulator instructor console software according to claim 1, characterized in that, The voice command information includes compound command delimiters obtained from a compound command delimiter library; The step of parsing the command text based on the voice command knowledge base to obtain the target control and target command action corresponding to the command text includes: Based on the compound instruction delimiter, the instruction text is divided into sub-instruction texts; Based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

3. The voice control method for the simulator instructor console software according to claim 1, characterized in that, The method further includes: If no compound instruction separator word is detected in the instruction text, and if the instruction text is found to include at least two of the instruction action elements, mark the position information corresponding to each of the control element and the instruction action element in the instruction text; Based on the location information and the preset instruction boundary recognition rules, the instruction boundaries in the instruction text are determined; The instruction text is segmented based on the instruction boundary to obtain sub-instruction text; Based on the control elements and command action elements in the voice command knowledge base, the sub-command text is parsed to obtain the target control and target command action corresponding to each sub-command text.

4. The voice control method for the simulator instructor console software according to claim 3, characterized in that, The preset instruction boundary recognition rules include at least one of the following: The end position of the unit word associated with the control element is determined as the instruction boundary; The end position of the numeric type instruction parameter is determined as the instruction boundary; wherein, the end position of the numeric type instruction parameter is not adjacent to the unit word in the instruction text; The starting position of the control element that serves as the new instruction body in the instruction text is determined as the instruction boundary; wherein, the control element of the new instruction body is adjacent to at least one instruction action element in the instruction text.

5. The voice control method for the simulator instructor console software according to any one of claims 1 to 4, characterized in that, The step of determining the control commands for controlling the flight simulator simulation parameters based on the target control and the target command action includes: Determine the instruction type of the instruction text; the instruction type includes numeric type instructions, Boolean type instructions, and string type instructions; When the instruction type is a numeric type instruction or a string type instruction, extract the instruction parameters associated with the target instruction action from the instruction text; The control command is determined based on the target control, the target instruction action, and the instruction parameters.

6. The voice control method for the simulator instructor console software according to claim 5, characterized in that, The step of determining the control instruction based on the target control, the target instruction action, and the instruction parameters includes: When the instruction type is the numerical type instruction, the parameter value range corresponding to the target control is obtained from the voice instruction knowledge base, and the parameter value range includes a minimum value and a maximum value; If the instruction parameter is less than the minimum value, the minimum value will be used as the output parameter; If the instruction parameter is greater than the maximum value, the maximum value is used as the output parameter; If the instruction parameter is not less than the minimum value and not greater than the maximum value, the instruction parameter will be used as the output parameter; The control command is determined based on the target control, the target instruction action, and the output parameters.

7. The voice control method for the simulator instructor console software according to any one of claims 1 to 4, characterized in that, The step of determining control commands for controlling flight simulator simulation parameters based on the target control and the target command action further includes: Send the control commands to the background simulation system of the flight simulator; Obtain the real-time status data corresponding to the target control in the background simulation system of the flight simulator; Based on the voice command knowledge base, a feedback text including the real-time status data is constructed; The feedback text is processed by speech synthesis to obtain synthesized speech, which is then broadcast.

8. The voice control method for the simulator instructor console software according to any one of claims 1 to 4, characterized in that, The acquisition of the voice command to be recognized includes: Listen to the preset voice control activation command; When the voice control activation command is detected, the voice command to be recognized is obtained.

9. A voice control device for simulator instructor console software, characterized in that, include: The acquisition unit is used to acquire a configuration file set that includes interface control information and voice command information; A construction unit is used to determine control elements based on the interface control information, determine command action elements based on the voice command information, and construct a voice command knowledge base based on the control elements and the command action elements. A conversion unit is used to acquire the voice command to be recognized and convert the voice command to be recognized into command text; The determining unit is used to parse the instruction text based on the voice instruction knowledge base to obtain the target control and target instruction action corresponding to the instruction text, and to determine the control instruction for controlling the simulation parameters of the flight simulator based on the target control and the target instruction action.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the voice control method of the simulator instructor console software as described in any one of claims 1 to 8.

Images