Human-computer interaction method, device and equipment based on voice recognition flight parameter software and medium

Abstract

The invention provides a flight parameter software man-machine interaction method and device based on voice recognition, equipment and a medium, and belongs to the technical field of avionics, and the method specifically comprises the steps: analyzing flight parameter software according to a use scene, and screening out a functional region which can be used for voice recognition and control in the flight parameter software; constructing a voice recognition model according to flight parameter data characteristics, and embedding the voice recognition model into the functional area; acquiring an audio data file of a user, and recognizing and converting the audio data file through the voice recognition model to obtain semantic text information; and transmitting the semantic text information to a semantic control center, and controlling functional areas related to the semantic text information to perform interaction work through the semantic control center. Through the processing scheme provided by the invention, operation without mouse and keyboard peripherals is realized in external field maintenance, the audio data listening time of a user is shortened in data processing, the flight parameter maintenance and data analysis efficiency is greatly improved, and the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of avionics, in particular to a flight parameter software human-computer interaction method and device based on voice recognition, equipment and medium. BACKGROUND

[0002] With the rapid development of military aircraft digital application technology, the performance, task capability and data processing demand of the aircraft are continuously improved, the scale and complexity of flight parameter data show a rapid growth trend, and a large amount of flight parameter data needs to be effectively sorted, analyzed and displayed so as to obtain valuable information and insights therefrom. How to effectively output and display flight parameter data information and form a good human-computer interaction experience to meet the data application demand of maintenance and support has become a challenging task. It is crucial to research the flight parameter software interaction method based on voice recognition and innovate the related technology research of the current flight parameter ground data processing software operation mode through the understanding of the current flight parameter software human-computer interaction status, the use of software start, text input and other specific scenes. SUMMARY

[0003] Therefore, the embodiments of the present application provide a flight parameter software human-computer interaction method based on voice recognition to solve the technical problem of low flight parameter data interpretation efficiency in the prior art. The method comprises: According to the use scene, the flight parameter software is analyzed, and the function area in the flight parameter software that can be used for voice recognition and control is screened out; According to the flight parameter data characteristics, a voice recognition model is constructed, and the voice recognition model is embedded into the function area; An audio data file of a user is acquired, the audio data file is recognized and converted through the voice recognition model, and semantic text information is obtained; The semantic text information is transmitted to a semantic control center, and the function area related to the semantic text information is controlled and interacted through the semantic control center.

[0004] The embodiments of the present application also provide a flight parameter software human-computer interaction device based on voice recognition to solve the technical problem of low flight parameter data interpretation efficiency in the prior art. The device comprises: The screening module is configured to analyze the flight parameter software according to the use scene, and screen out the function area in the flight parameter software that can be used for voice recognition and control; The voice recognition model construction module is configured to construct a voice recognition model according to the flight parameter data characteristics, and embed the voice recognition model into the function area; The recognition and conversion module is configured to acquire an audio data file of a user, recognize and convert the audio data file through the voice recognition model, and obtain semantic text information; An interaction module is configured to transmit the semantic text information to a semantic control center, and control a functional area related to the semantic text information to interact with the functional area through the semantic control center.

[0005] The embodiment of the present application also provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned any voice recognition-based flight parameter software human-computer interaction method when executing the computer program, so as to solve the technical problem of low flight parameter data interpretation efficiency in the prior art.

[0006] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program for executing the above-mentioned any voice recognition-based flight parameter software human-computer interaction method, so as to solve the technical problem of low flight parameter data interpretation efficiency in the prior art.

[0007] Advantages: The voice recognition-based flight parameter software human-computer interaction method in the embodiment of the present application is designed for flight parameter data ground processing application, and provides strong technical support for audio rapid processing interpretation of flight parameter data, parameter input, data query and starting of flight parameter software. Main advantages are as follows: 1) Reducing user analysis waiting time The present application applies advanced voice recognition technology to the field of flight parameter data audio playback, reforms the use scene of flight parameter data, changes the playback function of audio data in flight parameter data from frame by frame to one-time display of all audio data information, reduces the time of listening to audio data by the user, and greatly improves the data analysis efficiency; 2) Liberating user's hands The present application designs a voice recognition model based on flight parameter scene, constructs a flight parameter processing software system based on voice recognition control, changes the operation control of software from traditional mouse and keyboard to user voice control in the field portable auxiliary maintenance equipment, liberates the user's hands, reduces product hardware materials, and is convenient for user field operation support; 3) Improving user experience The flight parameter data processing software can realize rapid interpretation of audio data based on voice recognition, rapid input of parameter name and data name, improve flight parameter operation response speed, and improve user experience. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 A flow chart of a hybrid voice recognition and control human-machine interaction process for flight parameter maintenance in a PMA according to an embodiment of the present application; Figure 2 A flow chart of a hybrid voice recognition and control human-machine interaction process for flight parameter data processing and analysis in a flight parameter ground station according to an embodiment of the present application; Figure 3 A flow chart of a process for applying voice noise reduction to flight parameters according to an embodiment of the present application; Figure 4 A flow chart of a process for flight parameter voice semantic recognition according to an embodiment of the present application; Figure 5 A flow chart of a process for flight parameter voice recognition model training, recognition and iteration according to an embodiment of the present application. DETAILED DESCRIPTION

[0010] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0011] The above embodiments are only some of the embodiments of the present application, but not all of them. The present application can be implemented or applied in other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0012] It should be noted that various aspects of the embodiments described below are within the scope of the appended claims. It should be apparent that the aspects described herein can be implemented in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that an aspect described herein can be implemented both as any number of software and / or hardware structures and as any number of processes and / or operations. For example, an aspect can be implemented as a software routine running on a general purpose computer or workstation, a purpose-built computer, a networked computer, a mobile device, a device having a mobile device, or as a software routine running on a general purpose computer or workstation, a purpose-built computer, a networked computer, a mobile device, a device having a mobile device, and so on.

[0013] It is also need to be explained that the figures provided in the following embodiments only illustrate the basic concept of the present application in a schematic way, and only show the components related to the present application, not the number, shape and size of the components when actually implemented, the shape, number and ratio of each component when actually implemented can be a random change, and the component layout pattern can also be more complex.

[0014] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, one skilled in the art will understand that the described aspects can be practiced without these specific details.

[0015] The embodiment of the present application provides a kind of based on speech recognition flight parameter software man-machine interaction method, when specifically implemented, the embodiment takes flight parameter software man-machine interaction as the breakthrough point, for flight parameter software use scene demand, develop the flight parameter software man-machine interaction platform based on speech recognition, the platform includes speech recognition model, user instruction matching model, task search model, constructs complete speech recognition man-machine interaction system, realizes the speech recognition, voice control of flight parameter software, flight parameter audio recognition analysis and interpretation.

[0016] The following will be described in detail with reference to the drawings, specifically, based on speech recognition flight parameter software man-machine interaction method includes the following steps: Step 1, according to use scene, flight parameter software is analyzed, and the function area that can be used for speech recognition and control in flight parameter software is screened out; Step 2, according to flight parameter data characteristics, construct speech recognition model, embed the speech recognition model in the function area; Step 3, obtain the audio data file of user, identify and convert the audio data file by the speech recognition model, obtain semantic text information; Step 4, the semantic text information is transmitted to semantic control center, and the function area related to the semantic text information is controlled and interacted by semantic control center.

[0017] Specifically, when system design, first, according to use scene, flight parameter data processing software is classified and designed, flight parameter data processing software module based on speech recognition control is designed, and specific speech recognition model is developed according to flight parameter data characteristics, so as to realize the speech recognition capability under flight parameter scene, finally, speech recognition model is embedded in software start based on speech recognition control, parameter search, audio data rapid interpretation and aircraft data query module.

[0018] By constructing a flight parameter data processing platform based on voice recognition, the operation and use scenarios of flight parameter data processing software are combed, flight parameter software operation modules suitable for voice recognition control are formed, voice recognition models suitable for flight parameter processing scenarios are developed, and they are applied to flight parameter software functions such as flight parameter data processing software startup, parameter search, audio playback, and data query, thereby improving the efficiency of flight parameter data interpretation.

[0019] In one embodiment, the use scenarios include scenarios of maintaining the flight parameter system by using a portable auxiliary maintenance device and flight parameter data processing and analysis scenarios.

[0020] Further, the function areas available for voice recognition and control in the maintenance scenarios include software startup, device connection, maintenance self-check, real-time monitoring, interface testing, clock calibration, data download, and software shutdown, and the function areas available for voice recognition and control in the flight parameter data processing and analysis scenarios include software startup, parameter information input, search, and audio data playback analysis.

[0021] In specific implementation, the flight parameter software is mainly used for maintaining the flight parameter system and processing and analyzing data. Since flight parameter maintenance is performed in the field by using a portable auxiliary maintenance device (PMA), it is inconvenient to use a mouse and a keyboard for operation, and the maintenance functions are all displayed by mouse clicking, so the flight parameter maintenance scenarios in the flight parameter software can be controlled by voice recognition human-computer interaction, as described in Figure 1 ; for the flight parameter data processing and analysis module in the flight parameter ground station, the function areas of software startup, information input, and audio data playback analysis can be controlled by voice recognition human-computer interaction, as described in Figure 2 .

[0022] Therefore, the overall architecture of the flight parameter software is redesigned in this embodiment, a flight parameter software system based on voice recognition and traditional mouse and keyboard control is designed, and it is clear in the system which function areas are controlled by voice recognition human-computer interaction, which is the first time to realize a voice control system for flight parameter software in China.

[0023] In one embodiment, the voice recognition model uses a frequency recursive convolutional recurrent network model for noise reduction of voice data. The specific process of the voice recognition human-computer interaction method based on the flight parameter software is described in Figure 5 , first, the noise reduction model and the recognition model are trained to obtain the trained noise reduction model and the recognition model, then the voice of the user is recognized to obtain the recognized real data, and the real data is reduced and recognized by the noise reduction model and the recognition model respectively to obtain the recognition result.

[0024] In one embodiment, the voice recognition model includes an encoder and a decoder, the encoder extracts high-level feature representations from a complex spectrum graph, and the decoder reconstructs a target feature graph.

[0025] In one embodiment, the speech recognition model employs a non-autoregressive speech semantic recognition model, the speech recognition model further comprising a predictor and a sampler, the predictor predicting token quantities and generating speech embeddings using a continuous integral transmit mechanism, the sampler generating semantic embeddings using a fast language model.

[0026] In specific implementation, for the speech collection, recognition and conversion model, specifically includes: For the application scenarios of PMA and flight parameter data processing and analysis, the system designs real-time audio collection and offline audio input interface. The user's voice is collected through a microphone in the PMA device and the flight parameter ground station, and an offline audio recognition model is used to convert the audio information into text in real time, realizing real-time audio collection, recognition and conversion. Offline audio recognition is mainly for the pilot's voice data files recorded by the flight parameter system. The audio data files are input into the offline audio recognition model, and the audio data files are converted into text through the speech recognition model, realizing the recognition and conversion of the audio files recorded by the flight parameter system.

[0027] For real-time audio recognition and conversion or offline voice data file recognition and conversion scenarios, due to the existence of external large mechanical noise such as aircraft engines; at the same time in the flight parameter use scene, whether between pilots or between mechanics professional communication exists such as "hole, you, two, three" and other special numbers and "Hp, Ɵ, β, α, ψ" and other special parameter names, the corresponding audio information needs to be correctly converted into professional semantic text. Therefore, in the system, an offline audio recognition model is designed to realize audio noise reduction and audio recognition of the offline audio recognition model suitable for the flight parameter application scenario, so as to accurately convert the real-time or offline audio data between the mechanics and the pilots into the corresponding semantic text information.

[0028] Referring to Figure 3 As shown in the figure, the system uses a frequency recursive convolution cycle network (FRCRN) model for noise reduction of speech data, uses an encoder to extract high-level feature representation from a complex spectrum graph, and a decoder to reconstruct a target feature graph, thereby completing speech enhancement and noise reduction.

[0029] After completing speech enhancement and noise reduction, the relevant speech information needs to be combined with the semantic scene of the actual use of the software to realize the semantic recognition and matching of the speech information and the actual use scene. The semantic recognition of the system uses a non-autoregressive speech semantic recognition model, referring to Figure 4, the number of tokens is predicted and speech embeddings are generated using a continuous integration transmitter mechanism, semantic embeddings are generated using a fast language model (GLM) sampler, thereby enhancing the context modeling capability of the bidirectional parallel decoder. Since it does not need to recursively wait for recognition results like an autoregressive speech semantic recognition model, its offline coding speed is more than 10 times faster than an autoregressive model. Compared with general speech semantic recognition models, the model adds a predictor and a sampler module, thereby realizing efficient and accurate speech semantic recognition.

[0030] For speech model integration and control, when the correct conversion of speech information into corresponding semantic text information is completed, it is transmitted into the semantic control center of the system, and in the PMA device, the semantic control center is used to control the flight parameter maintenance module to perform software startup, device connection, maintenance self-check, clock calibration, real-time monitoring, interface test, data download and software shutdown and other function modules; in the flight parameter ground station, the semantic control center is used to control the flight parameter data processing and analysis module to perform software startup, automatic input of parameter information, search, automatic matching of technical data to control speech recognition human-computer interaction, and at the same time, for the pilot voice data file recorded by the flight parameter system, the voice data file is automatically converted into semantic text, and the key information such as "warning", "locking", "low", "prompt" and the fields such as "switching control right", "left turn" and "right turn" in user communication are extracted, labeled, and the audio data file automatic interpretation result is generated, so as to free the user's hands, and the user does not need to play back the audio data file frame by frame, saving the user's interpretation waiting time.

[0031] In one embodiment, the method further comprises: Collecting information of speech recognition control, when the semantic control center fails to correctly recognize the semantic text information delivered, the semantic text information is fed back to the speech recognition model, and the speech recognition model is retrained and iteratively upgraded.

[0032] In specific implementation, the embodiment also includes a model iterative upgrade function. The offline audio recognition model (speech recognition model) is the main function area for realizing speech recognition flight parameter software human-computer interaction control, and the correctness of semantic text conversion provides an important basis for correct execution of flight parameter software human-computer interaction control instructions. In the system, information of speech recognition control is collected, when the semantic control center fails to correctly recognize the semantic text delivered, the semantic text is fed back to the offline audio recognition model, the semantic recognition model is retrained and iteratively upgraded to form a system closed loop, so as to improve the accuracy of speech recognition human-computer interaction control of the system.

[0033] The method of the present application is based on human-computer interaction technology, speech recognition technology and matching control technology, and finally can realize the human-computer interaction operation of the flight parameter software by voice control, so as to provide better flight parameter software use experience for users and provide technical means for the ground rapid deployment guarantee of the aircraft.

[0034] In the embodiment, a computer device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned any voice recognition-based flight parameter software human-computer interaction method when executing the computer program.

[0035] Specifically, the computer device can be a computer terminal, a server or a similar computing device.

[0036] In the embodiment, a computer readable storage medium is provided, which stores a computer program for executing the above-mentioned any voice recognition-based flight parameter software human-computer interaction method.

[0037] Specifically, the computer readable storage medium includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition herein, the computer readable storage medium does not include transitory computer readable media such as modulated data signals and carriers.

[0038] Based on the same inventive concept, the voice recognition-based flight parameter software human-computer interaction device is also provided in the embodiments of the present application, as described in the following embodiments. Since the voice recognition-based flight parameter software human-computer interaction device solves the problem by the similar principle as the voice recognition-based flight parameter software human-computer interaction method, the implementation of the voice recognition-based flight parameter software human-computer interaction device can be referred to the implementation of the voice recognition-based flight parameter software human-computer interaction method, and the repeated parts will not be described herein. The term "unit" or "module" used below can be a combination of software and / or hardware that realizes the predetermined function. Although the device described in the following embodiments is preferably realized by software, the implementation of hardware or the combination of software and hardware is also possible and conceived.

[0039] The embodiment of the present application also provides a voice recognition-based flight parameter software human-computer interaction device, which comprises: A screening module is configured to analyze flight parameter software according to a use scenario, and screen out a function area in the flight parameter software that can be used for voice recognition and control; A voice recognition model construction module is configured to construct a voice recognition model according to flight parameter data characteristics, and embed the voice recognition model into the function area; An identification conversion module is configured to acquire an audio data file of a user, identify and convert the audio data file through the voice recognition model, and obtain semantic text information; An interaction module is configured to transmit the semantic text information to a semantic control center, and control a function area related to the semantic text information to work interactively through the semantic control center.

[0040] The embodiment of the present application achieves the following technical effects: The application is designed for flight parameter data ground processing, and provides strong technical support for audio rapid processing interpretation, parameter input, data query, and start of flight parameter software of the flight parameter data. The main beneficial effects are as follows: 1) Reducing user analysis waiting time The application applies advanced voice recognition technology to the field of flight parameter data audio playback, reforms the use scenario of flight parameter data, changes the playback function of audio data in flight parameter data from frame by frame to one-time display of all audio data information, reduces the time of listening to audio data by the user, and greatly improves the data analysis efficiency; 2) Liberating user hands The application designs a voice recognition model based on a flight parameter scenario, constructs a flight parameter processing software system based on voice recognition control, changes the operation control of software from a traditional mouse and keyboard to user voice control in an external field portable auxiliary maintenance device, liberates user hands, reduces product hardware materials, and is convenient for user field operation support; 3) Improving user experience The flight parameter data processing software can realize audio data rapid interpretation based on voice recognition, parameter name and data name rapid input, improve flight parameter operation response speed, and improve user experience.

[0041] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular combination of hardware and software.

[0042] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0043] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A human-computer interaction method for flight parameter software based on speech recognition, characterized in that, The method includes: Based on the usage scenarios, the flight data recorder software was analyzed, and the functional areas in the flight data recorder software that can be used for voice recognition and control were selected. A speech recognition model is constructed based on the characteristics of flight parameter data, and the speech recognition model is embedded into the functional area; The user's audio data file is obtained, and the audio data file is recognized and converted by the speech recognition model to obtain semantic text information; The semantic text information is transmitted to the semantic control center, which then controls the interactive functions related to the semantic text information.

2. The human-computer interaction method for flight parameter software based on speech recognition according to claim 1, characterized in that, The application scenarios include scenarios where portable auxiliary maintenance equipment is used to maintain the flight parameter system and scenarios where flight parameter data is processed and analyzed.

3. The human-computer interaction method for flight parameter software based on speech recognition according to claim 2, characterized in that, The functional areas available for voice recognition and control in the maintenance scenario include software startup, device connection, maintenance self-test, real-time monitoring, interface testing, clock calibration, data download, and software shutdown. The functional areas available for voice recognition and control in the flight parameter data processing and analysis scenario include software startup, parameter information input, search, and audio data playback and analysis.

4. The human-computer interaction method for flight parameter software based on speech recognition according to claim 1, characterized in that, The speech recognition model employs a frequency recursive convolutional recurrent network model to reduce noise in speech data.

5. The human-computer interaction method for flight parameter software based on speech recognition according to claim 4, characterized in that, The speech recognition model includes an encoder and a decoder. The encoder extracts high-level feature representations from a complex spectrogram, and the decoder reconstructs a target feature map.

6. The human-computer interaction method for flight parameter software based on speech recognition according to claim 5, characterized in that, The speech recognition model adopts a non-autoregressive speech semantic recognition model. The speech recognition model also includes a predictor and a sampler. The predictor uses a continuous integral emission mechanism to predict the number of tags and generate speech embeddings. The sampler uses a fast language model to generate semantic embeddings.

7. The human-computer interaction method for flight parameter software based on speech recognition according to claim 1, characterized in that, The method further includes: Collect information from speech recognition control. When the semantic control center fails to correctly recognize the transmitted semantic text information, it feeds the semantic text information back to the speech recognition model and retrains and upgrades the speech recognition model.

8. A human-computer interaction device based on speech recognition flight parameter software, characterized in that, include: The filtering module is used to analyze the flight data recorder software based on the usage scenario and filter out the functional areas in the flight data recorder software that can be used for voice recognition and control. The speech recognition model building module is used to build a speech recognition model based on the characteristics of flight parameter data and embed the speech recognition model into the functional area; The recognition and conversion module is used to acquire the user's audio data file, and to recognize and convert the audio data file through the speech recognition model to obtain semantic text information; The interaction module is used to transmit the semantic text information to the semantic control center, and control the functional areas related to the semantic text information to perform interactive work through the semantic control center.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the human-computer interaction method based on speech recognition flight parameter software as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that executes the human-computer interaction method based on speech recognition flight parameter software according to any one of claims 1 to 7.

Images