Simulation cabin multi-modal data processing method and system for training, equipment and medium

Abstract

The invention relates to the technical field of simulated flight equipment data processing, in particular to a simulated cabin multi-modal data processing method and system for training, equipment and a medium, and mainly comprises an acquisition module which is configured to acquire a multi-modal data set needing to be transmitted by a current simulated cabin, if the data type in the current multi-modal data set only comprises text data and audio data, directly sending the data without processing; if the data type in the current multi-modal data set not only comprises the text data and the audio data, establishing a data evaluation model, and obtaining an evaluation value of each piece of data through the data evaluation model; and sorting the evaluation values to obtain a transmission sequence of each piece of data, and transmitting all the data according to the current sorting. Through the method, the types of various data in the multi-modal data set are considered, a more appropriate evaluation model is constructed, a more reasonable transmission sequence of data transmission is obtained, and students can obtain important effective information as quickly as possible.

Description

Technical Field

[0001] The present invention relates to the technical field of simulated flight equipment data processing, and in particular to a multi-modal data processing method, system, equipment and medium for a simulated cabin for training. Background Art

[0002] The cockpit simulator is a highly simulated device used for flight training and aviation education. Its core function is to provide a training platform that is close to the real flight environment. With the rapid development of the aviation industry and the improvement of flight safety requirements, traditional flight training methods can no longer meet actual needs. The cockpit simulator uses computer technology, virtual reality and multimedia technology to reproduce various flight situations and emergency scenarios, helping pilots to train their skills in a safe environment.

[0003] The simulated cockpit can not only simulate the aircraft's operating interface and flight dynamics, but also simulate different weather conditions, navigation systems and communication environments, allowing trainees to practice in a variety of situations. In addition, the use of simulated cockpits reduces training costs and reduces dependence on real aircraft, while facilitating repeated training and evaluation.

[0004] In a simulated cabin, there is a huge amount of multimodal data to be transmitted and processed. Multimodal data generally includes text data, image data, audio data or video data. The size of each type of data generally varies greatly. Therefore, in the transmission processing of the existing technology, there is no reasonable transmission processing order, which may cause information delays and prevent trainees from obtaining effective information in a timely manner. Summary of the invention

[0005] The embodiments of the present application solve the technical problems in the prior art by providing a method and system, device, and medium for processing multimodal data of a simulated cabin for training.

[0006] In a first aspect, the present application provides a method for processing multimodal data of a simulated cabin for training, comprising: Acquire a multimodal data set that needs to be transmitted in the current simulated cabin, and determine the data type of each piece of data in the current multimodal data set, where the data type includes text data, image data, audio data, or video data; If the data types in the current multimodal data set only include text data and audio data, they are sent directly without being processed; If the data types in the current multimodal data set include not only text data and audio data, then the data types and basic information of all data are obtained, a data evaluation model is established, and the evaluation value of each data is obtained through the data evaluation model; Sort the evaluation values ​​to obtain the transmission order of each data, and transmit all data according to the current order.

[0007] Furthermore, the obtaining of the multimodal data set that needs to be transmitted in the current simulated cabin includes: Obtain the flight status of the current simulated cabin feedback. If the current simulated cabin feedback is a normal flight status, directly obtain the multimodal data set; If the current simulated cockpit feedback is an abnormal flight state, wait for the flight state to switch to a normal flight state, and set a waiting threshold. If the waiting threshold is exceeded, obtain a multimodal data set.

[0008] Further, the setting of the waiting threshold includes: , In the formula, is the waiting threshold time, is the flight load of the current simulated cabin, is the maximum allowable flight load of the current simulated cabin, is the current flight altitude, is the average flight altitude.

[0009] Furthermore, the determining the data type of each piece of data in the current multimodal data set includes: Establish a data file extension classification comparison table and obtain the file extension of the current data; The file extension of the current data is searched in the data file extension classification comparison table, and the data type of the current data is output.

[0010] Furthermore, it also includes preprocessing each piece of data in the multimodal data set, including: Determine whether the current data is damaged. If damaged, delete the data; and generating a data occupation file consistent with the deleted data, and retaining the memory occupied by the data through the data occupation file; The original data of the deleted data is obtained again. If the data is not damaged, the file occupying the data is replaced and the file occupying the data is deleted; If the data is still damaged, delete the files occupying the data.

[0011] Furthermore, the data evaluation model includes: , In the formula, is the evaluation value, For the Calculation coefficients for data types, is the data size, is the calculation coefficient of data urgency, is the maximum transfer size of the data transfer channel, For the The total number of entries of the data type.

[0012] Furthermore, the calculation coefficients include: Get the content of the data. If the content involves the flight status of the aircraft, it is emergency data. ; If the content does not involve aircraft flight conditions, it is general data .

[0013] In a second aspect, the present invention provides a multi-modal data processing system for a simulated cabin for training, comprising: An acquisition module is configured to acquire a multimodal data set that needs to be transmitted by the current simulated cabin, and determine a data type of each piece of data in the current multimodal data set, wherein the data type includes text data, image data, audio data or video data; The sending module directly sends the data without processing if the data types in the current multimodal data set only include text data and audio data; if the data types in the current multimodal data set include not only text data and audio data, the data types and basic information of all data are obtained, a data evaluation model is established, and an evaluation value of each data is obtained through the data evaluation model; the evaluation values ​​are sorted to obtain the transmission order of each data, and all data are transmitted according to the current sorting; A main control module, the main control module is connected to the acquisition module and the sending module, and is used to execute the above-mentioned method for processing multimodal data of a simulated cabin for training.

[0014] In a third aspect, the present invention provides an electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute to implement the above-mentioned method for processing multimodal data of a simulated cabin for training.

[0015] In a fourth aspect, the present invention provides a non-temporary computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to implement a method for processing multimodal data of a simulated cabin for training as described above.

[0016] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: The above method of the present invention mainly includes an acquisition module, which is configured to acquire the multimodal data set that needs to be transmitted in the current simulated cabin. If the data types in the current multimodal data set only include text data and audio data, they are directly sent without processing; if the data types in the current multimodal data set include not only text data and audio data, a data evaluation model is established, and the evaluation value of each data is obtained through the data evaluation model; the evaluation values ​​are sorted to obtain the transmission order of each data, and all data are transmitted according to the current sorting. Through the above method, the types of various data in the multimodal data set are taken into consideration, a more appropriate evaluation model is constructed, and a more reasonable transmission order of data transmission is obtained, so that students can obtain important and effective information as quickly as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0019] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0020] refer to Figure 1 The present application provides a method for processing multimodal data of a simulated cabin for training, comprising: S101: Acquire a multimodal data set that needs to be transmitted in the current simulated cabin, and determine the data type of each data in the current multimodal data set, where the data type includes text data, image data, audio data, or video data; Multimodal data processing refers to the techniques and methods for simultaneously analyzing and processing data from different modes (modalities). These modalities can be different types of data, such as text, images, audio, video, etc. Multimodal data processing aims to improve the performance of models and the understanding of complex problems by integrating multiple sources of information.

[0021] The simulated cabin will transmit data from multiple aspects and integrate multiple information sources, so this embodiment takes the type of data into consideration to improve the timeliness of data transmission and processing.

[0022] S102: If the data types in the current multimodal data set only include text data and audio data, the data are directly sent without being processed; Among them, the data size of text data and audio data is generally small and will not occupy too many transmission channels. With the transmission rate of the current existing technology, the transmission of text data and audio data can basically reach a very fast speed, and there is no need to sort and transmit.

[0023] Secondly, this solution does not consider text data and audio data with very large data size. In the simulated cabin, the transmitted data is mainly concise, accurate and efficient, so there will not be very large text data and audio data.

[0024] S103: If the data types in the current multimodal data set include not only text data and audio data, then the data types and basic information of all data are obtained, a data evaluation model is established, and an evaluation value of each data is obtained through the data evaluation model; Among them, if not only text data and audio data are included, but also image data and video data appear. These data may occupy a large transmission performance of the transmission channel, so they will be transmitted after comprehensive consideration.

[0025] S104: Sort the evaluation values ​​to obtain the transmission order of each piece of data, and transmit all the data according to the current order.

[0026] Secondly, it is necessary to determine whether the data size of each transmission uses the full transmission performance of the transmission channel. If not, the next bit of the data being sent will be split into the size of the data that the transmission of the previous data leaves the remaining transmission performance of the transmission channel.

[0027] The above method of the present invention mainly includes an acquisition module, which is configured to acquire the multimodal data set that needs to be transmitted in the current simulated cabin. If the data types in the current multimodal data set only include text data and audio data, they are directly sent without processing; if the data types in the current multimodal data set include not only text data and audio data, a data evaluation model is established, and the evaluation value of each data is obtained through the data evaluation model; the evaluation values ​​are sorted to obtain the transmission order of each data, and all data are transmitted according to the current sorting. Through the above method, the types of various data in the multimodal data set are taken into consideration, a more appropriate evaluation model is constructed, and a more reasonable transmission order of data transmission is obtained, so that students can obtain important and effective information as quickly as possible.

[0028] In one embodiment of the present invention, obtaining a multimodal data set that needs to be transmitted in the current simulated cabin includes: Obtain the flight status of the current simulated cabin feedback. If the current simulated cabin feedback is a normal flight status, directly obtain the multimodal data set; If the current simulated cockpit feedback is an abnormal flight state, wait for the flight state to switch to a normal flight state, and set a waiting threshold. If the waiting threshold is exceeded, obtain a multimodal data set.

[0029] Specifically, setting the waiting threshold includes: , In the formula, is the waiting threshold time, is the flight load of the current simulated cabin, is the maximum allowable flight load of the current simulated cabin, is the current flight altitude, is the average flight altitude.

[0030] In this embodiment, it is not ruled out that the current simulated cabin feedback is that the flight status is abnormal, and the current abnormal status needs to be processed first. If the abnormal status cannot be eliminated after exceeding the waiting threshold, a multimodal data set will be obtained. The multimodal data set also has certain importance, so you cannot keep waiting.

[0031] In one embodiment of the present invention, determining the data type of each piece of data in the current multimodal data set includes: Establish a data file extension classification comparison table and obtain the file extension of the current data; The file extension of the current data is searched in the data file extension classification comparison table, and the data type of the current data is output.

[0032] In one embodiment of the present invention, it further includes preprocessing each piece of data in the multimodal data set, including: Determine whether the current data is damaged. If damaged, delete the data; and generating a data occupation file consistent with the deleted data, and retaining the memory occupied by the data through the data occupation file; The original data of the deleted data is obtained again. If the data is not damaged, the file occupying the data is replaced and the file occupying the data is deleted; If the data is still damaged, delete the files occupying the data.

[0033] In one embodiment of the present invention, the data evaluation model includes: , In the formula, is the evaluation value, For the Calculation coefficients for data types, is the data size, is the calculation coefficient of data urgency, is the maximum transfer size of the data transfer channel, For the The total number of entries of the data type.

[0034] Specifically, the calculation coefficients include: Get the content of the data. If the content involves the flight status of the aircraft, it is emergency data. ; If the content does not involve aircraft flight conditions, it is general data .

[0035] Secondly, , When , it is text data, The calculation coefficient for the first data type is 4; When , it is audio data, The calculation coefficient for the second data type is 3; When , it is image data, The calculation coefficient for the third data type is 2; When is the video data, The calculation coefficient for the fourth data type is 1.

[0036] Based on the same inventive concept, the present invention provides a multi-modal data processing system for a simulated cabin for training, comprising: An acquisition module is configured to acquire a multimodal data set that needs to be transmitted by the current simulated cabin, and determine the data type of each data in the current multimodal data set, wherein the data type includes text data, image data, audio data or video data; A sending module is used to directly send the data without processing if the data types in the current multimodal data set only include text data and audio data; if the data types in the current multimodal data set include not only text data and audio data, obtain the data types and basic information of all data, establish a data evaluation model, and obtain the evaluation value of each data through the data evaluation model; sort the evaluation values, obtain the transmission order of each data, and transmit all data according to the current sorting; A main control module is used to connect the main control module with the acquisition module and the sending module, and is used to execute the above-mentioned multi-modal data processing method of a simulated cabin for training.

[0037] Since the electronic device introduced in this embodiment is an electronic device used to implement the information processing method in the embodiment of the present application, based on the information processing method introduced in the embodiment of the present application, a person skilled in the art can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as a person skilled in the art implements the electronic device used by the information processing method in the embodiment of the present application, it belongs to the scope of protection of this application.

[0038] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0039] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0040] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0041] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0042] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0043] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for processing multimodal data of a simulated cabin for training, characterized in that: include: Acquire a multimodal data set that needs to be transmitted in the current simulated cabin, and determine the data type of each piece of data in the current multimodal data set, where the data type includes text data, image data, audio data, or video data; If the data types in the current multimodal data set only include text data and audio data, they are sent directly without being processed; If the data types in the current multimodal data set include not only text data and audio data, then the data types and basic information of all data are obtained, a data evaluation model is established, and the evaluation value of each data is obtained through the data evaluation model; Sort the evaluation values ​​to obtain the transmission order of each data, and transmit all data according to the current order.

2. A method for processing multimodal data of a simulated cabin for training according to claim 1, characterized in that: The step of obtaining a multimodal data set that needs to be transmitted in the current simulated cabin includes: Obtain the flight status of the current simulated cabin feedback. If the current simulated cabin feedback is a normal flight status, directly obtain the multimodal data set; If the current simulated cockpit feedback is an abnormal flight state, wait for the flight state to switch to a normal flight state, and set a waiting threshold. If the waiting threshold is exceeded, obtain a multimodal data set.

3. The method for processing multimodal data of a simulated cabin for training according to claim 2, characterized in that: The setting of the waiting threshold comprises: ， In the formula, is the waiting threshold time, is the flight load of the current simulated cabin, is the maximum allowable flight load of the current simulated cabin, is the current flight altitude, is the average flight altitude.

4. The method for processing multimodal data of a simulated cabin for training according to claim 1, characterized in that: The step of determining the data type of each piece of data in the current multimodal data set includes: Establish a data file extension classification comparison table and obtain the file extension of the current data; The file extension of the current data is searched in the data file extension classification comparison table, and the data type of the current data is output.

5. The method for processing multimodal data of a simulated cabin for training according to claim 1, characterized in that: It also includes preprocessing each piece of data in the multimodal data set, including: Determine whether the current data is damaged. If damaged, delete the data; and generating a data occupation file consistent with the deleted data, and retaining the memory occupied by the data through the data occupation file; The original data of the deleted data is obtained again. If the data is not damaged, the file occupying the data is replaced and the file occupying the data is deleted; If the data is still damaged, delete the files occupying the data.

6. The method for processing multimodal data of a simulated cabin for training according to claim 1, characterized in that: The data evaluation model includes: ， In the formula, is the evaluation value, For the Calculation coefficients for data types, is the data size of the current data, is the calculation coefficient of data urgency, is the maximum transfer size of the data transmission channel, For the The total number of entries of the data type.

7. A method for processing multimodal data of a simulated cabin for training according to claim 6, characterized in that: The calculation coefficients include: Get the content of the data. If the content involves the flight status of the aircraft, it is emergency data. ; If the content does not involve aircraft flight conditions, it is general data .

8. A multi-modal data processing system for a simulated cabin for training, characterized in that: include: An acquisition module is configured to acquire a multimodal data set that needs to be transmitted by the current simulated cabin, and determine a data type of each piece of data in the current multimodal data set, wherein the data type includes text data, image data, audio data or video data; The sending module directly sends the data without processing if the data types in the current multimodal data set only include text data and audio data; if the data types in the current multimodal data set include not only text data and audio data, the data types and basic information of all data are obtained, a data evaluation model is established, and an evaluation value of each data is obtained through the data evaluation model; the evaluation values ​​are sorted to obtain the transmission order of each data, and all data are transmitted according to the current sorting; A main control module, the main control module is connected to the acquisition module and the sending module, and is used to execute the multimodal data processing method of a simulated cabin for training as described in any one of claims 1-7.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute to implement a multimodal data processing method for a simulated cabin for training as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to implement a multimodal data processing method for a simulated cabin for training as described in any one of claims 1 to 7.

Images