Integrated display device based on FPGA and double ARM processor architecture

Abstract

The invention provides a comprehensive display device based on an FPGA (Field Programmable Gate Array) and double-ARM processor architecture, and relates to the field of avionics, which is characterized in that flight attitude data of a target aircraft are firstly collected and transmitted to a first ARM processor through an MIPI (Mobile Industry Processor Interface), the data are preprocessed by the first ARM processor and then stored in an input buffer area of a double-port shared memory, and the input buffer area is used for storing the data into a second ARM processor; then the FPGA chip reads the preprocessed data from the input buffer area to generate a flight attitude indication image, the flight attitude indication image is written into the output buffer area, the second ARM processor reads the attitude indication image from the output buffer area, the attitude indication image and the navigation image are subjected to image synthesis based on semantic feature interaction to obtain a synthesized flight display image, the synthesized flight display image is stored in a display frame buffer area, and finally, the flight display image is displayed. And reading the synthesized image from the display frame buffer region, and transmitting the synthesized image to a multifunctional display screen of a cockpit for display through an LVDS interface. Therefore, more efficient, accurate and reliable flight information display can be realized.

Description

Technical Field

[0001] The present application relates to the field of avionics, and more particularly, to a comprehensive display device based on an FPGA and a dual-ARM processor architecture. Background Art

[0002] In modern avionics systems, the real-time and accurate display of flight attitude and navigation information is crucial for the safe operation of pilots. With the progress of aviation technology and the increasing complexity of the operating environment, the standards for flight control systems are rising, which also drives the strict requirements for the performance of display devices.

[0003] However, traditional display devices usually adopt a single or relatively simple processor architecture, which is difficult to meet the real-time processing requirements of a large amount of flight attitude data and navigation data of modern aircraft. For example, in the face of high-dynamic flight scenarios or complex meteorological conditions, when a large amount of sensor data floods in, traditional devices may experience processing delays and cannot update the display information in a timely manner, resulting in the lag of flight data obtained by pilots and affecting decision-making. In addition, traditional display devices may use older transmission interface technologies with low data transmission rates, and cannot quickly transmit a large amount of data such as high-resolution flight attitude indication images and navigation images.

[0004] Therefore, a comprehensive display solution based on an FPGA and a dual-ARM processor architecture is desired. Summary of the Invention

[0005] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide a comprehensive display device based on an FPGA and a dual-ARM processor architecture.

[0006] According to one aspect of the present application, a comprehensive display device based on an FPGA and a dual-ARM processor architecture is provided, which includes:

[0007] A data input module, configured to collect a time series set of flight attitude data of a target aircraft object and input it into the first ARM processor through a MIPI interface;

[0008] A data preprocessing module, configured to preprocess the time series set of flight attitude data through the first ARM processor and write it into the input buffer of the dual-port shared memory;

[0009] A flight attitude indication image generation module, configured to read the preprocessed time series set of flight attitude data from the input buffer through an FPGA chip, and write the generated flight attitude indication image into the output buffer of the dual-port shared memory;

[0010] A flight image synthesis module, configured to read the flight attitude indication image from the output buffer through a second ARM processor, perform flight image synthesis based on key semantic feature interaction between the flight attitude indication image and a navigation image to obtain a synthesized flight display image, and write the synthesized flight display image into a display frame buffer;

[0011] A display module, configured to read the synthesized flight display image from the display frame buffer and send the synthesized flight display image to a multi-functional display screen in a cockpit for display through an LVDS interface.

[0012] Compared with the prior art, the integrated display device based on an FPGA and a dual-ARM processor architecture provided in this application first collects flight attitude data of a target aircraft, transmits the data to a first ARM processor through an MIPI interface, stores the data in an input buffer of a dual-port shared memory after preprocessing the data by the first ARM processor, then an FPGA chip reads the preprocessed data from the input buffer to generate a flight attitude indication image, writes the generated image into an output buffer, a second ARM processor reads the attitude indication image from the output buffer, performs image synthesis based on semantic feature interaction with a navigation image to obtain a synthesized flight display image, stores the synthesized image in a display frame buffer, and finally reads the synthesized image from the display frame buffer and transmits the synthesized image to the multi-functional display screen in the cockpit for display through an LVDS interface. In this way, more efficient, accurate, and reliable display of flight information can be achieved. Description of the Drawings

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0014] Figure 1 It is a block diagram of an integrated display device based on an FPGA and a dual-ARM processor architecture according to an embodiment of the present application.

[0015] Figure 2 It is a block diagram of a flight image synthesis module in an integrated display device based on an FPGA and a dual-ARM processor architecture according to an embodiment of the present application.

[0016] Figure 3 It is a schematic diagram of data flow of a flight image synthesis module in an integrated display device based on an FPGA and a dual-ARM processor architecture according to an embodiment of the present application.

[0017] Figure 4It is a block diagram of a flight attitude - navigation status feature interaction unit in an integrated display device based on an FPGA and a dual - ARM processor architecture according to an embodiment of the present application. Detailed implementation manners

[0018] Hereinafter, exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0019] In modern avionics systems, the real - time and accurate display of flight attitude and navigation information is crucial for the safe operation of pilots. With the continuous progress of aviation technology and the increasing complexity of the flight environment, the technical standards of flight control systems are also continuously improving, and this trend also places more stringent requirements on the performance of display devices.

[0020] However, traditional display devices usually adopt a single or relatively simple processor architecture. When dealing with the huge flight attitude data and navigation information of modern aircraft, it is often difficult to achieve efficient real - time processing. For example, in a high - dynamic flight environment or complex meteorological conditions, when a large amount of sensor data is input simultaneously, traditional devices may have delays due to limited processing capabilities, resulting in untimely updates of display information, making the flight data obtained by pilots lag, thus affecting their decision - making. In addition, many traditional display devices still rely on relatively old transmission interface technologies, with low data transmission rates, and it is difficult to quickly process and present high - resolution flight attitude indication images and navigation information.

[0021] It should be understood that an FPGA (Field Programmable Gate Array) is a highly programmable integrated circuit, and users can configure its hardware logic according to requirements to achieve specific functions. The FPGA has the characteristics of high parallel processing ability, low latency and high reliability, and is suitable for application scenarios that require real-time data processing. A dual-ARM processor refers to a processor that contains two processing cores based on the ARM architecture. This architecture is usually used for high-performance computing tasks, with low power consumption, high efficiency and multitasking capabilities, and is suitable for applications that require parallel computing and real-time response. In a flight display system, the combination of an FPGA and a dual-ARM processor can significantly improve the performance and reliability of the system. Specifically, the FPGA has high-speed parallel processing capabilities and can be used to process sensor data, video signals and graphics rendering in real time, thus ensuring the rapid update and accurate display of flight information. The dual-ARM processor is responsible for running the operating system, executing flight management software and processing human-machine interaction tasks, providing flexible software control capabilities. The two complement each other, enabling the system to have both low power consumption and high stability while ensuring high computing performance. At the same time, the FPGA directly executes tasks through hardware logic, reducing the possibility of software failures and improving the reliability of the system, while the dual-ARM processor can efficiently run multiple tasks, further enhancing the flexibility of the system. In addition, the programmable characteristics of the FPGA enable the system to be upgraded according to requirements, while the ARM processor supports a rich software ecosystem, facilitating subsequent function expansion. Generally speaking, through the combined application of an FPGA and a dual-ARM processor, the flight display system can operate more efficiently and reliably, which is beneficial to improving flight safety and the operation experience of pilots.

[0022] Based on this, the present application proposes a comprehensive display device based on the FPGA and dual-ARM processor architecture. By adopting a high-speed data interface, a parallel processing architecture, advanced machine vision technology and intelligent image synthesis methods, it provides a more efficient, accurate and reliable flight information display solution, significantly improving flight safety and operation convenience. Figure 1 The block diagram of the comprehensive display device based on the FPGA and dual-ARM processor architecture according to an embodiment of the present application. As Figure 1As shown, in the integrated display device 100 based on the FPGA and dual-ARM processor architecture, it includes: a data input module 110, which is used to collect the time series set of the flight attitude data of the target aircraft object and input it into the first ARM processor through the MIPI interface; a data preprocessing module 120, which is used to preprocess the time series set of the flight attitude data through the first ARM processor and write it into the input buffer of the dual-port shared memory; a flight attitude indication image generation module 130, which is used to read the preprocessed time series set of the flight attitude data from the input buffer through the FPGA chip, and write it into the output buffer of the dual-port shared memory after generating the flight attitude indication image; a flight image synthesis module 140, which is used to read the flight attitude indication image from the output buffer through the second ARM processor, and perform flight image synthesis based on key semantic feature interaction between the flight attitude indication image and the navigation image to obtain the synthesized flight display image, and write it into the display frame buffer; a display module 150, which is used to read the synthesized flight display image from the display frame buffer and send the synthesized flight display image to the multifunctional display screen in the cockpit for display through the LVDS interface.

[0023] In the embodiment of the present application, the data input module 110 is used to collect the time series set of the flight attitude data of the target aircraft object and input it into the first ARM processor through the MIPI interface. Specifically, the flight attitude data includes pitch angle, roll angle, and heading angle. It should be understood that the time series set of the flight attitude data contains information such as the pitch angle, roll angle, and heading angle of the target aircraft object at different times. These data are real-time and dynamically changing, and need to be collected and transmitted at a relatively high frequency to ensure that the pilot can obtain the aircraft's attitude information in a timely and accurate manner. The MIPI interface has the ability to transmit data at high speed and has strong anti-interference ability. By transmitting data through the MIPI interface, it can ensure that a large amount of flight attitude data is accurately transmitted to the first ARM processor in a short time. In this way, it can be ensured that in a complex aviation electromagnetic environment, the flight attitude data is not affected by electromagnetic interference during the transmission process, so that the data quality transmitted to the first ARM processor is reliable, and the processor can immediately process the data, avoiding processing lag caused by data transmission delay, thereby ensuring that the entire device can respond to the change of flight attitude in real time, and further providing timely and accurate information for the pilot.

[0024] In an embodiment of the present application, the data preprocessing module 120 is used to preprocess the timing set of the flight attitude data through the first ARM processor and write it into the input buffer of the dual-port shared memory. Accordingly, considering that the flight attitude data may be interfered by various factors during the collection and transmission process, such as sensor errors, transmission noise, etc., resulting in certain noise and errors in the data. The first ARM processor preprocesses the timing set of the flight attitude data and can perform operations such as cleaning and filtering on the data. Through these preprocessing steps, the quality of the data can be improved, providing a reliable basis for subsequent processing and analysis. In addition, since the subsequent data processing requires efficient access and processing of these data, by storing the preprocessed data in the input buffer of the dual-port shared memory, different hardware components (such as FPGA and ARM processor) can quickly and in parallel access these data, thereby speeding up the response speed of the overall device.

[0025] The following is a detailed description of a specific implementation process of "preprocessing the timing set of the flight attitude data by the first ARM processor and writing it into the input buffer of the dual-port shared memory":

[0026] After receiving the timing set of flight attitude data, the ARM1 processor immediately starts data parsing. It extracts key valid fields from each ARINC429 standard frame, such as pitch angle, roll angle, etc. During this data parsing process, the ARM1 processor will carefully analyze and disassemble each frame of data according to the preset protocols and rules to ensure that no important information is missed.

[0027] Next, the ARM1 processor will perform a strict check on the parsed data. It verifies the integrity and accuracy of the data by checking the CRC (cyclic redundancy check) bits. Specifically, the CRC (cyclic redundancy check) technology calculates the received data according to a specific algorithm, generates a check value, and compares it with the CRC check bits attached to the data. If the two are inconsistent, it means that the data "encountered an accident" during transmission and there is an error. At this time, the ARM1 processor will decisively discard these erroneous data to prevent them from interfering with subsequent processing. This data verification mechanism not only improves the robustness of the device, but also ensures the reliability of the final displayed information. By strictly checking the data, the ARM1 processor can effectively filter out noise and outliers to ensure that only high-quality data can enter the next processing flow.

[0028] The parsed and verified data already has relatively high accuracy and usability, and then data caching processing is carried out. The first ARM processor establishes a connection with the dual-port shared memory through the Memory Management Unit (MMU) and successfully obtains access rights to the input buffer. At this time, the processor will write the parsed and verified attitude data into the specified storage location in the input buffer in an orderly manner according to the pre-defined data storage format. During this process, the processor will ensure the correctness of the data writing order and format so that the subsequent data can be accurately read out for further processing.

[0029] In the embodiment of the present application, the flight attitude indication image generation module 130 is configured to read the time series set of the preprocessed flight attitude data from the input buffer through an FPGA chip, and write the generated flight attitude indication image into the output buffer of the dual-port shared memory. In particular, here the time series set of the preprocessed flight attitude data is read from the input buffer of the dual-port shared memory through a DMA controller. By quickly and accurately obtaining the time series set of the preprocessed flight attitude data through the DMA controller, the FPGA chip can calculate and process in a timely manner according to the latest flight attitude data, generate an indication image that accurately reflects the current attitude of the aircraft, and write it into the output buffer of the dual-port shared memory, so that subsequent data processing steps can flexibly obtain the required data. For example, according to data such as pitch angle, roll angle, and heading angle, the FPGA can accurately draw the attitude graph of the aircraft in three-dimensional space, providing intuitive flight attitude information for the pilot. It is worth mentioning that the DMA controller can directly access the memory, avoiding the additional overhead brought by the processor participating in data transmission. This enables the data to be quickly read from the input buffer into the FPGA when processing a large number of time series sets of flight attitude data, meeting the requirement of the FPGA for rapid data processing. Specifically, in a specific example of the present application, the implementation steps for the FPGA chip to generate a flight attitude indication image include: after reading the flight attitude data, the FPGA chip first performs a coordinate transformation operation on it. The flight attitude data collected by the sensors on the aircraft is based on the aircraft's own body coordinate system. However, to visually present the flight attitude on the multifunctional display screen in the cockpit, these data must be converted into the screen coordinate system. Taking the pitch angle data of the aircraft as an example, the FPGA will accurately map it to the Y-axis of the screen according to a certain mapping rule; the roll angle data will be mapped to other relevant dimensions of the screen. This coordinate transformation process converts the abstract flight attitude data into coordinate information that can be visually represented on the screen, laying the foundation for the accurate generation of subsequent graphics. Based on the result of the coordinate transformation, the FPGA chip starts to generate the graphic elements of the attitude director indicator (ADI). The attitude director indicator is an important visual tool for the pilot to understand the flight attitude of the aircraft. The FPGA uses its powerful parallel processing ability to quickly and accurately draw various graphic elements according to the flight attitude data. For example, the line representing the horizontal plane will be accurately drawn. By observing the relative position of this line and the aircraft icon, the pilot can clearly judge whether the aircraft is in a horizontal flight state or an inclined state; the aircraft icon will also be accurately displayed according to the actual flight attitude, showing the specific position and direction of the aircraft in the current attitude. After generating the graphic elements of the attitude director indicator, the FPGA chip needs to superimpose them on the background map loaded from the second ARM processor. The background map provides the pilot with the geographical background information of the aircraft's flight, which is crucial for the pilot to understand the position of the aircraft and the surrounding environment.The FPGA will skillfully overlay the attitude indicator graphic elements on the background map to generate the final flight attitude indication image, enabling the pilot to obtain both flight attitude and geographical background information in one picture.

[0030] In the embodiment of the present application, the flight image synthesis module 140 is configured to read the flight attitude indication image from the output buffer through the second ARM processor, and perform flight image synthesis based on key semantic feature interaction on the flight attitude indication image and the navigation image to obtain the synthesized flight display image, and write it into the display frame buffer. It should be understood that synthesizing the flight attitude indication image and the navigation image can provide more comprehensive and intuitive flight information for the pilot. The flight attitude indication image shows the current attitude of the aircraft (such as pitch, roll, heading, etc.), while the navigation image shows information such as the flight route, geographical location, and surrounding terrain of the aircraft. The synthesized flight display image combines these two types of information, enabling the pilot to obtain both the attitude and navigation information of the aircraft in one picture and more accurately grasp the flight situation. During flight, the flight attitude and navigation situation change continuously, and the image content is also very complex. However, traditional rule-based synthesized images often use relatively simple and fixed feature extraction rules, which are difficult to adapt to this complex and changeable scenario and cannot timely and accurately extract the features that match the current flight state. For example, in the face of bad weather or special flight missions, the features in the image may change greatly, and traditional methods may not be able to effectively extract key features, resulting in poor image synthesis effects.

[0031] Based on this, the technical concept of the present application is to use image analysis and processing technology based on machine vision to enhance the image resolution of the flight attitude indication image and the navigation image. Then, feature extraction is performed on the preprocessed flight attitude indication image and the preprocessed navigation image, and based on the key semantic fast interaction representation between the extracted flight attitude features and navigation state features, the synthesized flight display image is automatically generated. The present application can dynamically adjust the feature extraction strategy under different environmental conditions. Whether it is clear weather or bad weather (such as heavy rain, fog, etc.), the system can accurately extract the key features that match the current flight state to ensure the accuracy and reliability of image synthesis.

[0032] Figure 2 It is a block diagram of the flight image synthesis module in the integrated display device based on the FPGA and dual-ARM processor architecture according to the embodiment of the present application. Figure 3 It is a schematic diagram of the data flow of the flight image synthesis module in the integrated display device based on the FPGA and dual-ARM processor architecture according to the embodiment of the present application. As Figure 2 and Figure 3As shown, the flight image synthesis module 140 includes: an image preprocessing unit 141, configured to enhance the image resolution of the flight attitude indication image and the navigation image to obtain a preprocessed flight attitude indication image and a preprocessed navigation image; a flight attitude feature extraction unit 142, configured to extract flight attitude features from the preprocessed flight attitude indication image to obtain a flight attitude feature vector; a navigation state feature extraction unit 143, configured to extract navigation state features from the preprocessed navigation image to obtain a navigation state feature vector; a flight attitude - navigation state feature interaction unit 144, configured to perform a key semantic fast interaction analysis between the flight attitude and the navigation state on the flight attitude feature vector and the navigation state feature vector to obtain a flight attitude - navigation state feature interaction coding vector; and a flight image synthesis unit 145, configured to obtain the synthesized flight display image based on the flight attitude - navigation state feature interaction coding vector.

[0033] In an embodiment of the present application, the image preprocessing unit 141 is configured to enhance the image resolution of the flight attitude indication image and the navigation image to obtain a preprocessed flight attitude indication image and a preprocessed navigation image. Correspondingly, considering that image acquisition devices are limited by factors such as cost, volume, and technical specifications, the resolution of the acquired flight attitude indication images and navigation images often fails to reach the ideal state. For example, when sensors installed on an aircraft acquire images, due to issues such as lighting conditions and device accuracy, the original images may be blurred, with details lost, etc. Moreover, the multifunctional display screen in the cockpit has specific resolution and display accuracy requirements. If the resolution of the original image is too low, problems such as image stretching and pixelation will occur when presented on the display screen, seriously affecting the pilot's reading of information. Based on this, in the technical solution of the present application, the image resolution of the flight attitude indication image and the navigation image is enhanced to obtain a preprocessed flight attitude indication image and a preprocessed navigation image. In this way, the preprocessed high - resolution images can provide more accurate data for subsequent feature extraction, thereby improving the accuracy of the extracted flight attitude features and navigation state features.

[0034] In the embodiment of the present application, the flight attitude feature extraction unit 142 is configured to extract flight attitude features from the preprocessed flight attitude indication image to obtain a flight attitude feature vector. Specifically, in the embodiment of the present application, the flight attitude feature extraction unit is configured to: pass the preprocessed flight attitude indication image through a flight attitude feature extractor based on the MobileNet model to obtain the flight attitude feature vector. Accordingly, considering that the preprocessed flight attitude indication image contains key features related to the flight attitude, such as the manifestation features of attitude information such as the pitch, roll, and heading of the aircraft in the image. Therefore, in order to be able to reflect the actual attitude of the aircraft, the present application passes the preprocessed flight attitude indication image through a flight attitude feature extractor based on the MobileNet model to obtain a flight attitude feature vector. It should be understood that MobileNet is a lightweight convolutional neural network model, and its design concept is to minimize the number of model parameters and the amount of computation while ensuring a certain accuracy rate. Compared with some large convolutional neural networks (such as ResNet, VGG, etc.), MobileNet requires less computing resources when processing images, can run quickly on an ARM processor, and will not impose too much burden on the processor, thus ensuring the real-time performance of the system. At the same time, the MobileNet model has been trained with a large amount of image data, has a strong feature extraction ability, can extract key implicit features related to the flight attitude from the image, and convert the image information into a set of compact and representative flight attitude feature vectors.

[0035] In the embodiment of the present application, the navigation state feature extraction unit 143 is configured to extract navigation state features from the preprocessed navigation image to obtain a navigation state feature vector. Specifically, in the embodiment of the present application, the navigation state feature extraction unit is configured to: pass the preprocessed navigation image through a navigation state feature extractor based on the ViT model to obtain the navigation state feature vector. It should be understood that the navigation image contains rich information such as the flight route of the aircraft, geographical location, and surrounding terrain, and there are complex long-sequence dependence relationships between these information. For example, the continuity of the flight route, the association between the geographical location and the surrounding terrain, etc. At the same time, the information in the navigation image may involve different scales, such as the macroscopic flight area map and the microscopic specific landmark details, etc. Based on this, in the technical solution of the present application, the preprocessed navigation image is passed through a navigation state feature extractor based on the ViT model to quantify and represent the key information in the navigation image through the self-attention mechanism of its Transformer architecture, and a navigation state feature vector is obtained. Specifically, the ViT model is based on the Transformer architecture and can well handle this long-sequence dependence relationship. It can more effectively capture the long-term dependence and context information between different regions and different elements in the navigation image, so as to more comprehensively understand the content of the navigation image. At the same time, through its self-attention mechanism, the ViT model can automatically model information at different scales, adaptively focus on important regions and features in the image, whether these features are at the global or local scale. These features include information such as the direction, distance, geographical location coordinates, and terrain features of the flight route, helping the pilot and the flight control system to accurately master the navigation state of the aircraft and ensure that the flight proceeds safely and accurately according to the predetermined route.

[0036] The following is a detailed elaboration of a specific implementation process of "passing the preprocessed navigation image through a navigation state feature extractor based on the ViT model to obtain the navigation state feature vector":

[0037] First, the preprocessed navigation image is subjected to block division and position encoding processing. The preprocessed navigation image contains rich information such as the flight route of the aircraft, geographical location, and surrounding terrain, and there are complex long-sequence dependence relationships between these information and it involves different scales. The navigation image is divided into multiple image blocks of a fixed size, and each image block becomes the basic unit for subsequent processing. Since the position information of the image blocks in the original image is crucial for understanding the content of the navigation image, such as the continuity of the flight route, the association between the geographical location and the surrounding terrain, etc., it is necessary to add position encoding to each image block. The position encoding is generated through a specific algorithm, which incorporates the position information into the image block features in numerical form, enabling the model to perceive the spatial relationship between different image blocks, thereby laying a foundation for accurately understanding various information in the navigation image.

[0038] After completing the block segmentation and position encoding, the linear embedding and feature fusion operations are then performed. The linear projection layer is used to perform linear embedding on the image blocks after segmentation and position encoding. This operation converts the original high-dimensional pixel values ​​of each image block into a low-dimensional feature vector, which retains key information to the greatest extent while reducing the data dimension and reducing the amount of computation for subsequent processing. Subsequently, these low-dimensional feature vectors are spliced ​​with learnable class embeddings (class tokens). Class embedding is a specially designed vector that aggregates global features and represents the comprehensive information of the entire image. Through this splicing method, the fusion of local features and global features of each image block is achieved, providing a more comprehensive and rich information basis for subsequent feature extraction, enabling the model to understand the navigation image from a more macro and micro perspective.

[0039] Then, the fused feature vector sequence enters the Transformer encoder for deep processing. The Transformer encoder is composed of multiple identical layers stacked together, each of which mainly contains a multi-head self-attention mechanism and a multi-layer perceptron (MLP). In the multi-head self-attention mechanism, the model performs parallel attention calculations on the feature vector sequence through different heads. The uniqueness of this mechanism is that it enables the model to pay attention to the relationship between image patches from multiple different perspectives, and adaptively capture the long-term dependencies and contextual information between different regions and elements in the navigation image. For example, when paying attention to the flight route, the model can simultaneously consider the impact of the geographical location and the surrounding terrain on the flight route; when processing the geographical location information, it can also be associated with the planning and direction of the flight route. The output of the multi-head self-attention mechanism is input into the multi-layer perceptron after layer normalization and residual connection. The multi-layer perceptron further transforms and maps the features, enhances the expressiveness of the features through nonlinear activation functions, and mines deeper and more representative feature information.

[0040] After multiple layers of deep processing by the Transformer encoder, a feature representation containing rich navigation information is obtained. From these complex feature representations, key features closely related to the navigation status are extracted. These key features cover many important aspects such as the direction, distance, geographic location coordinates, and terrain features of the flight route. In order to integrate and compress these key features into a compact vector that is convenient for subsequent processing, specific mapping or pooling operations are used. Through these operations, the key features scattered in different dimensions are effectively integrated, redundant information is removed, and finally a navigation status feature vector that can fully quantify and represent the key information of the navigation image is generated.

[0041] In the embodiment of the present application, the flight attitude-navigation state feature interaction unit 144 is configured to perform a fast key semantic interaction analysis between the flight attitude and the navigation state on the flight attitude feature vector and the navigation state feature vector to obtain a flight attitude-navigation state feature interaction coding vector. Specifically, Figure 4 It is a block diagram of the flight attitude-navigation state feature interaction unit in the integrated display device based on the FPGA and dual-ARM processor architecture according to the embodiment of the present application. As Figure 4 shown, the flight attitude-navigation state feature interaction unit 144 includes: a feature implicit coding sub-unit 1441, configured to perform principal component implicit coding and implicit feature extraction on the flight attitude feature vector and the navigation state feature vector respectively to obtain a set of flight attitude feature principal component implicit coding vectors and navigation state feature coding vectors; a flight attitude-navigation state implicit feature anchoring sub-unit 1442, configured to perform flight attitude-navigation state implicit key clue anchoring on the navigation state feature coding vectors and the set of flight attitude feature principal component implicit coding vectors to obtain a {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector} feature pair; a flight attitude-navigation state feature fine-grained interaction sub-unit 1443, configured to perform flight attitude-navigation state feature fine-grained interaction on the flight attitude feature vector and the navigation state feature vector based on the {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector} feature pair to obtain the flight attitude-navigation state feature interaction coding vector.

[0042] It should be understood that the flight attitude feature vector mainly reflects the attitude information of the aircraft, such as pitch, roll, and heading; the navigation state feature vector focuses on the navigation information of the aircraft, such as flight route and geographical location. Although they each have their own focuses, they are closely related in actual flight. The attitude change of the aircraft will affect its navigation path, and the navigation information determines the adjustment direction of the flight attitude. Therefore, in order to integrate the information carried by them, achieve complementarity, and comprehensively display the flight state of the aircraft, the present application performs a fast key semantic interaction analysis between the flight attitude and the navigation state on the flight attitude feature vector and the navigation state feature vector to obtain a flight attitude-navigation state feature interaction coding vector. That is, in actual flight, the environment is complex and changeable, and situations such as bad weather, terrain restrictions, and air traffic control may be encountered, and the flight attitude and navigation state will also change frequently. Performing interaction analysis on the feature vectors can enable the system to better adapt to this complex dynamic change, comprehensively capture the impact of various key factors on flight, and provide a richer and more accurate feature representation for subsequent processing.

[0043] Specifically, in the embodiments of the present application, the feature implicit encoding subunit is configured to: perform principal component analysis on the flight attitude feature vector to obtain a set of flight attitude feature principal component encoding vectors, and this process can be expressed by the formula:

[0044]

[0045] where V2 is the flight attitude feature vector, PCA(·) is the principal component analysis operation, C2 is the flight attitude feature sample covariance matrix calculated from V2, U2 is the flight attitude feature principal component orthogonal matrix, v 21 , v 22 , v 2i and v 2m are respectively the 1st, 2nd, i-th, and m-th flight attitude feature principal component encoding vectors in the set of flight attitude feature principal component encoding vectors, Λ2 is the flight attitude feature diagonal matrix, λ 21 , λ 2m are respectively the eigenvalues corresponding to v 21 and v 2m , and U2 T is the transpose matrix of U2;

[0046] Perform flight attitude point convolution implicit feature extraction on each flight attitude feature principal component encoding vector in the set of flight attitude feature principal component encoding vectors to obtain the set of flight attitude feature principal component implicit encoding vectors, and this process can be expressed by the formula:

[0047]

[0048] where Conv 1×1 is point convolution encoding, sigmoid is the sigmoid function, h 21 , h 22 , h 2i and h 2m are respectively the 1st, 2nd, i-th, and m-th flight attitude feature principal component implicit encoding vectors in the set of flight attitude feature principal component implicit encoding vectors, and H2 is the set of flight attitude feature principal component implicit encoding vectors;

[0049] Perform navigation state point convolution implicit feature extraction on the navigation state feature vector to obtain the navigation state feature encoding vector, and this process can be expressed by the formula:

[0050] H1 = sigmoid[Conv 1×1 (V1)]

[0051] where Conv 1×1It is dot convolution encoding, sigmoid is the sigmoid function, V1 is the navigation state feature vector, and H1 is the navigation state feature encoding vector.

[0052] It should be understood that the flight attitude feature vector encompasses information in multiple dimensions such as pitch, roll, and heading. Although this information is rich, it also results in a high data dimension. When processing a large amount of flight attitude data, the high dimension brings computational challenges, not only increasing the computational time but also potentially leading to excessive consumption of resources. By performing principal component analysis on the flight attitude feature vector and mapping the original features to a new low-dimensional feature space, data dimensionality reduction and a decrease in computational complexity can be achieved. Specifically, during the construction of this low-dimensional feature space, the variance distribution in the data can be keenly captured, enabling data analysis to focus on the directions with larger variances, that is, to focus on the feature data carrying rich information. In this way, the most crucial part of the original flight attitude features can be retained. That is, the generated set of flight attitude feature principal component encoding vectors expresses most of the key information in the original high-dimensional feature vector with fewer dimensions, which can make the entire system operate more efficiently.

[0053] Correspondingly, considering that although the set of flight attitude feature principal component encoding vectors has reduced dimensions through principal component analysis, there may be complex interaction relationships between the feature space channels represented by each vector. These interaction relationships contain deeper and more essential information about the flight attitude, which helps the model accurately understand and describe the flight attitude. Based on this, in this application, it is necessary to perform flight attitude dot convolution implicit feature extraction processing on each flight attitude feature principal component encoding vector in the set of flight attitude feature principal component encoding vectors to deeply explore the interaction relationships between the feature space channels of each vector, and then discover the potential laws hidden behind the data. Specifically, dot convolution integrates information at the feature channel level through cross-dimensional weight mapping, enabling the information between different feature channels to be fused with each other. This helps eliminate redundancy between features while retaining important information. For example, in flight attitude analysis, different sensors may provide related but not completely identical information. Through the information integration of dot convolution, these information can be effectively fused to form a more accurate and comprehensive description of the flight attitude. And by combining with a non-linear activation function, the feature expression ability can be significantly improved. That is, the obtained set of flight attitude feature principal component implicit encoding vectors can express flight attitude information more accurately and richly.

[0054] It should be understood that point convolution does not depend on the locality of the original feature space, but directly performs cross-dimensional weight mapping at the feature channel level. This feature enables effective integration of information in different dimensions of the navigation state feature vector. Taking the geographical location information and flight route information of an aircraft as an example, they belong to features in different dimensions, and it may be difficult for traditional methods to quickly and accurately establish the connection between the two. However, through weight mapping, point convolution can directly perform correlation analysis on the geographical location information and flight route information, enabling the system to quickly understand the specific situation of the aircraft's current position in its planned flight route, such as determining whether the aircraft has deviated from the predetermined route, as well as the degree and direction of deviation. This cross-dimensional information integration greatly improves the processing efficiency and accuracy of navigation state information. Moreover, during the processing of the navigation state feature vector, by introducing a non-linear activation function, the expression ability of the navigation state features can be further enhanced. Specifically, the navigation state of an aircraft presents complex and variable characteristics during actual flight, and the non-linear activation function allows the model to learn these complex feature patterns. When encountering temporary changes in the flight route by air traffic control, or irregular changes in flight speed and direction due to air currents, the non-linear activation function can help the model capture these non-linear change features, enabling the navigation state feature encoding vector to more accurately and richly represent the actual navigation state. Generally speaking, the navigation state feature encoding vector obtained after extracting the implicit features of the navigation state by point convolution integrates rich navigation information, discovers the internal connections between features, enhances the expression ability of complex navigation states, and can lay a solid foundation for subsequent interactive analysis with flight attitude features.

[0055] Next, perform flight attitude-navigation state implicit key clue anchoring on the set of the navigation state feature encoding vector and the flight attitude feature principal component implicit encoding vector to obtain a feature pair {navigation state implicit feature encoding vector, anchored flight attitude principal component implicit feature encoding vector}. The above process can be expressed by the formula:

[0056]

[0057] F bestpair ={H1; h 2k}

[0058] where h 2i is the i-th flight attitude feature principal component implicit encoding vector in the set of flight attitude feature principal component implicit encoding vectors, H1 is the navigation state feature encoding vector, <·> represents the inner product, ‖·‖ is the Euclidean norm for calculating the vector, ε is the modulation coefficient, argmax j (·) returns the j value corresponding to the maximum value, k is the position to find the maximum approximate matching value in the set of flight attitude feature principal component implicit encoding vectors, h2k is the anchored flight attitude principal component implicit feature encoding vector, F bestpair is the feature pair {navigation state implicit feature encoding vector, anchored flight attitude principal component implicit feature encoding vector}.

[0059] It should be understood that flight attitude and navigation state are two different but closely related aspects during the flight of an aircraft. The feature encoding vectors generated by them come from different sources and have different semantics and feature distributions. The navigation state feature encoding vector mainly reflects navigation information such as the flight route and geographical location of the aircraft, while the set of flight attitude feature principal component implicit encoding vectors focuses on attitude information such as the pitch, roll, and heading of the aircraft. In an actual flight scenario, although these two types of information affect each other, due to the differences in their sources and representation methods, there is semantic inconsistency. Through the anchored processing of flight attitude - navigation state implicit key clues, the features from these two different sources can be semantically aligned. It establishes a unified semantic standard for different features in the multi - dimensional feature space, enabling the navigation state features and flight attitude features to be understood and analyzed within the same semantic framework. For example, when the aircraft makes a turning operation, the flight attitude changes (such as roll), and at the same time, the flight route in the navigation state also changes accordingly. Through semantic alignment, the model can clearly identify the internal connection between these two changes and accurately correspond the attitude change with the adjustment of the navigation route. In specific implementation, by calculating the correlation between the navigation state feature encoding vector and the set of flight attitude feature principal component implicit encoding vectors, the internal connection between them can be found. Based on this correlation, the most valuable feature pairs are extracted, such as the flight attitude features corresponding to the aircraft being at a specific navigation position. These feature pairs provide clear association relationships for subsequent feature interactions, enabling the system to better understand the mutual influence between flight attitude and navigation state, and thus achieving more accurate flight state analysis and prediction.

[0060] Finally, based on the feature pair {navigation state implicit feature encoding vector, anchored flight attitude principal component implicit feature encoding vector}, perform fine - grained interaction of flight attitude - navigation state features on the flight attitude feature vector and the navigation state feature vector to obtain the flight attitude - navigation state feature interaction encoding vector. The above process can be expressed by the formula:

[0061]

[0062] where, V2 is the flight attitude feature vector, V1 is the navigation state feature vector, H1 is the navigation state feature encoding vector, h 2k is the anchored flight attitude principal component implicit feature encoding vector, h 2k T is h 2kThe transposed vector, softmax is the activation function, S is h 2k Length, is a matrix multiplication, α and β are weighted hyperparameters, V f It is the flight attitude-navigation state feature interaction encoding vector.

[0063] It should be understood that after completing the semantic alignment of the implicit key clues of flight attitude and navigation state, the fine-grained interaction of flight attitude and navigation state features can be carried out to explore deeper and more complex nonlinear associations between flight attitude and navigation state features. Specifically, the flight process of an aircraft is a complex dynamic system, and the changes in flight attitude (such as pitch, roll, and yaw) and navigation state (such as flight route and geographical location) are not simply linear. For example, when flying in mountainous areas, in order to avoid mountain peaks, the aircraft needs to constantly adjust the flight attitude, and the navigation system will also plan a new flight route in real time according to the terrain and target location. In this case, the mutual influence between flight attitude and navigation state is highly nonlinear. Through fine-grained interaction, these complex relationships can be captured, making the system's understanding of the flight state more in-depth and accurate. The multi-head self-attention mechanism is one of the core technologies for realizing fine-grained interaction of flight attitude and navigation state features. It can simultaneously focus on different parts of the input features and capture long-distance dependencies and multi-dimensional semantic interactions. In the interaction of flight attitude and navigation state features, the multi-head self-attention mechanism can analyze and fuse features from multiple angles. For example, one "head" can focus on the impact of changes in the pitch angle in the flight attitude feature on the navigation route, and the other "head" can focus on how changes in the geographic location in the navigation state prompt the adjustment of the flight attitude. In this way, feature fusion is given greater flexibility and can adapt to the complex and changing relationship between flight attitude and navigation state in different flight scenarios. The flight attitude-navigation state feature interaction encoding vector finally generated integrates the key information of flight attitude and navigation state, and fully considers the complex nonlinear relationship between the two, which can provide a solid feature data foundation for the subsequent generation of high-quality synthetic flight display images.

[0064] Preferably, in another embodiment of the present application, the flight attitude-navigation state feature fine-grained interaction subunit is configured to: perform weak virtual semantic correction based on eigenvalues on the {navigation state implicit feature encoding vector, anchored flight attitude principal component implicit feature encoding vector} feature pair to obtain the corrected {navigation state implicit feature encoding vector, anchored flight attitude principal component implicit feature encoding vector} feature pair; based on the corrected {navigation state implicit feature encoding vector, anchored flight attitude principal component implicit feature encoding vector} feature pair, perform flight attitude-navigation state feature fine-grained interaction on the flight attitude feature vector and the navigation state feature vector to obtain the flight attitude-navigation state feature interaction encoding vector. The above process is expressed by the formula:

[0065]

[0066] where, H 1i is the eigenvalue at the i-th position in H1, h 2ki is the eigenvalue at the i-th position in h 2k , H 1i ′ is the corrected eigenvalue of H 1i , h 2ki ′ is the corrected eigenvalue of h 2ki , H1′ is the corrected navigation state implicit feature encoding vector, h 2k ′ is the corrected anchored flight attitude principal component implicit feature encoding vector, V f is the flight attitude-navigation state feature interaction encoding vector.

[0067] Furthermore, aiming at the alignment ambiguity caused by source uncertainty between the navigation state implicit feature and the anchored flight attitude principal component implicit feature, a weakening and virtualizing mechanism is introduced for weak virtual power-law expansion. That is, the 3 / 8 exponent is used as the precursor prior expansion to perform power-law prior responsive fuzzification convergence on the alignment condition of the navigation state implicit feature parameter boundary, and the 1 / 4 exponent is used as the main body alignment expansion to perform strict constraint on the power-law distribution alignment attenuation relaxation of the anchored flight attitude principal component implicit feature, so as to avoid the prior ambiguity of the system behavior under a single mechanism through the value correlation mechanism when the alignment boundary condition constraint within the correlation effective range is ill-defined, to correct the semantic distribution consistency fuzzification within the alignment interval, and to improve the mining intuitiveness of the implicit fine-grained interaction correlation of the anchored feature.

[0068] In the embodiment of the present application, the flight image synthesis unit 145 is used to obtain the synthesized flight display image based on the flight attitude-navigation state feature interaction coding vector. Specifically, in the embodiment of the present application, the flight image synthesis unit is used to: pass the flight attitude-navigation state feature interaction coding vector through the flight display synthesizer based on AIGC to obtain the synthesized flight display image. That is, the flight attitude-navigation state feature interaction coding vector obtained by the key semantic interaction of the above-mentioned flight attitude feature vector and the navigation state feature vector is used for generation and processing, so as to use the flight display synthesizer based on AIGC (artificial intelligence generated content) to use the deep learning algorithm to understand the complex semantic information in the flight attitude-navigation state feature interaction coding vector, and based on this information, learn the composition mode of the image under different flight states from a large amount of training data, so as to generate a high-quality, synthesized flight display image that conforms to the actual flight situation. The synthesized flight display image contains detailed flight attitude indication information and navigation information, which can provide an intuitive and comprehensive view for the pilot. In this image, an icon representing the aircraft itself clearly shows the aircraft's current attitude, including bank angle and pitch angle; at the same time, the horizon indicator can be used to determine whether the aircraft is in an ascending, descending or horizontal flight state, and the heading indicator shows the aircraft's current heading angle. In addition, the image also includes map information of the current location, the destination, the preset route and the overview of the surrounding terrain, as well as waypoint marking information.

[0069] The following is a detailed description of a specific implementation process of "passing the flight attitude-navigation state feature interactive coding vector through an AIGC-based flight display synthesizer to obtain the synthesized flight display image":

[0070] The model training phase is the foundation and key of the entire synthesis process, which determines the flight display synthesizer's understanding of flight data and image generation capabilities. First of all, it is necessary to widely collect rich data in various flight scenarios, including flight attitude data, navigation data and corresponding actual flight display images in different weather conditions (such as sunny days, heavy rain, fog, etc.), flight stages (take-off, cruising, landing, etc.) and geographical environments (plains, mountains, over cities, etc.). The diversity and comprehensiveness of these data are crucial, allowing the flight display synthesizer to learn the flight information characteristics under various complex situations. After the collection is completed, data preprocessing is carried out immediately. By cleaning the data to remove outliers and noise, the accuracy and reliability of the data are ensured, and the flight attitude, navigation data and actual flight display images are accurately associated and annotated to construct a high-quality training data set for model learning.

[0071] Subsequently, a suitable deep learning architecture is selected to build an AIGC-based flight display synthesizer. Taking the Generative Adversarial Network (GAN) as an example, it consists of two important components: a generator and a discriminator. The generator is responsible for generating a synthetic image based on the input flight attitude-navigation state feature interaction encoded vector, while the discriminator is used to judge the difference between the generated image and the real flight display image. The two cooperate and compete with each other. During the training process, the preprocessed training data is continuously input into the synthesizer, and the parameters of the generator and the discriminator are continuously adjusted using the backpropagation algorithm. The generator tries to generate high-quality images that can deceive the discriminator, while the discriminator strives to accurately distinguish between real images and generated images. After a large number of training iterations, the flight display synthesizer gradually stabilizes and reaches a good performance state, capable of deeply understanding the complex semantic information in the flight attitude-navigation state feature interaction encoded vector.

[0072] When the model training is completed and the expected performance is achieved, it enters the image generation stage. At this time, the flight attitude-navigation state feature interaction encoded vector obtained by processing through the flight attitude-navigation state feature interaction unit is input into the generator of the trained flight display synthesizer. The generator decodes the encoded vector according to the composition patterns of images in different flight states learned during the training stage. It starts from the rich feature patterns extracted from the training data and methodically constructs a synthetic image. For example, according to the attitude information in the encoded vector, it accurately determines the attitude of the icon representing the aircraft in the image, including the tilt angle and pitch angle; based on the navigation information, it generates map information of the current position, destination, preset route, overview information of the surrounding terrain, and waypoint marking information, etc. These elements are reasonably combined in the image according to a certain logic and layout, initially forming a synthetic image containing flight attitude indication information and navigation information.

[0073] However, the initially generated synthetic image still needs to be strictly evaluated by the discriminator (in the GAN architecture). The discriminator comprehensively judges the generated image from various aspects of the image, such as the accuracy of elements, the rationality of the layout, and the matching degree with the real flight scene. If the generated image meets the preset quality standards, it is output as the final synthesized flight display image; if it does not meet the requirements, the generator will further adjust its own parameters according to the information feedback by the discriminator and regenerate the image. This process will be repeated until the generated image meets the high-quality requirements. The finally output synthesized flight display image contains detailed and accurate flight attitude indication information and navigation information, providing the pilot with an intuitive and comprehensive view of the flight state.

[0074] In summary, the flight image synthesis module 140 is clearly described. It uses machine vision-based image analysis and processing technologies to enhance the image resolution of the flight attitude indication image and the navigation image. Then, feature extraction is performed on the preprocessed flight attitude indication image and the preprocessed navigation image, and based on the key semantic quick interaction representation between the extracted flight attitude features and navigation state features, the synthesized flight display image is automatically generated. In this way, the feature extraction strategy can be dynamically adjusted under different environmental conditions, thereby ensuring the accuracy and reliability of the synthesized flight display image.

[0075] In the embodiment of the present application, the display module 150 is configured to read the synthesized flight display image from the display frame buffer and send the synthesized flight display image to the multi-functional display screen in the cockpit for display through the LVDS interface. It should be understood that the display frame buffer is like a data transfer station that pre-stores the synthesized flight display image. Reading the image from the buffer can ensure the stability and continuity of the data. During flight, the processing of flight data and the generation of images are continuous dynamic processes. If the image is directly obtained from the data generation end for display, it may cause display jitter, flicker, or even data loss due to data processing delays, fluctuations, etc. The existence of the buffer enables the display module to read the image data at a stable rate, ensuring smooth display. The synthesized flight display image contains rich flight attitude and navigation information, and the data volume is large. In order to transfer these image data to the multi-functional display screen completely in a short time, a high-speed data transmission interface is required. The LVDS interface has the characteristic of high-speed transmission, and its data transmission rate can meet the requirements of real-time display of flight images, ensuring that the pilot can obtain the latest flight information in a timely manner. By observing the image on the display screen, the pilot can intuitively understand key information such as the flight attitude, flight route, and geographical location of the aircraft, so as to make correct flight decisions in a timely manner and ensure flight safety.

[0076] In summary, the integrated display device 100 based on the FPGA and dual-ARM processor architecture according to the embodiments of the present application is described. It first collects the flight attitude data of the target aircraft and transmits it to the first ARM processor through the MIPI interface. After the first ARM processor preprocesses the data, it stores the data in the input buffer of the dual-port shared memory. Then, the FPGA chip reads the preprocessed data from the input buffer to generate a flight attitude indication image and writes it into the output buffer. The second ARM processor reads the attitude indication image from the output buffer and performs image synthesis based on semantic feature interaction with the navigation image to obtain the synthesized flight display image, and stores it in the display frame buffer. Finally, the synthesized image is read from the display frame buffer and transmitted to the cockpit multifunctional display screen through the LVDS interface for display. In this way, more efficient, accurate, and reliable flight information display can be achieved.

Claims

1. A comprehensive display device based on FPGA and dual ARM processor architecture, characterized in that: include: A data input module, used to collect a time series set of flight attitude data of the target aircraft object and input it into the first ARM processor through a MIPI interface; A data preprocessing module, used for preprocessing the timing set of the flight attitude data through the first ARM processor and writing it into the input buffer of the dual-port shared memory; A flight attitude indication image generation module, used for reading the pre-processed timing set of the flight attitude data from the input buffer through the FPGA chip, and writing the flight attitude indication image into the output buffer of the dual-port shared memory after generating the flight attitude indication image; A flight image synthesis module, used for reading the flight attitude indication image from the output buffer through the second ARM processor, and performing flight image synthesis based on the interaction of key semantic features on the flight attitude indication image and the navigation image to obtain a synthesized flight display image, and writing the synthesized flight display image into a display frame buffer; The display module is used to read the synthesized flight display image from the display frame buffer area, and send the synthesized flight display image to the multi-function display screen in the cockpit for display through the LVDS interface.

2. The integrated display device based on FPGA and dual ARM processor architecture according to claim 1, characterized in that: The flight attitude data includes a pitch angle, a roll angle and a heading angle.

3. The integrated display device based on FPGA and dual ARM processor architecture according to claim 2, characterized in that: The flight image synthesis module comprises: An image preprocessing unit, used for performing image resolution enhancement on the flight attitude indication image and the navigation image to obtain a preprocessed flight attitude indication image and a preprocessed navigation image; A flight attitude feature extraction unit, used for extracting flight attitude features from the pre-processed flight attitude indication image to obtain a flight attitude feature vector; A navigation state feature extraction unit, used for extracting navigation state features from the pre-processed navigation image to obtain a navigation state feature vector; A flight attitude-navigation state feature interaction unit, used for performing a rapid interaction analysis of key semantics between flight attitude and navigation state on the flight attitude feature vector and the navigation state feature vector to obtain a flight attitude-navigation state feature interaction coding vector; The flight image synthesis unit is used to obtain the synthesized flight display image based on the flight attitude-navigation state feature interactive coding vector.

4. The integrated display device based on FPGA and dual ARM processor architecture according to claim 3, characterized in that: The flight attitude feature extraction unit is used to: pass the preprocessed flight attitude indication image through a flight attitude feature extractor based on a MobileNet model to obtain the flight attitude feature vector.

5. The integrated display device based on FPGA and dual ARM processor architecture according to claim 4, characterized in that: The navigation state feature extraction unit is used to: pass the pre-processed navigation image through a navigation state feature extractor based on a ViT model to obtain the navigation state feature vector.

6. The integrated display device based on FPGA and dual ARM processor architecture according to claim 5, characterized in that: The flight attitude-navigation state feature interaction unit comprises: A feature implicit coding subunit, used for performing principal component implicit coding and implicit feature extraction on the flight attitude feature vector and the navigation state feature vector respectively to obtain a set of flight attitude feature principal component implicit coding vectors and a navigation state feature coding vector; A flight attitude navigation state implicit feature anchoring subunit, used for anchoring the flight attitude-navigation state implicit key clues to the set of the navigation state feature coding vector and the flight attitude feature principal component implicit coding vector to obtain a feature pair of {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector}; The flight attitude-navigation state feature fine-grained interaction subunit is used to perform flight attitude-navigation state feature fine-grained interaction on the flight attitude feature vector and the navigation state feature vector based on the feature pair of {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector} to obtain the flight attitude-navigation state feature interaction coding vector.

7. The integrated display device based on FPGA and dual ARM processor architecture according to claim 6, characterized in that: The feature implicit coding subunit is used for: Performing a principal component analysis on the flight attitude feature vector to obtain a set of flight attitude feature principal component encoding vectors; Performing flight attitude point convolution implicit feature extraction on each flight attitude feature principal component coding vector in the set of flight attitude feature principal component coding vectors to obtain the set of flight attitude feature principal component implicit coding vectors; Performing navigation state point convolution implicit feature extraction on the navigation state feature vector to obtain the navigation state feature encoding vector.

8. The integrated display device based on FPGA and dual ARM processor architecture according to claim 7, characterized in that: The flight attitude-navigation state feature fine-grained interaction subunit is used to: Performing a weak virtualization semantic correction based on the eigenvalue on the feature pair of {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector} to obtain a corrected feature pair of {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector}; Based on the corrected {navigation state implicit feature coding vector, anchored flight attitude principal component implicit feature coding vector} feature pair, a flight attitude-navigation state feature fine-grained interaction is performed on the flight attitude feature vector and the navigation state feature vector to obtain the flight attitude-navigation state feature interaction coding vector.

9. The integrated display device based on FPGA and dual ARM processor architecture according to claim 8, characterized in that: The flight image synthesis unit is used to: pass the flight attitude-navigation state feature interactive coding vector through an AIGC-based flight display synthesizer to obtain the synthesized flight display image.

Images