An automated image verification tool for civil aircraft display systems
The automated image verification tool enables efficient and accurate image verification in civil aircraft display systems, solving the problems of subjectivity, inflexibility, and poor compatibility of traditional manual verification methods, and achieving high reliability and high precision image verification results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-06-23
AI Technical Summary
Traditional manual image verification methods cannot meet the requirements of efficient and accurate verification for civil aircraft display systems, and traditional methods suffer from subjectivity, inflexibility and poor compatibility in the image verification process.
An automated image verification tool is used, which includes an image synchronization module, an image acquisition module, an image caching module, an image difference detection module, and a verification result output module. It achieves pixel-level image verification through image processing algorithms, automatically identifying and correcting image errors.
It achieves highly reliable and high-precision image verification, can process a large amount of image data in a very short time, avoids human error, and meets the verification requirements of complex scenarios.
Smart Images
Figure CN122265810A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a civil aircraft display system, and more particularly to an automated image verification tool for a civil aircraft display system, ensuring that the graphics produced by the display system can be displayed correctly. Background Technology
[0002] The display system is the primary human-machine interface for aircraft and pilots, providing pilots with flight parameters and guidance signals required for each phase of flight, from takeoff / go-around, climb, cruise, descent, approach, and landing. Civil aviation cockpit displays have evolved from simple instrument displays to integrated displays, and from rudimentary integrated displays to highly integrated displays. As the proportion of displays, controllers, and automatic controls in civil aviation cockpit layouts increases, the cockpit has evolved from a "traditional" cockpit to a glass cockpit, and is currently in its third generation. The cockpit display systems of large civil transport aircraft are Class A systems, the highest Development Assurance Level (DAL) system, meaning the probability of the display system showing critical flight information that could mislead the pilot is less than 10%. -9 The probability of this is high. This indicator needs to be guaranteed through a rigorous system development process and through complex security design and fault-tolerant management technologies for monitoring key data information, such as integrity monitoring of the key data graphics generation process, integrity verification of key data input and output, and ARINC661 display screen freeze monitoring. The correctness of the screen display is a prerequisite for security design. Only by ensuring that the graphics produced by the display system software modules can be displayed correctly can the correctness of the security design be guaranteed, and display misjudgments be avoided.
[0003] Therefore, display system software requires high security, high reliability, and high precision. Thus, in the actual development process of display system software, it is necessary to verify the correctness of the displayed image, including whether screen elements are missing, redundant, misaligned, or whether element shapes and colors are correctly rendered. These testing and verification activities are crucial. However, in traditional testing processes, the entire testing process is completed manually. The graphic verification work, which relies on visual inspection to find differences, cannot meet the efficient and accurate verification requirements of avionics display systems, nor can it meet the needs of rapid iterative development of display system software. Furthermore, traditional testing methods often have the following problems in the verification work during the development phase: 1) In the early stages, all image verification processes were conducted manually to verify the consistency between the drawn images and the requirements. The judgment of image correctness was often based on the subjectivity of the testers, and there was a lack of precise verification standards. 2) Using image annotation involves marking the specific location of each UI component and then identifying the location of related UI controls. This requires a lot of manpower to annotate the images in the early stages, and each version change requires re-annotation. It has poor flexibility and compatibility and cannot meet the verification work in the process of product prototype iteration development.
[0004] Based on this, the present invention constructs a high-precision automated image verification tool that supports the verification of two video streams, automatically completes the acquisition, verification, and automatic reporting of image verification results, and has high reliability and accuracy, ensuring the consistency of verification rules. Summary of the Invention
[0005] The purpose of this invention is to provide an automated image verification tool for civil aircraft display systems. Based on image processing algorithms, it achieves pixel-level image verification, which can promptly detect and correct errors in images, ensuring that pilots receive accurate information and meeting the requirements for high reliability and high precision verification.
[0006] The objective of this invention is achieved through the following technical solution: An automated image verification tool for civil aircraft display systems includes an image synchronization module, an image acquisition module, an image caching module, an image difference detection module, and a verification result output module. The image synchronization module is used to continuously send synchronization signals outwards; The image acquisition module synchronously acquires lossless image data from two video streams based on the synchronization signal and saves it to the hard disk; one stream is the image data rendered to the screen by the GPU and read from the IDU video memory as the image to be tested, and the other stream is the simulated image data as the standard image; The image caching module reads the two image data streams from the hard disk into memory in a fixed data format according to the verification rate; The image difference detection module obtains the pixel difference between the image under test and the standard image by performing pixel subtraction, then removes noise interference and highlights the difference information through various graphics operations, and finally marks each difference information explicitly with a bounding box. The image verification result output module is used to record difference information.
[0007] Preferably, the synchronization signal is also used as a unique identifier for the standard image and the image under test to uniquely identify each frame of the image in the subsequent process.
[0008] Preferably, the image acquisition module includes a single-board acquisition and transmission submodule, a network driver submodule, a PC-side image receiving and storage submodule, a SERVERIOM submodule, and a SERVER simulation acquisition submodule; The single-board acquisition and transmission submodule is used to simultaneously read the image data rendered to the screen by the GPU from the IDU video memory whenever it receives a synchronization signal sent by the synchronization module, compress it, and then send it to the PC-side image receiving and saving submodule via UDP. The network driver submodule acts as a virtual partition, communicating with other partitions via virtual ports. It is responsible for sending and receiving image data and the stimulus data sent by the 661UA data stimulus tool.
[0009] The PC-side image receiving and saving submodule receives image data, reassembles and decompresses it, and saves it to the hard drive. The SERVERIOM module parses the incentive data sent by the 661UA data incentive tool and sends it to the A661 SERVER; The SERVER simulation acquisition module collects simulated image data and saves it to the hard drive.
[0010] Preferably, the single-board acquisition and transmission submodule and the PC-side image receiving and storage submodule act as the compression and decompression units for the "image to be tested," respectively, and employ the following compression and decompression algorithms: During compression, the alpha channel of each pixel is checked first. If the alpha channel value of the pixel is zero, the number of consecutive pixels with a alpha channel value of zero is recorded, and the original pixel data is discarded. Otherwise, the original pixel data is retained. During decompression, the alpha channel of each pixel is checked. If the alpha channel value is zero, the pixel is filled with zeros. Otherwise, the original pixel is retained.
[0011] Preferably, the single-board acquisition and transmission submodule and the PC-side image reception and storage submodule act as the sender and receiver of the "image under test", respectively, and adopt the following image transmission protocol: The single-board acquisition and transmission submodule splits the compressed image data into multiple UDP data packets for transmission. Each UDP data packet contains: Head: The data packet header, indicating the start of the data packet; Fsn: Frame sequence number, ensuring the order of data; Total: The number of packets in which the frame image data is segmented; Index: Represents the data packet index number of this frame of image; Len: Indicates the length of the valid data; Data: Acquired image data; CRC: Checksum of image data; Eof: End of data packet, indicating the end of the data packet; The PC-side image receiving and saving submodule performs the following three integrity checks upon receiving an image: Data content integrity: The original data is calculated using the same CRC generating polynomial as the sender. If the calculation result matches the CRC in the received UDP data packet, the data frame is retained; otherwise, the data frame is discarded and a retransmission is requested. Sequential integrity: First, check if the index bit of the currently received UDP packet is 1. If not, discard the packet. Continue until the index of the received packet is 1. Retain the FSN, total, and index of the previous UDP packet, and then continue receiving the next packet. If the FSN of the current data is the same as the FSN of the previous packet, and the index is continuously increasing, retain the packet; otherwise, discard it. When the index equals the total, it is considered that a complete frame has been received, and then it is saved as an image. Time integrity: Whether UDP packets are received within the specified delay range.
[0012] Preferably, the CRC generator polynomial is: and .
[0013] Preferably, the detection process of the image difference detection module is as follows: Step 1) Convert the read "standard image" and "image to be tested" from three-channel 24-bit RGB images to single-channel 8-bit grayscale images; Step 2) Traverse all pixels of the "standard image" and the "image to be tested", subtract the pixel values at corresponding positions, and take the absolute value; Step 3) Binarize the image result obtained in Step 2) to obtain a binary image; Step 4) Perform median filtering on the binary image to remove noise; Step 5) Use the closing operation of computer graphics to first dilate and then erode, so that discrete but closely spaced pixels are merged. Step 6) Locate the outline of the differing parts; Step 7) Perform polygon fitting; Step 8) Use a rectangle to outline each difference and draw the rectangle on the corresponding "image to be tested".
[0014] Preferably, the image verification result output module records the difference information, including the number of differences, the type of differences, and the location of the differences, and outputs an image with a bounding box and a Log file.
[0015] The beneficial effects of this invention are as follows: This invention accurately identifies image differences by precisely verifying each pixel of an image, achieving efficient automatic image testing at 10 frames per second. This technology not only processes large amounts of image data in a very short time but also effectively avoids subjective judgment errors caused by human factors, ensuring the consistency and objectivity of the verification results. Furthermore, parameters and algorithms can be flexibly adjusted according to the needs of different display system application scenarios to meet the image calibration requirements of various complex scenarios. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the module composition of an automated image inspection tool.
[0018] Figure 2 This is a schematic diagram of the sub-functional modules of the image acquisition module.
[0019] Figure 3 This is a schematic diagram of a frame data packet.
[0020] Figure 4 This is a schematic diagram of the image data sending and receiving ends.
[0021] Figure 5 This describes the image detection and processing workflow.
[0022] Figure 6 This is the RGB image of the "image to be tested".
[0023] Figure 7 This is a grayscale image of the "image to be tested".
[0024] Figure 8 shows the effect of "pixel subtraction".
[0025] Figure 9 This is a binary image.
[0026] Figure 10 This is a contour map of the differences.
[0027] Figure 11 This is a diagram illustrating the differences. Detailed Implementation
[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0030] This embodiment illustrates an automated image verification tool for civil aircraft display systems. It employs an automatic sampling and comparison method involving two video streams to accurately identify and pinpoint the differences between the "standard image" and the "image under test," generating a corresponding difference test report. For example... Figure 1As shown, the automated image verification tool consists of five main modules: image synchronization module, image acquisition module, image caching module, image difference detection module, and verification result output module.
[0031] The image synchronization module's main function is to continuously send synchronization signals. Through UDP communication, it sends a unified signal variable, Sync, to the image acquisition module. The Sync value controls the acquisition unit to synchronously sample images from the two video streams. This is especially crucial during dynamic image verification, ensuring the synchronization of a set of images acquired from the two video streams; otherwise, the verification will be ineffective. Simultaneously, the Sync value serves as a unique identifier for each frame in the subsequent process, facilitating subsequent processing and troubleshooting.
[0032] The image acquisition module is primarily responsible for synchronously acquiring lossless image data from two video streams based on a synchronization signal, and converting it into a specified format to save it to the hard drive. One stream is the image data rendered to the screen by the GPU from the IDU's video memory, serving as the "image to be tested," while the other stream is the simulated image data, serving as the "standard image."
[0033] The image caching module's main function is to read two image data streams from the hard drive into memory in a fixed data format according to the verification rate. Since the image sampling rate and the image verification rate cannot be guaranteed to be completely consistent, in order to enable sampling and verification to be performed asynchronously, the sampled image data is first cached in the hard drive, and then read from the hard drive into memory during verification.
[0034] The image difference detection module is the core of the entire image verification tool. Its main function is to identify and locate the differences between the "standard image" and the "image to be tested" through image processing algorithms. To ensure high accuracy in image verification, after image preprocessing, the image difference detection module obtains the pixel differences by performing "pixel subtraction" between two frames. Then, through graphics operations such as binarization, median filtering, closing operation, contour finding, polygon fitting, and bounding box drawing, noise interference is eliminated and the difference information is highlighted. Finally, each difference is explicitly marked with an identification box.
[0035] The image verification result output module's main function is to record difference information. This allows developers to quickly and easily locate the various difference data of the "image under test" after discovering a difference issue, including the number, type, and location of the differences. The module's output includes an image with a bounding box and a log file.
[0036] As an example, such as Figure 2As shown, in this embodiment, the image acquisition module consists of sub-modules such as a single-board acquisition and transmission sub-module, a network driver sub-module, a PC-side image receiving and saving sub-module, a SERVERIOM sub-module, and a SERVER simulation acquisition sub-module. Single-board acquisition and transmission submodule: The main function of this module is to simultaneously read the image data rendered to the screen by the GPU from the IDU video memory whenever a Sync signal sent by the synchronization module is received, compress it, and then send it to the PC-side image receiving and storage submodule via UDP. Based on the pixel data currently drawn by the IDU application, at a rate of 60 frames per second, a bandwidth of 365M / s is required for real-time image transmission. Since the hardware conditions do not meet this requirement, image data compression and a corresponding transmission protocol need to be designed to ensure stable and complete data transmission.
[0037] Network driver submodule: The main function of this module is to act as a virtual partition, communicate with each partition through virtual ports, and be responsible for sending and receiving image data and stimulus data sent by the 661UA data stimulus tool.
[0038] PC-side image receiving and saving submodule: The main function of this module is to receive image data, reassemble it, and decompress and save it as a lossless image. The image data format is a three-channel 24-bit RGB image, which is saved to the hard drive as one input of the image verification tool.
[0039] SERVERIOM Module: This module is ported from the IOM board to the GPM board and changes the inter-board shared memory between SERVERIOM and SERVER to intra-board shared memory. Its main function is to parse the stimulus data sent by the 661UA data stimulus tool and send it to the A661 SERVER.
[0040] SERVER Simulation Acquisition Module: This module is mainly responsible for acquiring simulation image data, running on the PC, and saving it to the hard drive as another input for the image verification tool.
[0041] The single-board acquisition and transmission submodule and the PC-side image receiving and storage submodule act as the compression and decompression units for the "image to be tested," respectively, and employ the following compression and decompression algorithms: The data format of IDU rendered images is a four-channel 32-bit RGBA image with a resolution of 1680×1050. Therefore, the data size of a single frame is approximately 6MB, which is too large for transmission. During compression, the alpha channel of each pixel is checked first. If the alpha channel value of a pixel is zero, the number of consecutive pixels with a zero alpha channel value is recorded, and the original pixel data is discarded; otherwise, the original pixel data is retained. After compression using the above algorithm, the data size of each frame is reduced to 800KB, achieving a compression ratio of approximately 87%. During decompression, the alpha channel of each pixel is checked. If the alpha channel value is zero, the pixel is filled with zeros; otherwise, the original pixel is retained. The compression algorithm is shown below: Algorithm 1 image compression algorithm Input: Number of pixels, Uncompressed pixel data Output: compressed pixel data 1: for(i=0; I< pixelCount; i++) 2: if(pixel& 0xFF000000)==0) 3: Discard the original pixel data; 4: else 5: Keep the original pixel data. The decompression algorithm is as follows: Algorithm 2 image decompression algorithm Input: Number of pixels, compressed pixel data. Output: Uncompressed pixel data. 1: for(i=0; I <pixelCount; i++) 2: if(pixel&0xFF000000)==0 3: The pixel fill is zero; 4: else 5: Keep the original pixel data. The single-board acquisition and transmission submodule and the PC-side image reception and storage submodule act as the sender and receiver of the "image under test," respectively, and adopt the following image transmission protocol: The single-board acquisition and transmission submodule splits the compressed image data into multiple UDP data packets for transmission. The UDP data packets are as follows: Figure 3 As shown, it includes: Head: The data packet header, indicating the start of the data packet. Fsn: Frame sequence number, ensuring the order of data.
[0042] Total: The number of packets in which the frame image data is segmented. Since the maximum length of UDP transmission is approximately 64kb, and our single frame image is often larger than 64kb, the image data needs to be segmented for transmission.
[0043] Index: Represents the data packet index number of this frame image.
[0044] Len: Indicates the length of the valid data.
[0045] Data: The collected image data.
[0046] CRC: Checksum of image data.
[0047] Eof: The end of the data packet, indicating the end of the data packet.
[0048] When receiving images, the PC-side image receiving and saving submodule needs to consider the following three aspects of integrity verification: a) Data content integrity The data content of the message packet is not corrupted during transmission. The integrity check of the UDP payload is implemented using a CRC checksum. The checksum is used to ensure data integrity during transmission. It is calculated using a specified algorithm to obtain a check value from the original data. When the receiver receives the data, it uses the same checksum algorithm to calculate the same value. If the calculated result matches the received checksum, the data is verified correctly and the data frame can be used. If they do not match, an error occurred during transmission, the data frame is discarded, and a retransmission is requested. The CRC generator polynomial can be determined by configuring 16-bit polynomial coefficients. The default coefficient of the highest bit is 1, and it is best to choose two different polynomials to cover different undetectable error modes, ensuring that any change in any data bit within the CRC calculation range will result in a different CRC checksum. This protocol uses the following two international standard CRC-16 generator polynomials: and .
[0049] b) Sequential integrity Message packets are received in the order they were sent. Order integrity is checked by comparing the fsn, total, and index of the currently received message with those of the previously received messages. At the receiving end, the index bit of the currently received data packet is first checked to see if it is 1. If not, it indicates that the image data for that frame is incomplete, and the packet is discarded. This process continues until the index of a received data packet is 1. The fsn, total, and index of the previous data are retained, and then the next data packet is received. If the fsn of the current data packet is the same as the fsn of the previous packet, and the index is continuously increasing, the data packet is retained; otherwise, it is discarded. When the index equals the total, a complete frame is considered to have been received and is then saved as an image.
[0050] c) Temporal integrity: Message packets are received within a limited delay range. Time synchronization is required before time integrity checks, because when terminals of different devices exchange messages, their clock sources are asynchronous. To ensure the reliability of time integrity checks, time synchronization is required through network time synchronization technology, relying on continuously receiving synchronization signals to ensure time consistency.
[0051] The process of sending and receiving image data is as follows: Figure 4 As shown.
[0052] As an example, to achieve pixel-level accurate verification, this embodiment uses a pixel comparison method based on image processing algorithms to detect the differences between the "standard image" and the "image to be tested". Specifically, the two images are first preprocessed, then pixel differences between the two images are obtained through "pixel subtraction", and finally, a series of graphics operations are used to identify and locate the differences. The specific technical process is as follows: Figure 5 As shown: The read "standard image" and "image to be tested" are converted from three-channel 24-bit RGB images to single-channel 8-bit grayscale images, such as... Figure 6 , Figure 7 As shown.
[0053] Iterate through all pixels in both frames of the image, subtract the pixel values at corresponding positions, and take the absolute value. Figure 8 As shown.
[0054] Binarize the image result obtained in step 2) to obtain a binary image, as shown below. Figure 9 As shown.
[0055] Median filtering is applied to the binary image to remove noise.
[0056] Using the closing operation of computer graphics, first dilate and then erode to merge discrete but closely spaced pixels.
[0057] Use the findContours function to find the contours of the differing parts, such as... Figure 10 As shown.
[0058] Use the approxPolyDP function for polygon fitting.
[0059] Use a rectangle to outline each difference, and then draw the rectangle on the corresponding location of the "image to be tested". The final difference recognition result is shown in the image below. Figure 11 As shown.
[0060] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.
Claims
1. An automated image verification tool for civil aircraft display systems, characterized in that... It includes an image synchronization module, an image acquisition module, an image caching module, an image difference detection module, and a verification result output module; The image synchronization module is used to continuously send synchronization signals outwards; The image acquisition module synchronously acquires lossless image data from the two video streams according to the synchronization signal and saves it to the hard disk; in One path is the image data rendered to the screen by the GPU and read from the IDU's video memory, which serves as the image to be tested; the other path is the simulated image data, which serves as the standard image. The image caching module reads the two image data streams from the hard disk into memory in a fixed data format according to the verification rate; The image difference detection module obtains the pixel difference between the image under test and the standard image by performing pixel subtraction, then removes noise interference and highlights the difference information through various graphics operations, and finally marks each difference information explicitly with a bounding box. The image verification result output module is used to record difference information.
2. The automated image verification tool for civil aircraft display systems according to claim 1, characterized in that... The synchronization signal is also used as a unique identifier for the standard image and the image under test to uniquely identify each frame of the image in the subsequent process.
3. The automated image verification tool for civil aircraft display systems according to claim 1, characterized in that... The image acquisition module includes a single-board acquisition and transmission submodule, a network driver submodule, a PC-side image receiving and saving submodule, a SERVERIOM submodule, and a SERVER simulation acquisition submodule. The single-board acquisition and transmission submodule is used to simultaneously read the image data rendered to the screen by the GPU from the IDU video memory whenever it receives a synchronization signal sent by the synchronization module, compress it, and then send it to the PC-side image receiving and saving submodule via UDP. The network driver submodule acts as a virtual partition, communicating with other partitions via virtual ports. It is responsible for sending and receiving image data and the stimulus data sent by the 661UA data stimulus tool. The PC-side image receiving and saving submodule receives image data, reassembles and decompresses it, and saves it to the hard drive. The SERVERIOM module parses the incentive data sent by the 661UA data incentive tool and sends it to A661SERVER; The SERVER simulation acquisition module collects simulated image data and saves it to the hard drive.
4. The automated image verification tool for civil aircraft display systems according to claim 3, characterized in that... The single-board acquisition and transmission submodule and the PC-side image receiving and storage submodule act as the compression and decompression units for the "image to be tested," respectively, and employ the following compression and decompression algorithms: During compression, the alpha channel of each pixel is checked first. If the alpha channel value of the pixel is zero, the number of consecutive pixels with a alpha channel value of zero is recorded, and the original pixel data is discarded. Otherwise, the original pixel data is retained. During decompression, the alpha channel of each pixel is checked. If the alpha channel value is zero, the pixel is filled with zeros. Otherwise, the original pixel is retained.
5. The automated image verification tool for civil aircraft display systems according to claim 3, characterized in that... The single-board acquisition and transmission submodule and the PC-side image receiving and saving submodule act as the sender and receiver of the "image to be tested," respectively, and adopt the following image transmission protocol: The single-board acquisition and transmission submodule splits the compressed image data into multiple UDP data packets for transmission. Each UDP data packet contains: Head: The data packet header, indicating the start of the data packet; Fsn: Frame sequence number, ensuring the order of data; Total: The number of packets in which the frame image data is segmented; Index: Represents the data packet index number of this frame of image; Len: Indicates the length of the valid data; Data: Acquired image data; CRC: Checksum of image data; Eof: End of data packet, indicating the end of the data packet; The PC-side image receiving and saving submodule performs the following three integrity checks upon receiving an image: Data content integrity: The original data is calculated using the same CRC generating polynomial as the sender. If the calculation result matches the CRC in the received UDP data packet, the data frame is retained; otherwise, the data frame is discarded and a retransmission is requested. Sequential integrity: First, check if the index bit of the currently received UDP packet is 1. If not, discard the packet. Continue until the index of the received packet is 1. Retain the FSN, total, and index of the previous UDP packet, and then continue receiving the next packet. If the FSN of the current data is the same as the FSN of the previous packet, and the index is continuously increasing, retain the packet; otherwise, discard it. When the index equals the total, it is considered that a complete frame has been received, and then it is saved as an image. Time integrity: Whether UDP packets are received within the specified delay range.
6. The automated image verification tool for a civil aircraft display system according to claim 5, characterized in that... The CRC generator polynomial is: G(X) = x 16 +x 15 +x 2 +1 and G(X) = x 16 +x 12 +x 5 +1.
7. The automated image verification tool for civil aircraft display systems according to claim 1, characterized in that... The detection process of the image difference detection module is as follows: Step 1) Convert the read "standard image" and "image to be tested" into grayscale images; Step 2) Traverse all pixels of the "standard image" and the "image to be tested", subtract the pixel values at corresponding positions, and take the absolute value; Step 3) Binarize the image obtained in Step 2) to obtain a binary image; Step 4) Perform median filtering on the binary image to remove noise; Step 5) Use the closing operation of computer graphics to first dilate and then erode, so that discrete but closely spaced pixels are merged. Step 6) Locate the outline of the differing parts; Step 7) Perform polygon fitting; Step 8) Use a rectangle to outline each difference and draw the rectangle on the corresponding "image to be tested".
8. The automated image verification tool for civil aircraft display systems according to claim 1, characterized in that... The image verification result output module records the difference information, including the number of differences, the type of differences, and the location of the differences, and outputs an image with a bounding box and a Log file.