Automated Test Method for Airborne Display System Based on ARINC818

Abstract

The present invention discloses an automated testing method for an airborne display system based on ARINC818. The coordinate positions of the graphic regions of the objects to be tested are extracted 1:1 from the ARINC818 standard images to determine whether the objects to be tested use image recognition or character recognition; the coordinate position data of the graphic regions of the objects to be tested are associated with the expected results of the test cases; an image standard library and an OCR grammar library are established to save the standard images and characters of the expected results of the test cases; automated testing is performed, and the test results are verified through image comparison and OCR recognition. The present invention has good versatility and portability. Only by marking the object regions in the new display configuration items and referring to the object regions in the test cases can the configuration and application of the present invention be completed.

Description

Technical Field

[0001] The present invention relates to the field of avionics system and display system verification, and particularly to an automatic test method for an airborne display system based on ARINC818. Background Art

[0002] With the continuous increase in the functions of modern aircraft and the number of related electronic devices, the display and interaction logics of the Cockpit Display System (CDS) have become increasingly complex. Therefore, how to more comprehensively and efficiently implement the testing of the CDS throughout the development process has become an urgent problem to be solved.

[0003] During the daily CDS testing process, a combination of test scripts and manual observation (picture changes) is usually adopted. However, with the growth of the number of test scripts and the accelerating iteration and update speed of the software under test, the efficiency and accuracy of this method will gradually be greatly affected. With the rise of machine vision technology in recent years, the accuracy and speed of image processing technology have been greatly improved. In addition, the introduction of the ARINC818 high-definition video transmission bus in the airborne display system provides clear, low-noise and stable raw data for the image recognition test. Therefore, this technology is introduced in the CDS testing process to replace manual observation as the means for determining test results, so as to further improve the efficiency and accuracy of testing and reduce costs. Summary of the Invention

[0004] Aiming at the problems that the current research and development level of airborne display systems is relatively high and high reliability and high efficiency verification are required, the purpose of the present invention is to provide an automatic test method for an airborne display system based on ARINC818. For the display requirements of different configuration items of the airborne display system, the image of the original configuration item is obtained through the ARINC818 image comprehensive processing module, the coordinate area position of the test object is marked on the coordinate system of the image through the marking software and the position data is saved, and then the position data of the test object is referenced by the expected result of the test case; for the test cases of image comparison, the test cases are run for the first time, and screenshots of the corresponding area positions are obtained through the ARINC818 image comprehensive processing module, and a standard image library is established through manual confirmation. For the test cases of OCR recognition, a character grammar standard library is established; finally, the test cases are run, and the automatic verification of the test results is completed by automatically calling the image comparison and OCR algorithms of the standard image library. The present invention is based on the original high-definition ARINC818 images, is not interfered by external environmental factors, has extremely low image noise, and has extremely high stability and reliability, providing a general, flexible, convenient and low-cost way for the verification of airborne display systems.

[0005] The invention purpose of the present invention is achieved through the following technical solutions:

[0006] An automated test method for an airborne display system based on ARINC818, comprising the following steps:

[0007] Step 1): Extract the coordinate positions of the graphic regions of the objects under test from the ARINC818 standard image at a ratio of 1:1 to determine whether the objects under test use image recognition or character recognition;

[0008] Step 2): Associate the coordinate position data of the graphic regions of the objects under test with the expected results of the test cases;

[0009] Step 3): Establish an image standard library and an OCR grammar library, and save the standard images and characters of the expected results of the test cases;

[0010] Step 4): Execute the automated test; wherein:

[0011] For the regions of image recognition, the automated test platform automatically runs the test cases, then automatically reads the position information of the object regions of the test cases, and monitors the images at this position output by the objects under test in real time. Compare this image with the standard image of the expected results of the test cases in the image standard library to verify whether the test case meets the expected results;

[0012] For the regions of OCR recognition, the automated test platform automatically runs the test cases, automatically reads the region position information of the test case objects, recognizes the characters in the region through the OCR algorithm, and recognizes the color of the font through the character edge calculation extraction algorithm. After calibration processing by the OCR grammar library, then compare with the characters in the expected results of the test cases to verify whether the test case meets the expected results.

[0013] Preferably, step 1 specifically includes:

[0014] Intercept the original complete display page of the configuration item to be tested from the ARINC818 standard image to ensure that the pixel position coordinate information of the marked image is consistent with the pixel position coordinate information of the real-time display image; for different test object graphic regions of the configuration item, respectively use the pixel annotation extraction tool to mark the boundary position coordinates of the test object region and export them to the graphic coordinate position data file.

[0015] Preferably, step 2 specifically includes:

[0016] Establish a corresponding relationship between them by referring to the saved coordinate position data of the object regions under test in the expected results of the test cases.

[0017] Preferably, step 3 specifically includes:

[0018] Establish an image standard library: By running test cases, the ARINC818 image comprehensive processing module crops the original image according to the expected object position area information included in the cases using the ROI algorithm, automatically obtains the expected result images of the test cases, and stores them in the image standard library after manually verifying the accuracy of the results;

[0019] Establish an OCR grammar library: Collect all the characters of the airborne display system to establish an OCR grammar library.

[0020] The beneficial effects of the present invention are as follows:

[0021] The present invention innovatively uses the ARINC818 signal as the picture source. Compared with the traditional camera image capture technology, it has high reliability, avoids the interference of natural environment light source noise and jitter. Therefore, the algorithm program can directly use the original image collected by the ARINC818 signal for image recognition and comparison, with higher confidence and greater stability, meeting the strict requirements of airborne system verification; The all-weather automatic test greatly improves the test efficiency, significantly reduces the labor cost and time cost. At the same time, the environmental deployment is simple and has excellent versatility, which can greatly save costs. Description of the Drawings

[0022] Figure 1 Original image test area annotation diagram;

[0023] Figure 2 Result display diagram of the exported position information of the annotation area;

[0024] Figure 3 Annotation area mapping relationship diagram;

[0025] Figure 4 Process diagram of the mapping between the use case and the annotation area;

[0026] Figure 5 Overall diagram of the annotation information in the standard library;

[0027] Figure 6 Cropped diagram of the annotation target in the standard library;

[0028] Figure 7 Schematic diagram of the processing process of the library building software;

[0029] Figure 8 Standard character library file diagram;

[0030] Figure 9 Automated test operation flow chart;

[0031] Figure 10 Overall process schematic diagram of an automated test method for an airborne display system based on ARINC818. Detailed Implementation Manner

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0033] This embodiment relates to an automated test method for an airborne display system based on ARINC818, which is mainly implemented by an ARINC818 image comprehensive processing module. Its functions include online image acquisition, information file parsing, target area acquisition, image comparison or semantic recognition, and returning test results. The specific implementation steps are as follows:

[0034] 1. Extract the coordinate positions of the graphic areas of the objects under test from the ARINC818 standard image at a ratio of 1:1

[0035] The ARINC818 image comprehensive processing module collects the ARNIC818 standard image, intercepts the original complete display page of the configuration item to be tested, and ensures that the pixel position coordinate information of the marked image is consistent with the pixel position coordinate information of the real-time display image. For different test object graphic areas of the configuration item, the pixel annotation extraction tool is used to mark the boundary position coordinates of the test object area, Figure 1 as shown, and exported as an xml format graphic coordinate position data file to facilitate the script to find the ROI pixel boundary area of the corresponding test object, such as Figure 2 as shown. Based on the requirements of the test to be performed, an information list of the test objects is extracted, including the following attribute columns: the page to which the configuration item belongs, the unique number, the test object element, the recognition method, etc., and an xlsx format mapping file is generated, such as Figure 3 as shown. Through this mapping file, the recognition method of the position area of the object under test is classified to confirm whether it is an image recognition method or a text recognition method.

[0036] The original display image obtained through the ARINC818 image comprehensive processing module is clearer, and there is no distortion and other external noise interference, thus ensuring that the marked coordinate system is completely consistent with the real-time display coordinate system, meeting the strict requirements of the airborne verification environment, and there is no need to add additional filtering algorithms or performance losses and misjudgments caused by model recognition, which is more stable and reliable.

[0037] The pixel-level marking of the position area of the test object can accurately locate all test objects in the same configuration item according to the pixel position; the refinement of the test object marking granularity can effectively prevent background interference caused by the instability of other test objects during the image recognition process.

[0038] 2. Associate the coordinate positions of the graphic areas of the objects under test with the expected results of the test cases

[0039] Annotate the use case object for information coupling. The unique identifier of the object area coordinate position information in the graphical coordinate position data file in xml format needs to be coupled and associated with the reference identifier of the test object in the use case. The reference identifier in the test case has the meaning of the actual object name. The expected result of the test case should reference the coordinate position data of the area of the object under test that has been saved, and a corresponding relationship should be established between the two.

[0040] In the expected result of the test case, reference the position data of the object under test, and project the object under test to the specific coordinate position on the display page through the mapping file. For example, the use case expected result: Displayed as Pump is ON[Fuel_L2_AC_Pump_Sym,status,Pump is ON]. The mapping process path is as Figure 4 shown. By this method, an association between the test case and the graphical position of the object under test is established, enabling the function of automatically identifying the test target area during automated testing.

[0041] 3. Establish an image standard library and an OCR (Optical Character Recognition) grammar library

[0042] For the area of image recognition, standard images in different states of this area need to be established. The process of establishing the image standard library is as follows: By running the test case, the automated testing software obtains the use case information element in the expected result, looks up the corresponding position serial number and recognition method of the element according to the mapping relation table, and then looks up the area position coordinate information of this image in the xml file through the position serial number. According to the obtained area position coordinate information, the ARINC818 image synthesis processing module captures a 1:1 screenshot image of this area in real time, saves the screenshot, and attaches image status information such as Figure 5 and Figure 6 shown. The software processing flow is as Figure 7 , and the manually confirm for the first time whether the cropped image meets the expected result. If it meets the requirement, it is stored in the standard library to improve reliability.

[0043] For the area of text recognition, since the characters of the on-board display system are a limited combination set or library, compared with the open-source OCR infinite character set algorithm, the character recognition grammar should be corrected to adapt to the character set of the on-board display; the extraction and preservation of the current on-board display character library should be used as the OCR grammar library for OCR grammar correction.

[0044] 4. Execute automated testing

[0045] The overall process of automated testing is as Figure 9, the excitation data is sent to the ARINC818 airborne display device through the automatic test software, and at the same time, the expected result information of the excitation is synchronously sent to the ARINC818 image synthesis processing module. After receiving the expected result information, the ARINC818 image synthesis processing module will compare the timestamp of this information with the timestamp of the image output by the collected ARINC818 airborne display device. If the timestamp of the collected image is after receiving the expected result information, the expected result information will be parsed to extract the coordinate position of the target area and the recognition mode information; according to the coordinate position information of the target area, the target image is extracted from the collected real-time image through ROI pixel cropping, and the extracted target image is subjected to image comparison or OCR character recognition by the parsed recognition mode information.

[0046] For the area of image comparison, the target image is compared with the standard library image, which mainly includes the following steps:

[0047] (1) Perform grayscale processing on the target image.

[0048] In order to facilitate image data processing, the color of the target image needs to be converted from color to grayscale, and the conversion formula is as follows:

[0049] Gray=R*0.299+G*0.587+B*0.114 (Formula 1)

[0050] (2) Solve the threshold segmentation point of the grayscale image.

[0051] The image segmentation algorithm uses the OTSU algorithm (maximum inter-class variance threshold segmentation algorithm) to divide the image into two parts: background and foreground; first, it is assumed that there is a threshold TH that divides all pixels of the image into two categories C1 (less than TH) and C2 (greater than TH), then the means of these two categories of pixels are m1 and m2 respectively, and the global mean of the image is mG. At the same time, the probabilities of pixels being divided into C1 and C2 categories are p1 and p2 respectively. Therefore, there is:

[0052] p1*m1+p2*m2=mG (Formula 2)

[0053] p1+p2=1 (Formula 3)

[0054] According to the concept of variance, the inter-class variance expression is:

[0055]

[0056] Finally, it can be obtained:

[0057] σ 2 =p1*p2*(m1-m2) 2 (Formula 5)

[0058] Among them:

[0059]

[0060]

[0061]

[0062] Solve for the gray level k that maximizes the squared difference of Equation 4, which is the threshold segmentation point thresh of the grayscale image.

[0063] (3) Binarize the grayscale processed target image using the threshold segmentation point.

[0064] Substitute the threshold segmentation point calculated in the previous step into the following formula, with maxval taking the value 255; after binarizing the grayscale image, it becomes a black and white region image.

[0065]

[0066] (4) Perform an opening operation on the binarized target image.

[0067] There may be some noise in the binarized target image. Therefore, first process it through the erosion formula 10 and then through the dilation formula 11 to eliminate the noise in the target image and output X'.

[0068] (Formula 10)

[0069] (Formula 11)

[0070] X is the target image and S is the structuring element.

[0071] (5) Perform a masking operation on the target image after the opening operation to extract the region with comparative significance.

[0072] After the opening operation, the target image retains the characteristic detail regions of the original image. These regions become the mask, and by performing a pixel AND operation with the original color image, the region with comparative significance in the original image can be extracted.

[0073] X masked = X orgin ∩ X' (Formula 12)

[0074] (6) Extract the positions of the feature corner points in the region with comparative significance.

[0075] Calculate the feature corner points for the extracted region with comparative significance. In this process, the Harris corner detection method is used, and the mathematical definition is as follows in Formula 13:

[0076]

[0077] After simplification by Taylor's formula, the following formula 14 is obtained:

[0078]

[0079] Finally, the Hessian matrix M is obtained as follows:

[0080]

[0081] By calculating the eigenvalues of the Hessian matrix and defining its response function as the following formula 16, the corners, edges, and smooth regions of the image can be further identified. The value of k is taken to be around 0.02 - 0.04 according to experience.

[0082] R = det M - k(traceM) 2 (Formula 16)

[0083] Where:

[0084] det M = λ1λ2 = AC - B

[0085] traceM = λ1 + λ2 = A + C

[0086] A large R value represents a corner, R < 0 represents an edge, and |R| being very small represents a smooth region; the positions of the characteristic corners of the image can be obtained through the above sliding window calculation.

[0087] (6) Determine the similarity between the target image and the standard library image at the characteristic corners.

[0088] Displacement and rotation changes: According to the differences between the positions of the characteristic corners of the standard library image and those of the captured image, judge whether the positions of the characteristic corners of the captured image relative to the standard library image have shifted or rotated. In actual engineering practice, tolerances for displacement and rotation should be set for different regional objects, and software versions with a higher maturity level should set very low or zero tolerances.

[0089] Color change: Compare the pixels in the corner region of the standard library image with those in the corner region of the actually sampled image. If rotation and displacement occur and are allowed, or if no such deviations occur and the two are consistent, it is considered to pass.

[0090] (7) Automatically write the verification results into the results of the automated test to generate the results of the image comparison test report.

[0091] For the region of OCR recognition, the automatic recognition process has the following steps:

[0092] (1) Open-source OCR character recognition.

[0093] OCR recognition technology is already a mature technology in the field of machine learning. This process uses an open-source and pre-trained model for character recognition. The drawback is that the existing model may have recognition errors for some special fonts or symbols. However, the characters displayed on the aircraft are a limited set of character combinations, so they can be corrected through this limited set.

[0094] (2) OCR grammar correction.

[0095] OCR grammar correction is a process of correcting the characters recognized by the open-source OCR model through the character combination set displayed on the aircraft. By performing grammar matching between the locally saved aircraft character library file and the recognized characters, the string with the highest target similarity will become the final recognized character combination, thus ensuring that all recognized characters are within the character combination set specified by the aircraft display system.

[0096] (3) Font color recognition.

[0097] MSER (Maximally Stable Extremal Regions) maximum stable extremal region recognition. The formula for MSER is as follows. Through the MSER algorithm, the regions of text in the target image can be obtained.

[0098]

[0099] R(i) represents a certain connected region at threshold i, △ is a small increase in the gray threshold, q(i) is the change rate of region R(i) at threshold i, and |R(i)| represents the area of region R(i). When q(i) is a local minimum, it means that the change in region R(i) is very small, so R(i) can be considered the maximum stable extremal region. Then, an external rectangle is intercepted for all MSER regions as the target recognition region.

[0100] Since there are many MSER regions, the non-maximum suppression NMS algorithm formula is as follows. Calculate the overlap degree of the multiple regions of the MSER external rectangles, remove the overlapping regions, and suppress the elements that are not maxima.

[0101] IoU = (A ∩ B) / (A ∪ B) (Formula 0)

[0102] Use the region after NMS suppression as a mask, perform a mask operation with the captured image according to Formula 12 to obtain the color region of the text, and then obtain the font color by reading the RGB values of the pixels in the region.

[0103] (4) Character comparison.

[0104] Compare the characters obtained by OCR grammar correction with the characters parsed from the expected results. If they are the same, the character comparison is successful. Then, compare the color information read from the text area with the color requirements parsed from the expected results. If they are the same, the character color comparison is successful.

[0105] (5) Automatically write the verified results into the results of the automated test to generate the results of the OCR recognition test report.

[0106] As described above, only the embodiments of the present invention are given and the present invention is not limited in any way. Any simple modifications, changes, and equivalent structural changes made to the above embodiments according to the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. An automated test method for an airborne display system based on ARINC818, comprising the following steps: Step 1): Extract the coordinate positions of the graphic regions of the objects under test 1:1 from the ARINC818 standard image to determine whether the objects under test use image recognition or character recognition methods, including: Intercept the original complete display page of the configuration item to be tested from the ARINC818 standard image output by the ARINC818 airborne display system, ensuring that the pixel position coordinate information of the marked image is consistent with that of the real-time display image; for different test object graphic regions of the configuration item, use the pixel marking extraction tool respectively to mark the boundary position coordinates of the test object region and export them to the graphic coordinate position data file; Step 2): Associate the coordinate position data of the graphic regions of the objects under test with the expected results of the test cases; including: Establish a corresponding relationship between the two by referring to the saved coordinate position data of the object region under test in the expected results of the test cases; Step 3): Establish an image standard library and an OCR grammar library to save the standard images and characters of the expected results of the test cases; including: Establish an image standard library: Run the test cases, and according to the expected object position region information contained in the test cases, crop the original image by the ROI algorithm to automatically obtain the expected result images of the test cases, and then deposit them into the image standard library after manually verifying the accuracy of the results; Establish an OCR grammar library: Collect all the characters of the airborne display system to establish an OCR grammar library; Step 4): Execute automated tests; where: For the regions of image recognition, the automated test platform automatically runs the test cases, then automatically reads the position information of the test case object regions, and monitors the images at these positions output by the objects under test in real time, and compares these images with the standard images of the expected results of the test cases in the image standard library to verify whether the test cases meet the expected results; For the regions of OCR recognition, the automated test platform automatically runs the test cases, automatically reads the region position information of the test case objects, recognizes the characters in the regions through the OCR algorithm, and recognizes the color of the font through the character edge calculation extraction algorithm, and then compares them with the characters in the expected results of the test cases after correction by the OCR grammar library to verify whether the test cases meet the expected results.

Images