Automatic test method for display picture of avionics display product based on machine vision

Abstract

The invention discloses a machine vision-based automatic test method for a display picture of an avionics display product. The method comprises the following steps of: shooting a real-time display picture of a tested avionics display product through an industrial camera; a pre-trained artificial neural network is adopted to judge the position of the tested avionics display product in the shot picture, and an interested test area is found; carrying out quantitative analysis and comprehensive evaluation on the test area through an image processing and analysis method provided in the OpenCV library to obtain quantitative data of screen brightness and luminescence color; and extracting features of the test area based on an optical character recognition technology, and recognizing and judging characters, numbers and English letters contained in the test area. According to the invention, automatic identification and testing of screen display contents of avionics display products are realized, instability caused by subjective factors of testers is reduced, and quantitative analysis of picture quality is realized.

Description

Technical Field

[0001] The invention belongs to the field of avionics product testing, and relates to an automatic testing method for display screens of avionics display products based on machine vision. Background Art

[0002] Avionics display products are important components of avionics equipment display and control systems, and are mainly used to display various information and data during aircraft flight missions. In recent years, avionics display products have shown development trends such as integration, digitization, networking, automation, intelligence, high precision, and high reliability. In order to ensure that product quality meets the above requirements, the testing of avionics display products also puts forward requirements such as process automation and intelligent diagnosis.

[0003] Generally, the test equipment for avionics display products is usually in an independent offline environment, and is generally composed of basic units such as a main control unit, a simulation resource unit, and a signal adapter unit. The main control unit provides a main control computer, runs the test software, issues test instructions, and provides corresponding power supply for the equipment; the simulation resource unit is usually composed of an industrial computer and various types of simulation resource boards, which completes the operation of receiving instructions from the main control computer and sending and receiving data with the tested product; the main function of the signal adapter unit is to collect and connect different hardware resources to achieve physical circuit connectivity. These basic units can realize information communication with the tested product according to the test instructions, complete automatic data transmission and reception, and automatically determine the test results according to the test standards. However, for avionics display products, in addition to automatic data transmission and reception, it is usually necessary to interact with people (human eye comparison, judgment of display screen, etc.). Therefore, the current testing of such avionics display products usually requires testers to participate in the test on site. Only by observing the difference between the product display screen and the preset screen can it be judged whether the product screen display content meets the design requirements. Due to the need for human eye comparison, it is difficult to realize automation and intelligence in the testing of display screens. Moreover, the testing method based on human visual observation and comparison is greatly affected by human subjectivity, and the accuracy and reliability of the judgment results are difficult to effectively quantify.

[0004] At present, image recognition and image comparison methods based on machine vision have been widely used in commercial applications, especially image recognition algorithms based on deep learning such as artificial neural networks. After training with a large amount of data, they have made great progress in license plate recognition, face recognition, etc. However, for aviation electronic products, due to the relative lack of images and specific display content of related commercial products, it is difficult to obtain neural networks specifically used to identify avionics products through training. Therefore, there is still a lack of targeted research on the automated testing of avionics display products.

[0005] In order to realize the automated testing of the information displayed on the screen of avionics display products, we can learn from the more mature methods in commercial development and deploy them offline in the test equipment to meet different test requirements. This requires the comprehensive application of multiple machine vision algorithms and the development of automated testing technologies for images in avionics display screens, such as visual inspection based on feature extraction, image color analysis based on color space transformation, etc., so as to realize objective evaluation and automated judgment of the display screen. Summary of the invention

[0006] The purpose of the present invention is to provide an automatic testing method for the display screen of avionics display products based on machine vision, which captures the real-time display screen of avionics display products through industrial cameras, evaluates the display screen of avionics display products from different angles according to different machine vision algorithms, and automatically determines whether the image screen meets the relevant design standards.

[0007] The object of the present invention is achieved through the following technical solutions:

[0008] An automatic testing method for display screens of avionics display products based on machine vision comprises the following steps:

[0009] Step 1: Use an industrial camera to capture the real-time display image of the avionics display product being tested;

[0010] Step 2: Use the pre-trained artificial neural network to determine the position of the avionics display product under test in the captured image and find the test area of ​​interest;

[0011] Step 3: Use the image processing and analysis methods provided in the OpenCV library to perform quantitative analysis and comprehensive evaluation on the test area, obtain quantitative data on screen brightness and luminous color, and compare them with the data corresponding to the standard picture to determine whether the product is qualified;

[0012] Step 4: Extract the features of the test area based on optical character recognition technology, identify and judge the characters, numbers and English letters contained in the test area, and compare them with the data corresponding to the standard picture to determine whether the product is qualified.

[0013] Preferably, in step 2, the YOLOv3 neural network model of the DNN module in OpenCV is first used to identify the product to be tested and its position contained in the photo, then the product is cropped according to its size, and finally the YOLOv3 neural network model is used to identify the cropped image to find the test area of ​​interest.

[0014] Preferably, in step 3, OpenCV is used to analyze the grayscale distribution histogram of the test area and the standard image, and the pixel distribution similarity between the test area and the standard product display image is calculated. When the similarity between the two is large enough, the pixel distributions of the two images are considered to be similar, otherwise they are not similar, and the product to be tested is judged as unqualified;

[0015] For the brightness and color change test, the image description model is transformed through the color space conversion function in OpenCV, and the image represented by the RGB color space is converted into an image described by the HSV model. The HSV index of the measured product display image is compared to see whether it falls within the standard range.

[0016] Preferably, step four is implemented using the PaddleOcr open source library. The neural network model used to recognize characters in PaddleOcr is downloaded in advance in a network environment and then imported into the test device. The test area is then used as the input of the neural network model to automatically identify and mark the character information contained in the image, and then compared with the preset display characters to determine whether the characters displayed by the tested product are correct.

[0017] The beneficial effects of the present invention are:

[0018] According to the automatic test method of the display screen of avionics display products based on machine vision provided by the present invention, it is possible to comprehensively analyze the features of the displayed image, automatically identify the display elements of avionics display products and their screens, extract the characteristic elements such as Chinese characters and letters of the display screen, and further compare the consistency of the measured photos of the avionics display product screen with the photos of the standard product screen to determine whether the tested product meets the design requirements. The machine vision algorithm solves the problems of avionics display products requiring the participation of testers and low automation rate, realizes the automatic recognition and testing of the display content of avionics display products, reduces the instability caused by the subjective factors of testers, and realizes the quantitative analysis of image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The test logic diagram of the automatic test method for the display screen of avionics display products based on machine vision.

[0020] Figure 2 This is an avionics display product identified using the YOLOv3 neural network.

[0021] Figure 3 This is a screen area map of an avionics display product identified using the YOLOv3 neural network.

[0022] Figure 4 The keyboard light guide color features are extracted using OpenCV.

[0023] Figure 5 The image content of an avionics display product identified by PaddleOcr. DETAILED DESCRIPTION

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0025] In view of the fact that traditional testing technology requires testers to observe on-site and judge whether the display screen is qualified based on subjective perception when testing the display information of avionics display products, resulting in low automation and intelligence, strong subjectivity and other problems, this embodiment shows an automatic testing method for the display screen of avionics display products based on machine vision. By introducing deep learning technologies such as pre-trained artificial neural networks, it is possible to automatically judge whether the display screen is correct, thereby realizing automated testing of avionics display products, reducing the instability of image evaluation caused by the subjective perception of operators, increasing the objectivity of display screen evaluation, and improving the automation rate of testing.

[0026] See also Figure 1 As shown, the automatic testing method of the display screen of an avionics display product based on machine vision shown in this embodiment includes the following steps:

[0027] Step 1: Use an industrial camera to capture the real-time display image of the avionics display product under test.

[0028] Step 2: Avionics product positioning technology based on artificial neural network, using pre-trained artificial neural network to determine the position of the avionics display product under test in the captured image and find the test area of ​​interest.

[0029] To identify and analyze the information displayed on the display screen of an avionics product, it is first necessary to locate the position of the avionics product in the photo and further determine the position of the screen in the product. At present, deep learning methods based on artificial neural networks can accurately identify the position of common objects, and combined with the identified product design information, the position of the display screen to be tested can be accurately found.

[0030] Artificial neural networks have been widely used in target recognition and target tracking. Based on convolutional neural networks, they use sliding window strategies to find features in the entire image, and then use classifiers to determine whether there is a target during the scanning process, thereby achieving target capture. At present, the target recognition algorithm based on artificial neural networks is usually based on pre-trained neural networks, supplemented by dedicated images to improve the recognition accuracy of a certain class or several specific items. However, for avionics products, since product data is usually difficult to obtain, pre-trained neural networks are required to directly identify products. At the same time, in the actual testing process, there will only be a unique product under test in the image captured by the industrial camera, and there will be no items similar to the product. Therefore, it is only necessary to pay attention to whether the product under test is correctly located, and it is not necessary to pay attention to whether the product identified after positioning has the correct label. Therefore, as long as the identified object is marked as a similar object (such as mobile phones, TVs and other objects with screens), it can be considered that the model has found the location of avionics display products.

[0031] For the positioning of objects, the YOLOv3 neural network model of the DNN module in OpenCV is used. It further subdivides the network based on YOLOv2 and improves the accuracy of small target detection. When using the YOLOv3 neural network model to identify products, you must first import the configuration files related to YOLOv3, including the network structure file, network weight file, and label file, and then use the photos of the products to be tested taken by the industrial camera as the input of the YOLOv3 neural network. Running the YOLOv3 model can automatically identify the products to be tested and their locations contained in the photos.

[0032] As mentioned above, it is impossible for the YOLOv3 neural network to get the correct label for avionics display products. However, since avionics display products are very similar to TV screens or mobile phones, and there are no other similar objects in the range of photos captured by industrial cameras during the test, the mobile phone or TV screen identified by the YOLOv3 neural network is the avionics display product to be tested. Figure 2 As shown, we can see that the YOLOv3 model can accurately find the area where the product is located (and mark it as a mobile phone), which lays the foundation for further locating the tested area and identifying the display content in the tested area.

[0033] After finding the position of the product in the photo, the test area of ​​interest can be determined based on the product's size design, such as the display screen or the area of ​​the illuminated buttons of the avionics product. In order to avoid errors caused by factors such as the installation accuracy of the product fixture, the present invention preliminarily determines the test area of ​​interest based on the size design, and then combines the YOLOv3 model to intelligently identify the screen position. Similar to identifying the position of the product above, when identifying the screen, it is only necessary to roughly crop the image based on the design size based on the identified product, and then use YOLOv3 to identify the cropped image to find the position of the display screen. Figure 3 shown.

[0034] Step 3: Based on OpenCV's image processing technology, the image processing and analysis methods provided in the OpenCV library are used to perform quantitative analysis and comprehensive evaluation on the test area of ​​interest, obtain quantitative data on features such as screen brightness and luminous color, and compare them with the data corresponding to the standard image to determine whether the product is qualified.

[0035] After completing the positioning of the area to be tested, before identifying the image content, it is usually necessary to conduct a comprehensive analysis of the screen color, luminous brightness, etc., and automatically compare it with the standard image to determine whether the actual product under test meets the design requirements.

[0036] OpenCV is an image processing library based on C / C++, and it encapsulates the Python interface in the outer layer, which can be directly called by Python. OpenCV provides a wealth of image processing functions, which can be easily used to perform statistical analysis on the image of the object photo. In the process of testing avionics products, when comprehensively analyzing the product display image, it is necessary to analyze the grayscale distribution histogram of the test area of ​​interest and the standard image, and calculate the similarity of the pixel distribution of the test area of ​​interest and the standard product display image. Only when the similarity between the two is large enough (greater than 0.8), the pixel distribution of the two images can be considered similar, otherwise they are not similar, and the product to be tested is judged as unqualified.

[0037] For tests such as brightness and color changes, it is necessary to use the color space conversion function in OpenCV to realize the conversion of the image description model, and convert the image represented by the RGB color space into an image described by the HSV (Hue, Saturation, Value) model. The latter can directly and quantitatively characterize the hue (H), saturation (S), and lightness (V) of the pixels in the image. By extracting color and other features, compare the HSV index of the measured product display image to see if it falls within the standard range. Figure 4This is an image of an avionics product that turns on the light guide illumination and when the button emits green light, it is converted into the HSV color space and the pixels in the green spectrum range are filled with standard green, while the pixels in the non-green spectrum range are filled with black (binarization processing). It can be seen that the relevant methods in OpenCV can accurately find the green light-emitting area, indicating that the corresponding position does emit green light.

[0038] Step 4: Extract the features of the test area based on optical character recognition (OCR) technology, and identify and judge the characters, numbers, and English letters contained in the test area.

[0039] For avionics display products, in addition to features such as color and brightness, whether the screen can correctly display preset content (such as characters, numbers, and English letters, etc.) is also an important indicator to measure whether the product meets the requirements. The current testing method usually requires testers to observe and judge whether the product is qualified. Since the display screen needs to be observed by human eyes, the current test automation rate of this type of avionics products is relatively low.

[0040] In order to improve the automated testing of the display screen elements of avionics products, it is necessary to use OCR technology to automatically locate and identify the element features of the screen image. For the recognition of Chinese characters, it is necessary to use the PaddleOcr open source library, which was developed by the Baidu Paddle team. It integrates multiple models such as convolutional neural networks and recurrent neural networks, and has high accuracy in Chinese character recognition. When recognizing the content displayed on the screen, it is necessary to call PaddleOcr in Python and select the corresponding language model. For the testing of avionics products, the neural network model used to recognize characters in PaddleOcr needs to be downloaded in advance in a network environment and then imported into the test equipment. Then, according to the position of the display screen located above, the part of the display screen in the image is cropped out. The cropped image is used as the input of the neural network, which can automatically identify and mark the character information contained in the image. By comparing it with the preset display characters, it can be determined whether the characters displayed by the product under test are correct. The results of the feature recognition of the display screen of an avionics product are shown below. Figure 5 As shown, it can be seen that the Chinese characters, numbers, symbols, etc. displayed on the screen can be correctly identified and recognized.

[0041] It is understandable that those skilled in the art can make equivalent substitutions or changes based on the technical solution and inventive concept of the present invention, and all these changes or substitutions should fall within the protection scope of the claims attached to the present invention.

Claims

1. An automatic testing method for display screens of avionics display products based on machine vision, characterized in that The following steps are involved: Step 1: Use an industrial camera to capture the real-time display image of the avionics display product being tested; Step 2: Use the pre-trained artificial neural network to determine the position of the avionics display product under test in the captured image and find the test area of ​​interest; Step 3: Use the image processing and analysis methods provided in the OpenCV library to perform quantitative analysis and comprehensive evaluation on the test area, obtain quantitative data on screen brightness and luminous color, and compare them with the data corresponding to the standard picture to determine whether the product is qualified; Step 4: Extract the features of the test area based on optical character recognition technology, identify and judge the characters, numbers and English letters contained in the test area, and compare them with the data corresponding to the standard picture to determine whether the product is qualified.

2. According to claim 1, a method for automatically testing the display screen of avionics display products based on machine vision is characterized in that In step 2, the YOLOv3 neural network model of the DNN module in OpenCV is first used to identify the product to be tested and its location contained in the photo, then the product is cropped according to its size, and finally the YOLOv3 neural network model is used to identify the cropped image to find the test area of ​​interest.

3. The automatic testing method for display screens of avionics display products based on machine vision according to claim 1 is characterized in that In step 3, OpenCV is used to analyze the grayscale distribution histogram of the test area and the standard image, and the pixel distribution similarity between the test area and the standard product display image is calculated. When the similarity between the two is large enough, the pixel distribution of the two images is considered to be similar, otherwise, they are not similar, and the product to be tested is judged as unqualified; For the brightness and color change test, the image description model is transformed through the color space conversion function in OpenCV, and the image represented by the RGB color space is converted into an image described by the HSV model. The HSV index of the measured product display image is compared to see whether it falls within the standard range.

4. The automatic testing method for display screens of avionics display products based on machine vision according to claim 1 is characterized in that Step four is implemented using the PaddleOcr open source library. The neural network model used to recognize characters in PaddleOcr is downloaded in advance in a network environment and then imported into the test equipment. The test area is then used as the input of the neural network model to automatically identify and mark the character information contained in the image, and then compared with the preset display characters to determine whether the characters displayed by the product under test are correct.

Images