Artificial intelligence avionics equipment quality inspection device and method

Abstract

The invention belongs to the technical field of equipment quality inspection, and provides an artificial intelligence avionics equipment quality inspection device and method, and the device comprises a plurality of cameras, an AI quality inspection server, a model training platform, and a handheld terminal. The camera is positioned and installed according to the delimited areas of the test equipment and the tested equipment, and respectively acquires video images of the test equipment, the tested equipment and an operator in a test environment; the AI quality inspection server comprises a video image recognition system, a process management system, a quality inspection table generation system, a man-machine interaction system and a query callback system, and AI detection of a quality inspection process procedure is realized; the model training platform comprises a computing power center, and the computing power center comprises a graphics processor and a neural network processor and is used for quality inspection model training and algorithm deployment migration; the handheld terminal comprises a quality inspection application program of each type of products and is used for automatically or manually intervening the quality inspection process of the products. The problems that a large amount of manpower is consumed in the traditional quality inspection process, and the inspection process cannot be restored and backtracked are solved.

Description

Technical Field

[0001] This invention belongs to the field of equipment quality inspection technology, and in particular relates to an artificial intelligence avionics equipment quality inspection device and method. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of avionics equipment maintenance, the quality inspection stage is crucial after the repair of traditional avionics equipment. It not only determines whether the equipment can be put into normal use, but also affects the safety and stability of subsequent flights. The quality inspection system mainly consists of testing equipment, the device under test (DUT), and connecting cables. The testing equipment undertakes key tasks such as providing power to the avionics equipment, inputting excitation signals, and receiving and judging the output response. The output response includes indicator lights, digital displays, and other display devices on the avionics equipment panel, as well as the data displayed on the PC testing software. A properly functioning DUT, upon receiving the excitation input, outputs a predictable value; once a function malfunctions, the output will deviate from the normal range.

[0004] In traditional quality inspection processes, operators must sequentially perform tasks such as setting inputs, checking output for normality, and recording results according to production inspection process requirements. Simultaneously, quality inspectors must supervise to prevent operational errors or missed inspections. However, this quality inspection model has several drawbacks: The high manpower required increases costs, and the requirement for at least two people to be present makes the appointment process cumbersome and inefficient. Furthermore, the entire process relies solely on paper records, making it impossible to reconstruct and trace the inspection process if problems arise, which greatly hinders subsequent troubleshooting. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention provides an artificial intelligence avionics equipment quality inspection device and method, which breaks down the quality inspection process into steps, identifies the operator's operation and the input and output status of the test equipment and the tested parts, confirms whether the input and output relationship is correct, and continues until the entire quality inspection process is completed. At the same time, the entire quality inspection process is recorded by video, which solves the problems of traditional quality inspection processes that consume a lot of manpower and cannot be restored and traced back.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of the present invention provides an artificial intelligence avionics equipment quality inspection device, the device including multiple cameras, an AI quality inspection server, a model training platform and a handheld terminal; The cameras are positioned and installed according to the designated areas of the testing equipment and the device under test, and are used to collect video images of the testing equipment, the device under test, and the operator in the testing environment. The AI ​​quality inspection server includes a video image recognition system, a process management system, a quality inspection form generation system, a human-computer interaction system, and a query callback system to realize AI detection of quality inspection process steps. The process management system interacts with the video image recognition system, the quality inspection form generation system, the human-computer interaction system, and the query callback system. The model training platform includes a computing center, which includes a graphics processor and a neural network processor, for quality inspection model training and algorithm deployment and migration. The handheld terminal includes a quality inspection application for each product type, used for automatic or manual intervention in the product quality inspection process.

[0007] Furthermore, the video image recognition system deploys an image processing and recognition algorithm related to the device under test, which is configured to: identify the corresponding video image results based on product attributes and quality inspection procedures; The identification process is as follows: It receives real-time video image data from the camera, including the input signals of the test equipment, the output status of the device under test, and images of the operator's actions; it uses the deployed image processing and recognition algorithm to accurately identify the video images and output the video image results. Among them, video image results refer to the relationship between the input signals of the test equipment, the output status of the tested equipment, and the operator's operation actions in the image, which are identified and determined according to each step of the quality inspection process, and serve as the basis for judging whether the process is qualified.

[0008] Furthermore, the process management system includes a data layer, a control layer, and an execution and interaction layer, and is configured to: break down the quality inspection process for each type of product into procedures, and realize AI detection of the quality inspection process procedures.

[0009] Furthermore, the data layer is configured to include an information import interface and a constructed product quality inspection basic database; the information import interface is used to import quality inspection processes in a specified format, product quality inspection form templates, basic information of the tested product, and information related to the testing environment; the product quality inspection basic database includes a product basic information database, a product quality inspection basic process database, an operator information database, a quality inspector information database, and a testing equipment information database. The control layer includes a process controller, which is configured to: provide product quality inspection process management services, obtain the current product's quality inspection process from the data layer, and convert it into process instructions; The execution and interaction layer is configured to automatically call the camera and image processing and recognition algorithm to complete quality inspection according to the process instructions of the control layer, and issue instructions to the operator or push the process progress through the handheld terminal.

[0010] Furthermore, the quality inspection form generation system is configured to: generate quality inspection forms based on information integrated in the process management system during the quality inspection process, fill in data, and associate evidence; the specific process is as follows: Information collected before quality inspection includes basic information of the operator, basic information of the inspector, time, product model, part number and dispatch number; During the quality inspection process, key data is extracted, including using a video image recognition system to identify the operator's actions, testing equipment, and input / output images of the tested parts at each step of the quality inspection process, and extracting the corresponding data from the image recognition results to fill in the quality inspection form. After the quality inspection is completed, a functional performance test report is generated, forming a quality inspection record.

[0011] Furthermore, the human-computer interaction system is configured to: provide human-computer interaction services; the operator operates the human-computer interaction interface of the handheld terminal to start or pause the product quality inspection process, and select automatic or manual intervention methods.

[0012] Furthermore, the query callback system is configured to: store and record the quality inspection process, including storing and recording information collection before the start of quality inspection, the operation and results of each step, key image data, and global video; and provide global, test equipment, tested equipment, and operator-related images for operators to review and collect evidence.

[0013] A second aspect of the present invention provides a quality inspection method for artificial intelligence avionics equipment, comprising: Import product data in a specified format into the background, build a basic database for product quality inspection, configure camera parameters, and deploy image recognition algorithms for avionics equipment. Log in to the application on the handheld terminal, select the product model to be tested, and enter the product information and inspection personnel information. Then, call up the product process flow and display all process nodes. After the start command is triggered, the process management system advances the process sequentially and prompts the current operation requirements through the application; the process management system issues instructions based on the process confirmation, and the camera captures an image of the target area; the video image recognition system calls the corresponding algorithm to recognize the image and outputs the result, and after automatic or manual confirmation by the system, it is transferred to the next process; After all procedures are completed, authorized personnel can use handheld terminals or the backend to query historical records, play back videos, and trace the quality inspection process. A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of an artificial intelligence avionics equipment quality inspection method as described in the second aspect of the present invention.

[0014] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in an artificial intelligence avionics equipment quality inspection method as described in a second aspect of the present invention.

[0015] The above one or more technical solutions have the following beneficial effects: This invention utilizes multiple cameras strategically positioned to capture images and videos of the entire inspection process, including the overall area, operator actions, testing equipment, and the component under test. This process is broken down into steps, identifying operator actions and the input / output states of the testing equipment and component under test, confirming the correctness of the input / output relationships, and continuing until the entire inspection process is complete. Simultaneously, the entire inspection process is recorded, facilitating spot checks and traceability by quality inspectors. This achieves the goals of saving labor, automatic recording, and convenient backtracking.

[0016] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is a structural diagram of the device according to the first embodiment; Figure 2 This is a schematic diagram of the camera distribution in the first embodiment; Figure 3 This is a schematic diagram of the image processing and recognition algorithm in the first embodiment; Figure 4 This is a flowchart of the screen display results and digital instrument recognition process in the first embodiment; Figure 5 This is a flowchart of the rotary switch recognition process for the first embodiment; Figure 6 This is a flowchart of the toggle switch recognition process for the first embodiment; Figure 7 This is a flowchart of the seven-segment display recognition process in the first embodiment; Figure 8 This is a schematic diagram of the image after edge extraction in the seven-segment display recognition of the first embodiment. Figure 9 This is a schematic diagram of the threading method in the seven-segment display recognition of the first embodiment; Figure 10 This is a flowchart of the pointer-type instrument recognition process for the first embodiment; Figure 11This is a flowchart of the embossed text processing for the first embodiment. Detailed Implementation

[0019] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0021] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0022] Example 1 like Figure 1 As shown in the figure, this embodiment discloses an artificial intelligence avionics equipment quality inspection device, which includes multiple cameras, an AI quality inspection server, a model training platform, and a handheld terminal.

[0023] A) Camera: like Figure 2 As shown, multiple cameras are positioned and installed according to the designated areas of the testing equipment and the device under test, and are used to collect video images of the testing equipment, the device under test, and the operator in the testing environment. Specifically, the cameras are used to collect video images of the entities involved in the quality inspection process (i.e., the testing equipment, the device under test, and the operator) from different angles, in different states, and under different environments, and the image data is transmitted to the AI ​​quality inspection server and model training platform.

[0024] In this embodiment, three cameras are used to capture video images of the test equipment, the device under test (DUT), and the operator, respectively. An additional camera captures the entire area. Following product quality inspection procedures, designated areas for the test equipment and DUT are marked within the testing environment. The test equipment and DUT are then positioned precisely to ensure the accuracy of the images captured by the cameras.

[0025] The tested equipment includes screen images, digital tubes, nameplates, panel switches, indicator light status, pointer instruments, QR codes, etc.

[0026] B) AI Quality Inspection Server: The AI ​​quality inspection server is used to collect and identify the corresponding video image results in the image data according to the quality inspection process, fill the data into the corresponding part of the quality inspection form, and realize AI detection of the quality inspection process.

[0027] Specifically, the AI ​​quality inspection server includes a video image recognition system, a process management system, a quality inspection form generation system, a human-computer interaction system, and a query callback system.

[0028] a) Video image recognition system like Figure 3 As shown, the video image recognition system integrates image processing and recognition algorithms related to the device under test, including algorithms for screen images, digital tubes, nameplate recognition, panel switches, indicator light status, pointer instruments, and QR code recognition. These algorithms are used to identify corresponding video image results based on product attributes and quality inspection processes. All of these algorithms have been trained, tested, and verified on the model training platform before being deployed to the AI ​​quality inspection server.

[0029] Video image results refer to the relationship between the input signals of the test equipment, the output status of the tested equipment, and the operator's actions in the image, which are identified and determined according to each step of the quality inspection process, and are used as the basis for judging whether the process is qualified.

[0030] In this embodiment, the specific process for recognizing the video image results is as follows: The first step is for the AI ​​quality inspection server's video image recognition system to receive real-time video image data from the camera, including the input signals of the test equipment, the output status of the tested equipment, and images of the operator's actions.

[0031] The second step involves the AI ​​quality inspection server's video image recognition system using deployed avionics equipment image recognition algorithms to accurately identify video images, such as screen display content, digital instrument readings, nameplate information, panel switch status, indicator light status, digital tube display, and pointer instrument readings.

[0032] The third step involves the AI ​​quality inspection server's video image recognition system combining information integrated into the process management system to identify the corresponding video image results.

[0033] By analyzing multiple relationships, the system serves as the basis for determining whether the current process meets the qualification standards. Through continuous optimization of deep learning algorithms, the AI ​​quality inspection server can more accurately capture subtle changes in images. Even in complex and ever-changing avionics equipment quality inspection scenarios, it can ensure the accuracy and reliability of the judgment, automatically compare against preset qualification standards, and issue timely warnings when potential problems are detected, prompting operators or the system to conduct further inspections or adjustments.

[0034] In this embodiment, (1) Display results and digital instrument recognition: Using OCR (Optical Character Recognition) technology, the EasyOCR deep learning algorithm model is used to recognize text and data displayed on the screen and in digital instruments, and automatically enters them into the backend database according to data type and data association rules. The specific process is as follows: like Figure 4 As shown, the camera takes a picture of the screen or digital instrument to obtain the original image, and performs preprocessing operations such as image grayscale conversion, binarization, and edge extraction on the image; the region to be identified is extracted using ROI (Region of Interest) to reduce interference from text information in other areas of the screen or digital instrument; the EasyOCR model automatically locates and recognizes the text area on the screen or digital instrument, and finally outputs the text information; according to the encoding rules of the corresponding physical parameter data such as voltage, current, and pressure, the output text information is converted into numerical values ​​and automatically uploaded to the backend server to complete the automatic recognition and recording of the test data.

[0035] (2) Panel switch identification: Panel switches mainly include rotary switches and toggle switches.

[0036] like Figure 5 As shown, a traditional image processing algorithm is used for rotary switches: a camera takes a picture of the switch panel to obtain the original image, and the image is processed by grayscale and binarization to reduce color interference; the contour of the binarized image is extracted, and the contour is fitted into a circle using the Hough transform algorithm; the circle boundary is found using the center point of the circular contour as the seed point and the region growing algorithm is used; the switch handle area image is obtained by performing an AND operation between the circular mask and the original binarized image; the linear equation of the handle is fitted using the Hough transform, and its rotation angle is calculated; based on the switch state range and rotation angle, the switch state is determined.

[0037] like Figure 6 As shown, for toggle switches, the images are reflective because the switches are made of reflective metal. In addition, the robustness of traditional visual algorithms is challenged due to the tilt of the camera. Therefore, a deep learning approach is adopted: data sets of switches from various angles and in various states are collected; the Yolov5 object detection model is used to locate the switches and classify their states to achieve the recognition of the toggle switch states.

[0038] (3) Indicator light status recognition: Indicator lights share similar characteristics in terms of color, shape, and status. They all have circular cross-sections of similar area, a limited range of colors, and two states: on and off. Therefore, the status recognition algorithm for indicator lights adopts traditional visual image processing algorithms. The image is converted to grayscale to eliminate color interference, focusing only on brightness variations. The Canny operator is used for edge detection, extracting prominent line features; the shape of indicator lights can be detected using these features. The line points in the image are then used to calculate the equations for circles or the dimensions and center points of rectangles. For circular status indicator lights on instruments, the Hough transform can be used for circle detection; for square indicator lights, the minimum bounding rectangle algorithm can be used for square detection. Based on the detected indicator light shape, the image within the circular or square region is cropped, and then the image is converted from the RGB color space to the HSV color space using the following formula:

[0039] Where R, G, and B are the pixel values ​​of the red, green, and blue channels of the original image, ranging from 0 to 255; R', G', and B' are the normalized RGB values, ranging from 0 to 1; the H component represents the color type of the pixel, with values ​​corresponding to the angle; the S component represents the vividness of the pixel color, ranging from 0 to 1; and the V component represents the color brightness of the pixel, ranging from 0 to 1.

[0040] The brightness value of the indicator light is obtained by averaging the V component values ​​of all pixels within the circular or square area. A threshold δ is set; if the brightness value is greater than δ, the indicator light is considered to be on, otherwise it is off.

[0041] (4) Seven-segment display recognition: like Figure 7 As shown, the seven-segment display recognition algorithm mainly consists of four parts: image preprocessing, region of interest localization, image segmentation, and character recognition.

[0042] After image preprocessing operations such as grayscale transformation, binarization, and denoising, a high-quality binarized image is obtained. The boundary of each character is located using an image projection algorithm, specifically by identifying the abrupt change in projection value from zero to non-zero in the vertical direction. Based on these abrupt changes in projection value in the horizontal direction, the segmented characters are normalized to ensure consistent character size. Finally, the Canny operator is used to extract character edges, resulting in an image like... Figure 8 The image shown is the one after edge extraction; like Figure 9 As shown, the threading method is used to recognize characters: line ① divides the image of a single character after edge extraction into two equal parts, left and right; lines ② and ③ divide the image after edge extraction into three equal parts, upper, middle, and lower. Character recognition is based on the number and characteristics of the intersections between the ten different digits 0-9 and lines ①, ②, and ③, as shown in the table below. Table 1. Criteria for Intersection Judgment

[0043] The numbers 2, 3, and 5 can be further distinguished based on the different positions of their intersections with the horizontal lines ② and ③: the intersection point of line ② for 2 is larger than that of line ③; the intersection point of line ② for 5 is smaller than that of line ③; and the intersection points of line ② and line ③ for 3 are not significantly different.

[0044] (5) Seven-segment display recognition: like Figure 10 As shown, pointer-type instruments mainly include single-pointer instruments such as voltmeters, ammeters, and pressure gauges. Their identification process mainly includes instrument position positioning, image correction, pointer position extraction, angle calculation, and actual physical quantity calculation. The identification process is as follows: (5.1) Image acquisition: Collect field instrument data and construct a rich instrument dataset; (5.2) Dashboard localization based on deep learning: The YOLOv5 model is used as the target detection model for instrument detection and localization. YOLOv5 mainly consists of three parts: backbone network, neck network, and detection head. It is a single-stage target detector with fast speed and rich data augmentation methods, including mosaic, brightness change, color change, noise, cropping, and copying, which can ensure the accuracy and generalization ability of the model.

[0045] (5.3) Dashboard Image Correction: Before inputting the image into the network model, a template matching + affine transformation (rotation) correction technique is used to correct the dashboard image to improve the model's dashboard recognition accuracy. Feature contours are extracted from the template image and the image to be corrected. The template image is slid across the image to be corrected to search for contours similar to the template contours to obtain matching results. The rotation angle is calculated, and the transformation matrix is ​​calculated based on the angle. The affine transformation algorithm is used to rotate the dashboard image to be corrected based on the transformation matrix to achieve the correction purpose, which can effectively handle dashboard images with different tilt angles. Through the precise cropping and image enhancement processing of YOLOv5, the correction difficulty for complex background images and lighting conditions is greatly reduced, so there is no need to manually select the reference area and the correction can be completed in a short time.

[0046] (5.4) Extracting pointer position: The image is grayscaled and binarized to reduce interference from colors, etc. The outer contour of the instrument is extracted by the contour extraction algorithm, and its largest inscribed rectangle is calculated to determine the center position of the pointer. Specifically: a central circle is created with the center of the rectangle as the center and the diameter is half of the short side of the rectangle. The largest connected region is found as the pointer region by the maximum connected component algorithm on the binarized image inside the central circle. The connected component of the pointer is extended to the entire instrument panel by the region growing algorithm, and pixels with the same or similar pixels are merged, thereby extracting the pointer position.

[0047] (5.5) Angle calculation: Based on the extracted pointer position, the center line position of the pointer is solved by Hough linear transformation; the deflection angle is calculated by angle analysis algorithm; based on the correspondence between the area through which the pointer rotates and the angle of the instrument measurement area and the actual measurement value range of the instrument, the angle is converted into an actual physical quantity.

[0048] (6) Nameplate text recognition: The text on the nameplate is mainly divided into two parts: printed text and embossed text.

[0049] For printed text recognition, the main algorithm process is the same as that for screen display results and digital instrument text recognition. For the specific process, please refer to (1) Screen display results and digital instrument recognition.

[0050] like Figure 11 As shown, embossed text has characteristics such as characters and background colors being similar, backgrounds being easily reflective, character imprints being indistinct, and characters being slanted. Therefore, it is necessary to construct an embossed font dataset; train the embossed font text based on the EasyOCR model and optimize the model; deploy the trained recognition model to the server for subsequent embossed font recognition.

[0051] (7) QR code recognition: For QR codes, it is necessary to detect their corner points. Image smoothing, thresholding, and contour extraction are used to determine the corner point positions. After perspective correction and other transformations, the QR code image is obtained. The QR code information is then extracted using the ZBAR QR code recognition library and saved to the database. The recognition process is as follows: The system scans the input image row by row and column by column to initially detect areas of varying brightness. It then extracts the brightness width stream for each row and column based on gradient transformation: during scanning, the system extracts the "brightness width stream" for each row and column, i.e., the width sequence of the black and white modules, based on pixel gradient changes (i.e., brightness transitions). Next, it checks whether these width streams meet the QR code positioning pattern characteristics: it checks whether these width streams satisfy the unique positioning pattern characteristics of QR codes (i.e., the black and white ratio mode of the finder pattern). Finally, it saves the width stream segments and sets the scanning type to QR code: if the characteristics are met, the system saves these width stream segments, confirms the current recognition target as a QR code, and enters the QR code decoding process. Finally, it locates the center point of the finder pattern: using the standard ratio (1:1:3:1:1) of the QR code positioning pattern, it filters and clusters the width stream segments, and by calculating the intersection points of the horizontal and vertical segments, it initially determines the finder pattern. The process involves several steps: First, identifying the center point of the QR code. This includes determining if the number of center points is greater than or equal to 3, then continuing processing if at least 3 are identified; otherwise, ignoring the current data. Next, adaptive binarization of the image threshold is performed, converting the image into a black and white binary image for subsequent extraction. Then, affine transformation is used to correct the image by sorting the three identified center points by direction and applying an affine transformation to the QR code body, correcting any tilted or distorted shapes into regular squares. Finally, the extraction of module and version information is completed, determining the pixel size of each module in the QR code and extracting version information. Finally, the decoding format information is determined by finding and... Decode the format information in the QR code, including the error correction level and mask mode; Clear the mask: Demask the QR code data according to the decoded mask mode to restore the original encoded data; Read and recover codewords line by line: Read the binarized and affine-transformed regular image line by line to extract the encoded bitstream; Error correction check: Use error correction codes to check and correct the extracted codewords to ensure data correctness; Encoded data conversion and output: Convert the error-corrected bitstream into the corresponding data according to the encoding mode and output the result, ignoring invalid data; If the conditions are not met or decoding fails in any of the above steps, ignore the current data; End of process.

[0052] b) Process Management System The process management system includes a data layer, a control layer, and an execution and interaction layer, which is used to break down the quality inspection process for each type of product into procedures and enable AI detection of the quality inspection process procedures.

[0053] The data layer includes an information import interface designed on the AI ​​quality inspection server and a basic product quality inspection database. The information import interface is used to import quality inspection processes in specified formats, product quality inspection form templates, basic information about the products under test (including product functional performance test items, product repair information, etc.), and information related to the testing environment. The basic product quality inspection database is deployed in the storage hardware of the AI ​​quality inspection server, breaking down the quality inspection process for each product type into procedures, covering the entire process from the start of quality inspection to the formation of quality inspection forms. This includes a basic product information database, a basic product quality inspection process database, an operator information database, a quality inspector information database, and a testing equipment information database. All these databases provide operation interfaces, supporting authorized quality inspection administrators to perform operations such as querying, adding, deleting, and modifying data.

[0054] Information imported through the information import interface is categorized and stored in the constructed product quality inspection basic database according to its type and purpose.

[0055] The control layer includes a process controller, which provides product quality inspection process management services, obtains the current product quality inspection process from the data layer, and transforms it into a series of executable process instructions, and is responsible for the status control of the entire quality inspection process.

[0056] The execution and interaction layer automatically invokes cameras and AI algorithms to complete quality inspection based on the process instructions from the control layer, and issues instructions or pushes process progress updates to operators via handheld terminals. Operators can operate the handheld terminals to view the inspection results of previous processes at any time, understand the operation prompts for the current process, and the system will automatically capture and identify the area required for the current process, providing results for operator confirmation or automatically proceeding to the next process.

[0057] The product quality inspection process management system is designed with interfaces for modifying and adding processes, and there are permission requirements for operators.

[0058] c) Quality Inspection Form Generation System The quality inspection form generation system provides quality inspection form generation services. During the quality inspection process, it intelligently generates quality inspection forms based on product quality inspection form templates integrated into the process management system, allowing users to fill in data and associate supporting evidence. The specific process is as follows: The first step is to collect information before quality inspection. The information collected includes basic information about the operator, basic information about the inspector, time, product model, part number, and dispatch number.

[0059] The second step is to extract key data during the quality inspection process. Following the quality inspection procedures, a video image recognition system is used to identify the operator's actions, testing equipment, and input / output images of the parts under test at each step. The corresponding data from the identification results is then entered into the quality inspection form.

[0060] The third step is to generate a functional performance test report after quality inspection is completed. After all process operations are completed, a comprehensive product functional performance test report is generated based on the generated quality inspection form, forming a quality inspection record.

[0061] In this embodiment, the quality inspection form generation system provides a quality inspection form adjustment interface, which allows authorized quality inspection administrators to adjust the quality inspection form template.

[0062] d) Human-computer interaction system The human-computer interaction system provides human-computer interaction services. The handheld terminal has a human-computer interaction interface, which the operator can use to start or pause the product quality inspection process. The process operation can be set to automatic mode or manual intervention mode.

[0063] In this embodiment, the human-computer interaction system provides an interface for querying quality inspection items.

[0064] e) Query callback system The query callback system is used to store and record the quality inspection process, including information collection before the start of quality inspection, the operation and results of each step, key image data, and global video. Based on the interface provided by the human-computer interaction system for querying quality inspection items by product model, serial number, operator, date, etc., it provides global, testing equipment, tested equipment, and operator-related images. Operators can view detailed playback of the quality inspection process for retrospective evidence collection.

[0065] In this embodiment, information import, the construction of the basic product quality inspection database, the services provided by each system, and the storage of quality inspection process records are all executed in the background, with advanced permissions set and managed by professional personnel.

[0066] C) Model training platform: The model training platform includes a computing center, which comprises a graphics processing unit (GPU) and a neural network processing unit (NPU) for quality control model training and algorithm deployment and migration.

[0067] Specifically, the model training platform receives image data from cameras, constructs large training and test sets, uses the training set to generate models at the computing center and performs remote training, and uses the test set for testing and verification. Once the expected recognition rate is achieved, the model is exported and deployed to the AI ​​quality inspection server.

[0068] In this embodiment, the deep learning framework used for model training is based on TensorFlow and PyTorch, which are used to construct various types of convolutional neural networks and recurrent neural networks, respectively.

[0069] D) Handheld terminal: The handheld terminal is interconnected with the human-computer interaction system in the AI ​​quality inspection server, providing human-computer interaction interfaces and services.

[0070] Specifically, the handheld terminal can be a tablet computer used by the operator, including a quality inspection application for each product model, presented to the user as an APP. The underlying operating system is Android, and the backend is connected to the corresponding product's process flow for automatic or manual intervention in the product's quality inspection process. Before quality inspection, the operator, inspector, product model, part number, and dispatch number can be entered through the handheld terminal. During quality inspection, the terminal can provide product quality inspection progress, operation procedure confirmation, and operations such as process forward, backward, and cancel according to the product's process flow. After quality inspection, the inspector or manager can query product quality inspection records, replay videos, and review the quality inspection process.

[0071] Example 2 This embodiment discloses a quality inspection method for artificial intelligence-based avionics equipment, mainly including: Import product data in a specified format into the background, build a basic database for product quality inspection, configure camera parameters, and deploy image recognition algorithms for avionics equipment. Log in to the application on the handheld terminal, select the product model to be tested, and enter the product information and inspection personnel information. Then, call up the product process flow and display all process nodes. After the start command is triggered, the process management system advances the process sequentially and prompts the current operation requirements through the application; the process management system issues instructions based on the process confirmation, and the camera captures an image of the target area; the video image recognition system calls the corresponding algorithm to recognize the image and outputs the result, and after automatic or manual confirmation by the system, it is transferred to the next process; After all procedures are completed, authorized personnel can use handheld terminals or the backend to query historical records, play back videos, and trace the quality inspection process.

[0072] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0073] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an artificial intelligence avionics equipment quality inspection method as described in Embodiment 2 of this disclosure.

[0074] Example 4 The purpose of this embodiment is to provide an electronic device.

[0075] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in an artificial intelligence avionics equipment quality inspection method as described in Embodiment 2 of this disclosure.

[0076] The steps involved in Embodiment 2 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0077] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0078] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based avionics equipment quality inspection device, characterized in that, The device includes multiple cameras, an AI quality inspection server, a model training platform, and a handheld terminal; The cameras are positioned and installed according to the designated areas of the testing equipment and the device under test, and are used to collect video images of the testing equipment, the device under test, and the operator in the testing environment. The AI ​​quality inspection server includes a video image recognition system, a process management system, a quality inspection form generation system, a human-computer interaction system, and a query callback system to realize AI detection of quality inspection process steps. The process management system interacts with the video image recognition system, the quality inspection form generation system, the human-computer interaction system, and the query callback system. The model training platform includes a computing center, which includes a graphics processor and a neural network processor, for quality inspection model training and algorithm deployment and migration. The handheld terminal includes a quality inspection application for each product type, used for automatic or manual intervention in the product quality inspection process.

2. The artificial intelligence avionics equipment quality inspection device as described in claim 1, characterized in that, The video image recognition system deploys image processing and recognition algorithms related to the device under test, which are configured to: identify the corresponding video image results based on product attributes and quality inspection processes; The identification process is as follows: It receives real-time video image data from the camera, including the input signals of the test equipment, the output status of the device under test, and images of the operator's actions; it uses the deployed image processing and recognition algorithm to accurately identify the video images and output the video image results. Among them, video image results refer to the relationship between the input signals of the test equipment, the output status of the tested equipment, and the operator's operation actions in the image, which are identified and determined according to each step of the quality inspection process, and serve as the basis for judging whether the process is qualified.

3. The artificial intelligence-based avionics equipment quality inspection device as described in claim 1, characterized in that, The process management system includes a data layer, a control layer, and an execution and interaction layer, and is configured to: break down the quality inspection process for each type of product into procedures and realize AI detection of the quality inspection process procedures.

4. The artificial intelligence avionics equipment quality inspection device as described in claim 3, characterized in that, The data layer is configured to include an information import interface and a constructed product quality inspection basic database. The information import interface is used to import quality inspection processes in a specified format, product quality inspection form templates, basic information of the tested product, and information related to the testing environment. The product quality inspection basic database includes a product basic information database, a product quality inspection basic process database, an operator information database, a quality inspector information database, and a testing equipment information database. The control layer includes a process controller, which is configured to: provide product quality inspection process management services, obtain the current product's quality inspection process from the data layer, and convert it into process instructions; The execution and interaction layer is configured to automatically call the camera and image processing and recognition algorithm to complete quality inspection according to the process instructions of the control layer, and issue instructions to the operator or push the process progress through the handheld terminal.

5. The artificial intelligence-based avionics equipment quality inspection device as described in claim 1, characterized in that, The quality inspection form generation system is configured to: generate quality inspection forms based on information integrated in the process management system during the quality inspection process, fill in data, and associate evidence; the specific process is as follows: Information collected before quality inspection includes basic information of the operator, basic information of the inspector, time, product model, part number and dispatch number; During the quality inspection process, key data is extracted, including using a video image recognition system to identify the operator's actions, testing equipment, and input / output images of the tested parts at each step of the quality inspection process, and extracting the corresponding data from the image recognition results to fill in the quality inspection form. After the quality inspection is completed, a functional performance test report is generated, forming a quality inspection record.

6. The artificial intelligence avionics equipment quality inspection device as described in claim 1, characterized in that, The human-computer interaction system is configured to: provide human-computer interaction services; the operator operates the human-computer interaction interface of the handheld terminal to start or pause the product quality inspection process, and select automatic or manual intervention methods.

7. The artificial intelligence-based avionics equipment quality inspection device as described in claim 1, characterized in that, The query callback system is configured to: store and record the quality inspection process, including storing and recording information collection before the start of quality inspection, the operation and results of each step, key image data, and global video; and provide global, test equipment, tested equipment, and operator-related images for operators to review and collect evidence.

8. A quality inspection method for artificial intelligence-based avionics equipment, characterized in that, include: Import product data in a specified format into the background, build a basic database for product quality inspection, configure camera parameters, and deploy image recognition algorithms for avionics equipment. Log in to the application on the handheld terminal, select the product model to be tested, and enter the product information and inspection personnel information. Then, call up the product process flow and display all process nodes. After the start command is triggered, the process management system proceeds with the processes sequentially and prompts the current operation requirements through the application. The process management system issues instructions based on the process confirmation, and the camera captures images of the target area; the video image recognition system calls the corresponding algorithm to recognize the image and outputs the results, and after automatic or manual confirmation by the system, it is transferred to the next process. After all procedures are completed, authorized personnel can use handheld terminals or the backend to query historical records, play back videos, and trace the quality inspection process.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the artificial intelligence avionics equipment quality inspection method as described in claim 8.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the artificial intelligence avionics equipment quality inspection method as described in claim 8.

Images