Aircraft skin rivet hole periphery defect magnetic detection system and method based on machine vision

Abstract

The invention provides an aircraft skin rivet hole periphery defect magnetic detection system and method based on machine vision, and the method comprises the following steps: photographing each region of an aircraft skin rivet, and obtaining an original to-be-spliced image; performing feature information extraction on the rivet image to obtain feature information of the rivet image; performing matching, image unified coordinate transformation and image fusion on the rivet image feature information to obtain an enhanced image; splicing the images to obtain 3D point cloud images of each region of the aircraft skin rivet; based on the 3D point cloud images, through multi-scale feature processing, rivet target detection is carried out on the spliced 3D point cloud images of the areas of the aircraft skin rivets, initial positioning is carried out on each rivet, and a photographing path of each rivet is generated; and photographing each rivet according to the generated rivet photographing path to obtain a 3D point cloud image of the local area of the rivet. According to the invention, the detection precision and the detection efficiency of rivet defect detection can be effectively improved.

Description

Technical Field

[0001] The invention belongs to the technical field of defect detection, and in particular relates to a magnetic detection system for defects around aircraft skin rivet holes based on machine vision. Background Art

[0002] Riveting is widely used in all walks of life, especially in the aviation field. Riveting is the main connection form on aircraft skins, and the quality of riveting directly affects the aerodynamic performance and fatigue life of the skins and even the entire aircraft. Under normal circumstances, the rivets should be flush with the surface of the aircraft skins, but during the riveting process, problems such as rivet head protrusion and countersunk hole abnormalities may occur. If the rivet head protrudes, it will inevitably affect the skin lap structure and reduce the performance of the aircraft. For a single rivet, if the diameter of the deformed end formed after riveting is large, the thickness of the deformed end will inevitably decrease. At this time, the mechanical properties of the rivet are reduced and it is easy to be damaged; if the diameter of the deformed end after riveting is small, the connection is unstable at this time, which easily leads to the rivet falling off. In addition, the connection between the aircraft skin parts is made by riveting multiple rivets. If the distance between two rivets is large, the load of the parts cannot be evenly distributed, and the load shared by some rivets is greater than that of other rivets. Some rivets are easily damaged during service.

[0003] The inspection of rivets mainly involves detecting small cracks, delamination and corrosion on their edges. Early researchers tried to use eddy current testing to detect such defects. With the development of various non-destructive testing technologies, magneto-optical, ultrasonic, visual and other technologies have been gradually applied to the detection of rivet hole positions. The inspection during the production and assembly of rivets and skins is relatively easy, but the inspection of rivets in the air intake area after the skin is assembled or during the service of the aircraft, as well as the improvement of the accuracy and efficiency of rivet defect detection, is the biggest difficulty in aircraft skin defect detection.

[0004] There are a large number of riveted structures in the aircraft skin, and there is a great need for riveting quality inspection. Traditional rivet defect inspection is mostly done manually, relying on human visual observation, which will lead to reduced efficiency due to human eye fatigue and limited individual cognition. This inspection method is slow and the results may have human errors. Summary of the invention

[0005] The present invention provides a machine vision-based magnetic detection system for defects around aircraft skin rivet holes, which can effectively solve the above-mentioned problems.

[0006] The present invention is achieved through the following technical solutions:

[0007] On the one hand, the present invention provides a method for magnetic detection of defects around aircraft skin rivet holes based on machine vision, comprising the following steps:

[0008] S10, taking photos of each area of ​​the aircraft skin rivets to obtain the original image to be stitched;

[0009] S20, after obtaining the original image to be spliced, extracting feature information of the rivet image to obtain feature information of the rivet image;

[0010] S30, matching the rivet image feature information, performing image unified coordinate transformation and image fusion to obtain an enhanced image;

[0011] S40, stitching the images to obtain 3D point cloud images of various areas of the aircraft skin rivets;

[0012] S50, based on the 3D point cloud image, through multi-scale feature processing, rivet target detection is performed on the 3D point cloud image of each area of ​​the rivet of the aircraft skin after splicing, each rivet is initially located, and a shooting path for each rivet is generated;

[0013] S60, taking a photo of each rivet according to the generated rivet photographing path to obtain a 3D point cloud image of a local area of ​​the rivet;

[0014] S70, classifying and segmenting the point cloud data in the 3D image within each rivet area, performing rivet area selection and feature extraction, performing rivet edge detection on the 3D image within each rivet local area, and obtaining a rivet edge path;

[0015] S80: Based on the rivet edge path information, a magnetic detector is used to detect around the rivet hole.

[0016] In some embodiments, the images are stitched to obtain 3D point cloud images of various areas of the aircraft skin rivets, including:

[0017] S401, reading enhanced images to be spliced;

[0018] S402, extracting feature points of each enhanced image;

[0019] S403, using the k-nearest neighbor algorithm to match the feature points of each enhanced image, and splicing each enhanced image to obtain a 3D point cloud image of each area of ​​the aircraft skin rivet.

[0020] On the other hand, the present invention provides a machine vision-based magnetic detection system for defects around aircraft skin rivet holes, comprising: a large field of view 3D camera module, which is configured to take pictures of various areas of the aircraft skin rivets to obtain the original images to be spliced; a small field of view 3D camera module, which is configured to take pictures of each rivet according to the generated rivet photo path; and a magnetic detector, which is configured to detect around the rivet hole.

[0021] In some of the embodiments, the magnetic detector includes: an excitation device, which is configured to generate sinusoidal alternating current; a sensor device, which includes two vertically distributed magnetic sensors, and the sensors are located at the exact center of the magnetic yoke, wherein the Z-axis sensor collects magnetic field signals perpendicular to the surface of the specimen, and the X-axis sensor collects magnetic field signals parallel to the surface of the specimen; and a signal processing device, which includes a differential amplifier module, a phase-sensitive detection module, and a digital-to-analog conversion module.

[0022] In some of the embodiments, a machine vision-based magnetic inspection system for defects around aircraft skin rivet holes is integrated into a composite robot.

[0023] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0024] The present invention detects, identifies and locates the rivets on the fuselage of the aircraft through 3D vision, and guides the collaborative robot to carry the detector to detect the surface of the rivet area. The large field of view 3D camera module can capture the overall information and perform preliminary positioning of the rivet, and the small field of view 3D camera module can locate the precise position of the rivet. By combining the large field of view 3D camera module with the small field of view 3D camera module, the success rate of rivet defect detection can be higher. A magnetic detector is used to detect defects around the rivet hole. The magnetic field generated by the excitation coil is converged on the surface of the workpiece to be inspected through the cylindrical orthogonal magnetic yoke of the excitation device. The workpiece to be inspected will generate an induced current under the excitation magnetic field environment. Due to the difference between the conductivity at the defect and the conductivity of the material itself, the induced current will gather on the surface of the workpiece, resulting in abnormal changes in the induced magnetic field. The magnetic induction intensity on the surface of the workpiece is collected by the magnetic sensor of the sensor device to perform qualitative detection and quantitative evaluation of the defects around the hole. A complete set of intelligent electromagnetic detection system for rivet defects is realized to improve the detection accuracy and efficiency of rivet defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic flow chart of a method for magnetically detecting defects around aircraft skin rivet holes based on machine vision provided in some embodiments of the present invention;

[0027] Figure 2 A schematic diagram of the effect of stitching images provided by some embodiments of the present invention;

[0028] Figure 3 A schematic diagram of the effect of obtaining a 3D point cloud image of a local area of ​​a rivet provided in some embodiments of the present invention;

[0029] Figure 4 A schematic diagram of the effect of generating a rivet edge path to obtain a rivet edge path provided by some embodiments of the present invention;

[0030] Figure 5 A schematic diagram of the structure of a magnetic detection system for defects around aircraft skin rivet holes based on machine vision provided by some embodiments of the present invention;

[0031] Figure 6 A schematic diagram of the structure of a magnetic detection system for defects around aircraft skin rivet holes based on machine vision provided by other embodiments of the present invention;

[0032] Figure 7 A schematic diagram of the structure of a large field of view 3D camera module provided by some embodiments of the present invention;

[0033] Figure 8 A schematic diagram of the structure of a small field of view 3D camera module provided by some embodiments of the present invention;

[0034] Fig. 9 A schematic diagram of the structure of a magnetic detector probe provided in some embodiments of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0036] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. used to indicate the orientation or position relationship are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship commonly placed when the product of the invention is used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0037] In addition, if the terms "horizontal" or "vertical" appear in the description of the present invention, it does not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0038] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0039] The terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products, or devices.

[0040] On the one hand, the embodiment of the present invention provides a method for detecting defects around aircraft skin rivet holes by magnetic detection based on machine vision, please refer to Figure 1 , mainly including the following steps:

[0041] S10, taking photos of each area of ​​the aircraft skin rivet to obtain the original image to be spliced. In some examples, a large-view 3D camera module can be used to take photos of each area of ​​the aircraft skin rivet to obtain two original images to be spliced.

[0042] S20: After obtaining the original image to be spliced, extract feature information of the rivet image to obtain feature information of the rivet image.

[0043] S30, matching the rivet image feature information, performing image unified coordinate transformation and image fusion to obtain an enhanced image.

[0044] S40, stitching the images to obtain 3D point cloud images of each area of ​​the aircraft skin rivet. In some examples, the left and right enhanced images to be stitched can be firstly taken, feature points of the left and right enhanced images can be extracted, feature points of the left and right enhanced images can be matched using a k-nearest neighbor algorithm, and the left and right enhanced images can be stitched to obtain 3D point clouds of each area of ​​the entire rivet, such as Figure 2 shown.

[0045] S50, based on the 3D point cloud image, through multi-scale feature processing, rivet target detection is performed on the 3D point cloud image of each area of ​​the rivet of the spliced ​​aircraft skin, each rivet is initially located, and a shooting path for each rivet is generated.

[0046] S60, taking a photo of each rivet according to the generated rivet photo path, and obtaining a 3D point cloud image of the local area of ​​the rivet, such as Figure 3shown.

[0047] S70, classify and segment the point cloud data in the 3D image within each rivet area, select the rivet area and extract features, detect the rivets directly from the original 3D point cloud, perform rivet edge detection on the 3D image within each rivet local area, accurately locate the rivet edge path, and generate the rivet edge path based on the path planning intelligent algorithm, such as Figure 4 shown.

[0048] S80: Based on the rivet edge path information, a magnetic detector is used to detect around the rivet hole.

[0049] On the other hand, an embodiment of the present invention provides a magnetic detection system for aircraft skin rivet hole defects based on machine vision, see Figure 5 and Figure 6 , mainly including large field of view 3D camera module, small field of view 3D camera module and magnetic detector.

[0050] Among them, the wide field of view 3D camera module adopts line scanning laser structured light imaging, binocular structure, and optimal baseline ratio design to meet the requirements of high-precision data collection. The wide field of view 3D camera module is suitable for long-distance and large-field workpiece identification and positioning, such as Figure 7 As shown. The small field of view 3D camera module adopts line scanning laser structured light imaging, and its size is compact, which can be installed on a small load robot arm. The binocular structure and optimal baseline ratio design meet the requirements of high-precision data acquisition. The small field of view 3D camera module is suitable for workpiece identification and positioning at medium and short distances and small and medium fields of view, such as Figure 8 As shown. A magnetic detector is used to detect defects around rivet holes. The magnetic field generated by the excitation coil is converged on the surface of the workpiece to be inspected through the cylindrical orthogonal magnetic yoke of the excitation device. The workpiece to be inspected will generate an induced current under the excitation magnetic field environment. Due to the difference between the conductivity at the defect and the conductivity of the material itself, the induced current will gather on the surface of the workpiece, resulting in abnormal changes in the induced magnetic field. The magnetic induction intensity on the surface of the workpiece is collected by the magnetic sensor of the sensor device to perform qualitative detection and quantitative evaluation of the defects around the hole. The excitation device and the sensor device (i.e., the probe) are shown as follows. Fig. 9 shown.

[0051] In some of the embodiments, a machine vision-based magnetic detection system for aircraft skin rivet hole defects is integrated into a composite robot. In this embodiment, the composite robot is an advanced robot system that integrates multiple robot technologies, functions, and applications. Equipped with a large field of view 3D camera module, a small field of view 3D camera module, and a magnetic detector, the efficiency of target recognition and target detection will be greatly improved.

[0052] In some of the embodiments, the magnetic detector used mainly includes an excitation device, a sensor device and a signal processing device. The excitation device generates sinusoidal alternating current with adjustable amplitude and frequency by a signal generator, amplifies the signal through a power amplifier and transmits it to the induction coil, and converges the induced magnetic field generated by the coil on the surface of the workpiece through a magnetic yoke; the sensor device is composed of two vertically distributed magnetic sensors, and the sensor is located at the center of the magnetic yoke, wherein the Z-axis sensor collects magnetic field signals perpendicular to the surface of the specimen, and the X-axis sensor collects magnetic field signals parallel to the surface of the specimen, and the signals collected by the sensor device are further transmitted to the signal processing device; the signal processing device performs differential amplification, phase-sensitive detection, and digital-to-analog conversion on the signals collected by the sensor, and finally transmits the processed signals to the host computer. The magnetic detector adopts a cylindrical orthogonal design to apply an excitation magnetic field. After planning the detection path, the magnetic detector detects around the rivet hole.

[0053] The purpose of this invention is to build an intelligent electromagnetic detection system based on robot vision-guided aircraft fuselage rivet detection to complete the rivet defect detection during the service of the aircraft. The entire 3D vision-guided automated detection system consists of multiple components such as 3D sensors, collaborative robotic arms, composite robot AGV chassis, composite robot scheduling platform, intelligent algorithm engine, visual interaction system, magnetic detector, etc. Multi-component system calibration is required to guide the robotic arm to perform accurate detection tasks.

[0054] This embodiment also provides a computer storage medium on which a computer program is stored. The computer program is loaded by a processing module to implement any of the above-mentioned methods for magnetic detection of defects around aircraft skin rivet holes based on machine vision.

[0055] The above description is only a preferred embodiment of the present invention and does not constitute any limitation to the present invention. Any simple modification or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A magnetic detection method for defects around aircraft skin rivet holes based on machine vision, characterized in that: The following steps are involved: S10, taking photos of each area of ​​the aircraft skin rivets to obtain the original image to be stitched; S20, after obtaining the original image to be spliced, extracting feature information of the rivet image to obtain feature information of the rivet image; S30, matching the rivet image feature information, performing image unified coordinate transformation and image fusion to obtain an enhanced image; S40, stitching the images to obtain 3D point cloud images of various areas of the aircraft skin rivets; S50, based on the 3D point cloud image, through multi-scale feature processing, rivet target detection is performed on the 3D point cloud image of each area of ​​the rivet of the aircraft skin after splicing, each rivet is initially located, and a shooting path for each rivet is generated; S60, taking a photo of each rivet according to the generated rivet photographing path to obtain a 3D point cloud image of a local area of ​​the rivet; S70, classifying and segmenting the point cloud data in the 3D image within each rivet area, performing rivet area selection and feature extraction, performing rivet edge detection on the 3D image within each rivet local area, and obtaining a rivet edge path; S80: Based on the rivet edge path information, a magnetic detector is used to detect around the rivet hole.

2. The method for detecting defects around aircraft skin rivet holes by magnetic detection based on machine vision according to claim 1 is characterized in that: The images are stitched together to obtain 3D point cloud images of each area of ​​the aircraft skin rivets, including: S401, reading enhanced images to be spliced; S402, extracting feature points of each enhanced image; S403, using the k-nearest neighbor algorithm to match the feature points of each enhanced image, and splicing each enhanced image to obtain a 3D point cloud image of each area of ​​the aircraft skin rivet.

3. The aircraft skin rivet hole defect magnetic detection system based on machine vision is characterized by: include: A large-viewing-field 3D camera module, wherein the large-viewing-field 3D camera module is configured to take pictures of various areas of the aircraft skin rivets to obtain original images to be stitched; A small field of view 3D camera module, wherein the small field of view 3D camera module is configured to take a photo of each rivet according to the generated rivet photo path; A magnetic detector is configured to detect around the rivet hole.

4. The machine vision-based magnetic detection system for aircraft skin rivet hole defects according to claim 3 is characterized in that: The magnetic detector comprises: an excitation device configured to generate a sinusoidal alternating current; The sensor device comprises two vertically distributed magnetic sensors, the sensors are located in the center of the magnetic yoke, wherein the Z-axis sensor collects magnetic field signals perpendicular to the surface of the specimen, and the X-axis sensor collects magnetic field signals parallel to the surface of the specimen; A signal processing device, comprising a differential amplification module, a phase-sensitive detection module and a digital-to-analog conversion module.

5. The aircraft skin rivet hole defect magnetic detection system based on machine vision according to claim 3 or 4, characterized in that: The aircraft skin rivet hole circumference defect magnetic detection system based on machine vision is integrated into a composite robot.

Images