A device and method for measuring the concave-convex amount of aircraft fasteners

Abstract

The present invention discloses a device and method for measuring the concave-convex quantity of aircraft fasteners. The device includes a device operation module, a data acquisition module, an embedded system, a device power supply module and an auxiliary measurement module; the data acquisition module includes a laser rangefinder, a two-dimensional camera and a binocular structured light camera, the binocular structured light camera is fixed on the device box and under the device upper cover, and collects point cloud data and image data of the aircraft surface; the laser rangefinder and the two-dimensional camera are fixed in the device box, and respectively obtain the distance data between the laser rangefinder and the aircraft surface, and the image of the aircraft surface in the shooting area of ​​the two-dimensional camera; the embedded system receives the operation instruction of the device operation module, sends the acquisition instruction to the data acquisition module, and stores and processes the data obtained by the data acquisition module, and marks the fasteners on the aircraft surface and calculates the concave-convex quantity. The present invention can realize the concave-convex quantity measurement of aircraft fasteners more conveniently, accurately and efficiently.

Description

Technical Field

[0001] The invention belongs to the technical field of aircraft quality inspection, and in particular relates to a device and method for measuring the concave-convex amount of aircraft fasteners. Background Art

[0002] Aircraft fasteners are directly related to the safety, performance and reliability of aircraft. If the concave and convex amount of fasteners exceeds the requirements, it will cause the aircraft surface to be uneven, affecting air flow, thus affecting the aerodynamic performance and stability of the aircraft, and will also cause uneven force between the fasteners and the skin, thus affecting the integrity and durability of the aircraft structure. Therefore, it is particularly important to measure the concave and convex amount of aircraft fasteners, but the number of fasteners on the aircraft surface is huge, reaching tens of thousands, and the difference between them and the aircraft skin surface is often very small, and the concave and convex amount is difficult to clearly identify.

[0003] Traditional manual detection methods, such as manual molds or contact instruments, cannot meet the needs of efficient and accurate detection of fastener convexity. Therefore, it is urgent to carry out research on efficient detection methods and systems for fastener convexity to achieve fast and accurate detection of fastener convexity. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a device and method for measuring the concave-convex amount of aircraft fasteners in view of the deficiencies of the above-mentioned prior art. The device includes a device operation module, a data acquisition module, an embedded system, a device power supply module, and an auxiliary measurement module. The data acquisition module includes a laser rangefinder, a two-dimensional camera, and a binocular structured light camera, which are used to locate the measurement position and collect point cloud data and image data on the aircraft surface. The embedded system is used to guide the operation of the data acquisition module and process the collected data to realize the marking and concave-convex amount calculation of the fasteners on the aircraft surface. The device operation module is used to conveniently operate the device. The device power supply module meets the power supply requirements in different scenarios. The auxiliary measurement module provides convenient and stable measurement operations and the function of protecting the aircraft surface. The measurement method converts the data collected by the binocular structured light camera into images and point cloud data, and performs fastener detection and concave-convex amount calculation to realize the measurement of the concave-convex amount of aircraft fasteners. The present invention can more conveniently, accurately and efficiently measure the concave-convex amount of aircraft fasteners.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] A device for measuring the concave-convex quantity of aircraft fasteners, comprising an upper cover of the device, a lower cover of the device and a device box; and also comprising a device operation module, a data acquisition module, an embedded system, a device power supply module and an auxiliary measurement module;

[0007] The device operation module is used for the user to operate the measuring device;

[0008] The data acquisition module includes a laser rangefinder, a two-dimensional camera and a binocular structured light camera, wherein the binocular structured light camera is fixed on the equipment box and under the upper cover of the device, and is used to collect point cloud data and image data of the aircraft surface; the laser rangefinder and the two-dimensional camera are fixed in the equipment box, and respectively obtain the distance data between the laser rangefinder and the aircraft surface, and the image of the aircraft surface in the shooting area of ​​the two-dimensional camera;

[0009] The embedded system is used to receive operation instructions from the device operation module, send acquisition instructions to the data acquisition module, store and process the data acquired by the data acquisition module, mark the fasteners on the aircraft surface and calculate the concave and convex amount;

[0010] The device power supply module and the auxiliary measurement module are used for device power supply and auxiliary measurement respectively.

[0011] To optimize the above technical solutions, the specific measures taken also include:

[0012] The above-mentioned auxiliary measurement module includes an auxiliary handle, a measuring bracket and a status display light; wherein the auxiliary handle includes a lower handle and an anti-slip handle, the lower handle is connected to the equipment box through a threaded nail, and is used for the operator to support the device, the anti-slip handle is connected to the upper cover of the device through a threaded nail, and the rear end of the anti-slip handle is provided with anti-slip rubber; the measuring bracket includes a carbon fiber bracket and a soft support, the measuring bracket is connected to the threaded hole of the upper cover of the device through a threaded structure, the carbon fiber bracket provides a solid support for the device when measuring the fasteners on the aircraft surface, ensuring the accuracy and stability of the measurement, and the soft support enables the measuring bracket to fit closely with the aircraft surface, reduces vibration and interference, and protects the aircraft surface from damage.

[0013] The above-mentioned device operation module includes a display touch screen, a trigger trigger and a press button; wherein the display touch screen is fixed to the screen heat sink by fasteners, and the screen heat sink is connected to the upper cover of the device by a hinge to realize a 90° pitch rotation of the display touch screen; the trigger trigger is arranged at the front end of the lower handle, so that the operator can pull the trigger trigger to quickly start the measurement; the press buttons are respectively a power off button, a restart button and a screen off button, which are installed on the upper surface of the upper cover of the device, so that the operator can press them to realize the corresponding operation.

[0014] The above-mentioned device power supply module includes an aviation socket and a quick-release battery that can provide independent power supply; the aviation socket is installed inside the device box; a socket is provided at the upper end of the quick-release battery, which is connected to the socket provided at the bottom of the lower handle to realize the power supply of the entire device, and the quick-release battery and the lower handle are connected by a spring lock.

[0015] A method for measuring the concave-convex amount of aircraft fasteners, comprising the following steps:

[0016] S1. The device power supply module supplies power to the device, starts the measuring device, and sets the binocular structured light camera exposure parameters, fastener detection algorithm parameters, and concave-convex amount calculation algorithm parameters through the display touch screen of the device operation module;

[0017] S2, moving the device to the aircraft surface to be measured, the two-dimensional camera feeds back the measurement area in real time, the laser rangefinder feeds back the distance from the device to the aircraft surface in real time, and the measuring bracket of the auxiliary measurement module is closely attached to the aircraft surface and adjusted to the best measurement area;

[0018] S3, the device starts measuring, the binocular structured light phase obtains the aircraft surface data, and then transmits the data to the embedded system;

[0019] S4. After the measurement operation is completed, the embedded system converts the data obtained by the binocular structured light camera into image data and point cloud data, executes the fastener detection algorithm and the concave-convex amount calculation algorithm, identifies the fasteners in the image and marks them, and obtains the concave-convex amount results of the aircraft fasteners;

[0020] S5. The embedded system transmits the convex-concave measurement results of the aircraft fasteners to the device operation module, displays the results, prompts the fasteners whose convex-concave measurement does not meet the requirements, and guides the workers to repair or reinstall them.

[0021] The above-mentioned S4 specific process includes the following steps:

[0022] S41, the embedded system converts the scanned aircraft surface data into corresponding image data and point cloud data according to the SDK of the binocular structured light camera;

[0023] S42, executing a fastener detection algorithm based on the image data, identifying and marking fasteners in the image, wherein the marked pixels are mapped to point cloud data according to a corresponding relationship in the SDK of the binocular structured light camera to obtain projected point cloud data;

[0024] S43, executing a concave-convex amount calculation algorithm based on the projected point cloud data, segmenting the fastener point cloud, denoising and fitting the fastener and surface skin point clouds, and finally calculating the distance to obtain the concave-convex amount result of each fastener.

[0025] The above-mentioned S42 uses the fastener detection algorithm to regress the bounding box drawn by the fastener, using the following parameterized equation:

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] in and Represents the center horizontal and vertical coordinates, width and height of the bounding box; variables represents the value of the additional feature;

[0032] variable and Respectively represent the horizontal and vertical coordinates, width and height of the center of the anchor frame;

[0033] , and They are , and Normalized parameters.

[0034] In the above-mentioned S42, during the fastener detection process, the marked pixel points are mapped to the point cloud data according to the corresponding relationship in the SDK to generate the projected point cloud data. The specific corresponding relationship is as follows:

[0035] Each point in the point cloud is ,in , is the column number where the pixel is located, is the row number where the pixel is located, Is the overall pixel width of the image.

[0036] The above fastener detection algorithm is set with a confidence threshold and a bounding box threshold;

[0037] In target detection, a confidence score is estimated for each detection result, which indicates the probability or confidence that the detection result contains the target. A confidence threshold is used to filter out detection results with confidence scores lower than the threshold.

[0038] In the non-maximum suppression process, if the overlap index between two bounding boxes is If the value is greater than or equal to the bounding box threshold, the bounding boxes with lower confidence are suppressed;

[0039] The overlap index for:

[0040] ;

[0041] in, represents the intersection area of ​​two bounding boxes, represents the union area of ​​two bounding boxes, The characteristics of the intersection, Represents the characteristics of a union; and They are the weighting factors of the intersection area, union area, intersection features, and union features respectively.

[0042] The present invention has the following beneficial effects:

[0043] The present invention adopts a semi-automatic operation mode, utilizes computer vision technology, image processing algorithm and point cloud processing algorithm, and combines high-precision sensors and data analysis technology to achieve accurate measurement of the concave and convex amount of aircraft fasteners, improves the measurement accuracy and efficiency, greatly shortens the measurement time, and improves work efficiency;

[0044] The device of the present invention is compact in design, small in weight and volume, easy to carry and use, and can be flexibly applied to different work scenes, thereby improving the applicability and convenience of the device; the present invention is provided with ergonomic auxiliary means, thereby improving the user's comfort and operation convenience, so that the user feels more comfortable and convenient when using the device, and reduces the fatigue and discomfort of the operation;

[0045] The display touch screen of the device of the present invention provides a visual interface, making the measurement results intuitive, clear and easy to understand, which helps users quickly understand the concave and convex amount of the fastener and make necessary processing and decisions.

[0046] The present invention uses a fastener detection algorithm to identify the regression of the bounding box drawn by the fastener. , and Normalization is performed. During the normalization process, not only the relative position and size relationship of the bounding box and the anchor box are considered, but also more parameters are introduced, such as the distance, width and height square terms between the bounding box and the anchor box, which can more comprehensively describe the characteristics of the target box, making the model more flexible and generalizable, and more robust and reliable in practical applications.

[0047] The overlap index of the fastener detection algorithm of the present invention can achieve a more comprehensive evaluation of overlap, making it more expressive and adaptable, more flexible and accurate, and more adaptable to the needs of different scenarios and goals. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a structural schematic diagram of the device of the present invention;

[0049] Figure 2 It is a multi-angle view of the device of the present invention;

[0050] Figure 3 It is a schematic diagram of measuring aircraft fasteners in one embodiment of the device of the present invention;

[0051] Among them, 1-display touch screen, 2-screen heat sink, 3-device upper cover, 4-embedded system, 5-status display light, 6-anti-slip handle, 7-anti-slip rubber, 8-aviation socket, 9-network cable socket, 10-device lower cover, 11-device box, 12-lower handle, 13-spring lock, 14-quick release battery, 15-trigger, 16-laser rangefinder, 17-two-dimensional camera, 18-binocular structured light camera, 19-press button, 20-carbon fiber bracket, 21-flexible support. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.

[0054] like Figure 1 and Figure 2 As shown, the present invention is a portable aircraft fastener convex-concave measurement device, including a device upper cover 3, a device lower cover 10 and a device box 11; it also includes a device operation module, a data acquisition module, an embedded system 4 and a device power supply module, and is also provided with an auxiliary measurement module.

[0055] The device operation module includes a display touch screen 1, a trigger 15 and a push button 19, which are used for users to conveniently operate the device;

[0056] The data acquisition module is installed in the upper cover 3 of the device, the lower cover 10 of the device and the equipment box 11 by fasteners, and is used to collect point cloud data and image data of the aircraft surface and the distance data between the device and the aircraft surface; specifically, the data acquisition module includes a laser rangefinder 16, a two-dimensional camera 17 and a binocular structured light camera 18. The binocular structured light camera 18 is fixed on the equipment box 11 and under the upper cover 3 of the device by screws, and is used to collect point cloud data and image data of the aircraft surface; the laser rangefinder 16 and the two-dimensional camera 17 are fixed in the equipment box 11 by bonding, and respectively obtain the image of the aircraft surface in the shooting area of ​​the two-dimensional camera 17 and the distance data between the laser rangefinder 16 and the aircraft surface. The above two together constitute a set of visual-distance odometers, which are used to locate the binocular structured light camera 18 at the best measurement position.

[0057] The embedded system 4 is installed on the lower cover 10 of the device, and is used to send collection instructions to the data collection module, store and process the data obtained by the data collection module, mark the fasteners on the aircraft surface and calculate the convexity and concave amount through the obtained aircraft surface data, and receive the operation instructions of the device operation module, such as receiving the click signal of the display touch screen 1 and sending the display data stream.

[0058] The device power supply module and the auxiliary measurement module are used for device power supply and auxiliary measurement respectively.

[0059] The auxiliary measurement module includes an auxiliary handle, a measurement bracket and a status display light 5, which is intended to provide convenient and stable measurement operations and protect the aircraft surface. The auxiliary handle includes a lower handle 12 and an anti-slip handle 6. The lower handle 12 is connected to the device box 11 through a screw nail and is used for the operator to support the device. The anti-slip handle 6 is connected to the upper cover 3 of the device through a screw nail. The rear end of the anti-slip handle 6 uses an anti-slip rubber 7 to ensure the safety and stability of the operator during operation; the measurement bracket is composed of a carbon fiber bracket 20 and a soft support 21, which are fixedly connected by screw nails. The measurement bracket is connected to the threaded hole of the upper cover 3 of the device through a threaded structure, which can be quickly disassembled and assembled. The carbon fiber bracket 20 provides a solid support for the device when measuring the fasteners on the aircraft surface, ensuring the accuracy and stability of the measurement. The soft support 21 realizes the close fit between the measurement bracket and the aircraft surface, effectively reducing vibration and interference, and protecting the aircraft surface from damage, thereby improving the accuracy and reliability of the measurement. The status display light 5 serves as a status indicator of the device and provides instant feedback on the working status. In standby mode, the status indicator light is blue; during measurement, the indicator light is green; and when there is a problem with the device, it will be red. This localized use of the light bar design not only allows users to quickly understand the status of the device, but also provides instant feedback, helping to improve the user interaction experience.

[0060] In the device operation module, the display touch screen 1 is fixed on the screen heat sink 2 by fasteners, and the display touch screen 1 provides an interface for the user to interact with the device. The user can view the measurement results, operate the device, and set parameters through the touch screen. There are multiple rows of heat dissipation holes on the back of the screen heat sink 2 to ensure the normal heat dissipation of the display touch screen 1. The screen heat sink 2 is connected to the upper cover 3 of the device by a hinge, so that the display touch screen 1 can be pitched and rotated 90°. The hinge adopts a damping design. When the operator turns it to a certain position, it can stay at the operator's desired angle based on friction to avoid shaking during the measurement process, which is convenient for the operator to observe and operate; the trigger 15 is set at the front end of the lower handle 12, and the operator can quickly start the measurement by pulling the trigger. The press button 19 is installed on the upper surface of the upper cover 3 of the device. There are three press buttons 19, namely the shutdown button, the restart button and the screen off button. The operator can press to realize the corresponding operation, which can provide more operation options and convenience, especially when different functions or control options need to be used during the use of the device.

[0061] The power supply module of the device includes two power supply modes: aviation socket 8 and quick-release battery 14. Either aviation socket 8 or quick-release battery 14 can realize power supply for the whole device. The operator can directly connect the power source externally through aviation socket 8, so as to realize plug-and-play of the device; aviation socket 8 is installed inside the equipment box 11; the upper end of quick-release battery 14 is provided with a socket, which is connected to the socket provided at the bottom of the lower handle 12 to realize power supply for the whole device, and the quick-release battery 14 and the lower handle 12 are firmly connected by spring lock 13, which not only avoids the problem of power supply interruption caused by poor contact, but also effectively prevents the quick-release battery from accidentally falling off when moving the device, thereby ensuring the stable operation and safe use of the device.

[0062] In summary, in the device of the present invention, the display touch screen 1 is fixed to the screen heat sink 2 by fasteners. The screen heat sink 2 is connected to the upper cover 3 of the device by a hinge. The upper cover 3 of the device is fixed and buckled with the lower cover 10 of the device by screws, protecting the key components inside the device from the influence of the external environment and providing structural stability. The embedded system 4 is fixed above the lower cover 10 of the device by screws, responsible for controlling data acquisition, storage and processing, as well as interaction with the display touch screen. The anti-slip handle 6 is connected to the upper cover 3 of the device by screws. The network cable socket 9 and the aviation socket 8 are installed inside the device box 11 by fasteners, providing the network connection and power input interface of the device. The device box 11 is connected to the lower cover 10 of the device by screws. The lower handle 12 is connected to the device box 11 by screws to provide support for carrying and moving the device. A socket is provided at the upper end of the quick-release battery 14, which is connected to the socket provided at the bottom of the lower handle 12 to realize the power supply of the entire device. The trigger 15 is arranged at the front end of the lower handle 12, and the operator can quickly start the measurement by pulling the trigger. The push button 19 is installed on the upper surface of the upper cover 3 of the device. The carbon fiber bracket 20 and the soft support 21 constitute the measuring bracket of the device, and the measuring bracket is connected to the threaded hole of the upper cover 3 of the device through a nut, so as to realize the rapid disassembly and assembly of the measuring bracket and the device.

[0063] The data transmission inside the embedded system 4 is realized through the USB interface, which sends instructions to the data acquisition module to control it to perform data acquisition. The data acquisition module is responsible for acquiring the point cloud data, image data and distance data between the device and the aircraft surface of the aircraft surface. The embedded system 4 is responsible for storing and processing these data, marking the fasteners and calculating the concave and convex amount. Through this process, the position and concave and convex amount of the fasteners can be accurately determined, providing an accurate basis for subsequent maintenance and repair work. In addition, the embedded system 4 is also responsible for receiving click signals from the display touch screen 1 and sending display data streams accordingly. This allows users to intuitively view and operate the measurement results through the display touch screen 1, improving the convenience of operation and user experience. The external data transmission of the embedded system 4 is realized through the network cable socket 9, and the network cable is connected to the outside of the device to realize the transmission of data to the outside and the required debugging work.

[0064] A method for measuring the concave-convex amount of aircraft fasteners based on the above-mentioned measuring device of the present invention comprises the following steps:

[0065] S1, powering the system, starting the measuring device, setting the exposure parameters of the binocular structured light camera 18, the fastener detection algorithm parameters and the concave-convex amount calculation algorithm parameters;

[0066] First, start the device by connecting it to an external power source. During this process, you can choose to plug the aviation plug into the aviation socket or plug the quick-release battery into the socket on the lower handle to power the device. In this way, the measuring device can start working. On the software interface of the display touch screen, the user can set the number of exposures and exposure intensity parameters of the binocular structured light camera according to the light intensity of the measurement environment and the color of the aircraft fasteners. Ensure that the camera can obtain clear images under different lighting conditions.

[0067] Then, the confidence threshold and bounding box threshold of the fastener detection algorithm are set based on the size and spacing of the aircraft fasteners. This step is to ensure that the detection algorithm can accurately identify the fasteners on the aircraft surface and mark their locations. In object detection, the model estimates a confidence score for each detection result, which represents the probability or confidence that the detection result contains the object. The confidence threshold is used to filter out detection results with confidence below the threshold. The bounding box threshold represents (Intersection over union) threshold. It is a metric used to measure the degree of overlap between two bounding boxes. It is obtained by dividing the intersection area of ​​the two bounding boxes by their union area, expressed as:

[0068] ;

[0069] in, represents the intersection area of ​​two bounding boxes, represents the union area of ​​two bounding boxes, The characteristics of the intersection, Represents the features of the union. The features are the shape, color, and texture of the target object, which are extracted by using a convolutional neural network;

[0070] and are the weighting factors of the intersection area, union area, intersection features, and union features, respectively. and The value of The contribution of value; and They represent the weights of the features of the intersection and union respectively. In addition to considering the area, other features of the target object, such as shape, color, and texture, are also considered. and , these features can be adjusted according to the weights.

[0071] The above parameters and features are introduced to achieve a more comprehensive evaluation of overlap, making it more expressive and adaptable. More flexible and accurate, and more adaptable to the needs of different scenarios and goals.

[0072] In the non-maximum suppression process, if the distance between two bounding boxes is If the value is greater than or equal to the bounding box threshold, one of the bounding boxes will be suppressed, usually the one with lower confidence. This ensures that there will not be too many overlapping bounding boxes in the final detection result, thereby improving the accuracy and reliability of the detection.

[0073] Next, set the segmentation, denoising and fitting parameters of the fastener point cloud in the concave-convex amount calculation algorithm. By setting these parameters, the collected aircraft fastener data can be accurately processed. Then set the extension length, denoising and fitting parameters of the aircraft skin point cloud to obtain the skin point cloud around each fastener. The above parameters in the concave-convex amount algorithm are used to accurately obtain the concave-convex amount of each fastener.

[0074] S2, moving the measuring device to the aircraft surface to be measured, the two-dimensional camera 17 provides real-time feedback of the measuring area, the laser rangefinder 16 provides real-time feedback of the distance between the device and the aircraft surface, and the measuring bracket (20, 21) is closely attached to the aircraft surface and adjusted to the best measuring area;

[0075] The operator moves the measuring device to the aircraft surface to be measured. During this process, the two-dimensional camera is used to provide real-time feedback of the image of the measurement area so that the operator can clearly observe the situation on the aircraft surface. At the same time, the laser rangefinder provides real-time feedback of the distance between the device and the aircraft surface, helping the operator to control the distance between the device and the aircraft surface. During the movement, the operator needs to adjust the measuring bracket to ensure that it fits tightly against the aircraft surface and adjust it to the optimal measurement area. This ensures that there will be no errors caused by poor measurement position during the measurement process and accurate measurement results are obtained.

[0076] In this embodiment, the optimal measurement distance of the binocular structured light camera is set to 500mm. The operator uses the distance information fed back by the laser rangefinder to accurately move the entire device to 500mm from the aircraft surface. Once the set distance is reached, the operator will further adjust the position of the measurement bracket to ensure that it is completely in contact with the aircraft surface. At the same time, the operator will pay attention to rotating the nut at the connection to ensure that the measurement bracket is firmly connected to the upper cover of the device, ensuring the stability and reliability of the device during the measurement process, thereby ensuring the accuracy of the measurement results.

[0077] S3, the measuring device starts measuring, the binocular structured light phase 18 obtains the aircraft surface data, and then transmits the data to the embedded system 4;

[0078] The operator can choose to pull the trigger or click the start measurement button on the display touch screen to start the measurement device to start collecting data. Once started, the binocular structured light camera will begin to acquire data from the aircraft surface and transmit the data to the embedded system through the USB interface for processing. Figure 3 As shown in the figure, for a larger area of ​​aircraft surface, according to the real-time feedback of the measurement area image from the 2D camera, after completing one measurement, the operator needs to move the device to a new area for the next measurement. The purpose of this step is to ensure that the fastener data of the entire aircraft surface is fully collected, thereby ensuring the accuracy and comprehensiveness of the measurement results.

[0079] S4, after the measurement operation is completed, the embedded system 4 converts the data obtained by the binocular structured light camera 18 into image data and point cloud data, performs a fastener detection algorithm on the image data, and performs a concave-convex amount calculation algorithm on the point cloud data to obtain the concave-convex amount result of the aircraft fastener;

[0080] S41, the embedded system (4) converts the scanned aircraft surface data into corresponding image data and point cloud data according to the SDK of the binocular structured light camera (18);

[0081] After the data collection operation is completed, the embedded system begins to process the collected data. First, the system uses the software development kit (SDK) of the binocular structured light camera to convert the scanned aircraft surface data into corresponding image data and point cloud data.

[0082] S42, executing a fastener detection algorithm based on the image data, identifying and marking fasteners in the image, wherein the marked pixels are mapped to point cloud data according to a corresponding relationship in the SDK of the binocular structured light camera (18), thereby obtaining projected point cloud data;

[0083] The image data is then fed into a fastener detection algorithm that identifies the fasteners in the image and labels them. For the regression of the bounding box drawn by the identified fasteners, the following parameterized equation is used:

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] in and Represents the center horizontal and vertical coordinates, width and height of the bounding box; variables The values ​​of additional features related to the shape, background, texture, etc. of the target object;

[0090] variable and Respectively represent the horizontal and vertical coordinates, width and height of the center of the anchor frame;

[0091] , and They are , and Normalized parameters.

[0092] right , and Normalize it and map it to a fixed range of [0, 1]. This allows the regression parameter values ​​to be within a smaller range, making it easier to train and optimize. The normalized parameters are , and The normalization process not only considers the relative position and size relationship between the bounding box and the anchor box, but also introduces more parameters, such as the distance between the bounding box and the anchor box, the square of the width and the height. This can more comprehensively describe the characteristics of the target box, making the model more flexible and generalizable, and more robust and reliable in practical applications.

[0093] An anchor box is an auxiliary tool used in target detection to generate candidate boxes in an image. It is a predefined bounding box with a fixed size and aspect ratio, which is used to represent the position and shape of possible targets in the image.

[0094] During the fastener inspection process, the marked pixels are mapped to the point cloud data according to the corresponding relationship in the SDK, thereby generating the projected point cloud data. The specific corresponding relationship is as follows:

[0095] Each point in the point cloud is ,in , is the column number where the pixel is located, is the row number where the pixel is located, Is the overall pixel width of the image.

[0096] S43, executing a concave-convex amount calculation algorithm based on the projected point cloud data, segmenting the fastener point cloud, denoising and fitting the fastener and surface skin point clouds, and finally calculating the distance to obtain the concave-convex amount result of each fastener.

[0097] The projected point cloud data enters the concave-convex amount calculation algorithm. The algorithm first segments the fastener point cloud to distinguish the fastener from other point cloud data. After the segmentation is completed, the algorithm will denoise the segmented fastener area to eliminate the interference introduced by sensor noise or other factors. The filtering denoising method is adopted. Then, the algorithm will perform shape fitting operation on each fastener area to fit it into a circular surface. The least squares fitting algorithm is used to find the best circular fitting result. Next, for the skin point cloud around each fastener, the algorithm performs denoising and fitting operations to eliminate noise and fit a circular ring surface. The same method is used as the fastener area, but it is performed on the skin point cloud. After obtaining the circular surface fitted by the fastener and the circular ring surface fitted by the skin point cloud, the distance between them in the direction perpendicular to the fastener surface is calculated, so as to accurately obtain the concave-convex amount of each fastener.

[0098] S5. The embedded system 4 transmits the convex-concave measurement result of the aircraft fastener to the display touch screen 1, displays the result, prompts the fasteners whose convex-concave measurement does not meet the requirements, and guides the workers to repair or reinstall them.

[0099] The embedded system transmits the processed concave-convex results of aircraft fasteners to the display touch screen through the USB interface for intuitive display of the results. The display interface graphically displays the concave-convex information of each fastener on the aircraft surface. When the system detects that the concave-convex amount of some fasteners does not meet the requirements, it will warn the operator through red marks and text prompts to guide the worker to repair or reinstall.

[0100] In this embodiment, the present invention solves the problems of low accuracy and low efficiency in the conventional method for measuring the concave and convex amount of aircraft fasteners. A semi-automatic measurement method is adopted to make the measurement process more efficient. The measurement position is visually located, and the aircraft fastener data is quickly and accurately collected and processed, thereby getting rid of the problems of low efficiency and poor accuracy of conventional manual measurement. The measuring device is compact and lightweight in design, and is easy to carry and use. The user's comfort and ease of operation are also improved through the design of auxiliary structures.

[0101] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0102] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A device for measuring the concave-convex amount of aircraft fasteners, characterized in that: It comprises an upper cover (3), a lower cover (10) and a device box (11); it also comprises an apparatus operation module, a data acquisition module, an embedded system (4), an apparatus power supply module and an auxiliary measurement module; The device operation module is used for the user to operate the measuring device; The data acquisition module comprises a laser rangefinder (16), a two-dimensional camera (17) and a binocular structured light camera (18), wherein the binocular structured light camera (18) is fixed on the device box (11) and under the device upper cover (3), and is used to collect point cloud data and image data of the aircraft surface; the laser rangefinder (16) and the two-dimensional camera (17) are fixed in the device box (11), and respectively obtain the distance data between the laser rangefinder (16) and the aircraft surface, and the image of the aircraft surface in the shooting area of ​​the two-dimensional camera (17); The embedded system (4) is used to receive operation instructions from the device operation module, send acquisition instructions to the data acquisition module, store and process data acquired by the data acquisition module, mark the fasteners on the aircraft surface and calculate the concave-convex amount; The device power supply module and the auxiliary measurement module are used for device power supply and auxiliary measurement respectively; The auxiliary measurement module comprises an auxiliary handle, a measurement bracket and a status display light (5); wherein the auxiliary handle comprises a lower handle (12) and an anti-slip handle (6); the lower handle (12) is connected to the device box (11) through a screw thread and is used for an operator to support the device; the anti-slip handle (6) is connected to the device upper cover (3) through a screw thread; and the rear end of the anti-slip handle (6) is provided with an anti-slip rubber (7); the measurement bracket comprises a carbon fiber bracket (20) and a soft support (21); the measurement bracket is connected to the threaded hole of the device upper cover (3) through a screw thread structure; the carbon fiber bracket (20) provides a solid support for the device when measuring fasteners on the surface of an aircraft, thereby ensuring the accuracy and stability of the measurement; and the soft support (21) enables the measurement bracket to fit tightly with the surface of the aircraft, thereby reducing vibration and interference and protecting the surface of the aircraft from damage.

2. The device for measuring the concave-convex amount of aircraft fasteners according to claim 1, characterized in that: The device operation module comprises a display touch screen (1), a trigger (15) and a push button (19); wherein the display touch screen (1) is fixed to a screen heat sink (2) by means of a fastener, and the screen heat sink (2) is connected to a device upper cover (3) by means of a hinge, so as to achieve a 90° pitch rotation of the display touch screen (1); the trigger (15) is arranged at the front end of the lower handle (12), and is used by an operator to pull the trigger to quickly start measurement; the push buttons (19) are respectively a power off button, a restart button and a screen off button, and are installed on the upper surface of the device upper cover (3), and are used by an operator to press and achieve corresponding operations.

3. The device for measuring the concave-convex amount of aircraft fasteners according to claim 1, characterized in that: The device power supply module comprises an aviation socket (8) capable of independently supplying power and a quick-release battery (14); wherein the aviation socket (8) is installed inside the device box (11); a socket is provided at the upper end of the quick-release battery (14), which is connected to a socket provided at the bottom of the lower handle (12) to realize power supply for the entire device; the quick-release battery (14) and the lower handle (12) are connected via a spring lock (13).

4. A method for measuring the concave-convex amount of aircraft fasteners, characterized in that: The following steps are involved: S1. The device power supply module supplies power to the device, starts the measuring device according to any one of claims 1 to 3, and sets the exposure parameters of the binocular structured light camera (18), the fastener detection algorithm parameters and the concave-convex amount calculation algorithm parameters through the display touch screen of the device operation module; S2, moving the device to the aircraft surface to be measured, the two-dimensional camera (17) provides real-time feedback of the measurement area, the laser rangefinder (16) provides real-time feedback of the distance between the device and the aircraft surface, and the measuring bracket of the auxiliary measurement module is closely attached to the aircraft surface and adjusted to the optimal measurement area; S3, the device starts measuring, the binocular structured light camera (18) obtains the aircraft surface data, and then transmits the data to the embedded system (4); S4, after the measurement operation is completed, the embedded system (4) converts the data obtained by the binocular structured light camera (18) into image data and point cloud data, executes the fastener detection algorithm and the concave-convex quantity calculation algorithm, identifies the fasteners in the image and marks them, and obtains the concave-convex quantity results of the aircraft fasteners; S5. The embedded system (4) transmits the convex-concave measurement results of the aircraft fasteners to the device operation module, displays the results, prompts the fasteners whose convex-concave measurement does not meet the requirements, and guides the workers to repair or reinstall them.

5. A method for measuring the concavo-convexity of aircraft fasteners according to claim 4, characterized in that: The specific process of S4 includes the following steps: S41, the embedded system (4) converts the scanned aircraft surface data into corresponding image data and point cloud data according to the SDK of the binocular structured light camera (18); S42, executing a fastener detection algorithm based on the image data, identifying and marking fasteners in the image, wherein the marked pixels are mapped to point cloud data according to a corresponding relationship in the SDK of the binocular structured light camera (18), thereby obtaining projected point cloud data; S43, executing a concave-convex amount calculation algorithm based on the projected point cloud data, segmenting the fastener point cloud, denoising and fitting the fastener and surface skin point clouds, and finally calculating the distance to obtain the concave-convex amount result of each fastener.

6. A method for measuring the concavo-convexity of aircraft fasteners according to claim 4, characterized in that: The S42 uses the fastener detection algorithm to regress the bounding box drawn by the fastener, using the following parameterized equation: Where x, y, w and h represent the horizontal and vertical coordinates of the center of the bounding box and its width and height; The variable s represents the value of the additional feature; variable x a ,y a 、w a and h a Respectively represent the horizontal and vertical coordinates, width and height of the center of the anchor frame; and are the normalized parameters of x, y, w, h and s respectively.

7. The method for measuring the concave-convex amount of aircraft fasteners according to claim 4, characterized in that: In the fastener detection process, the marked pixel points are mapped to the point cloud data according to the corresponding relationship in the SDK to generate the projected point cloud data. The specific corresponding relationship is as follows: Each point in the point cloud is point(i), where i=x+y*w, x is the column number where the pixel is located, y is the row number where the pixel is located, and w is the overall pixel width of the image.

8. The method for measuring the concavo-convex amount of aircraft fasteners according to claim 4, characterized in that: The fastener detection algorithm is provided with a confidence threshold and a bounding box threshold; In target detection, a confidence score is estimated for each detection result, which indicates the probability or confidence that the detection result contains the target. A confidence threshold is used to filter out detection results with confidence scores lower than the threshold. In the non-maximum suppression process, if the IoU value between two bounding boxes is greater than or equal to the bounding box threshold, the bounding box with lower confidence is suppressed; The overlap index IoU is: Among them, Area intersection Represents the intersection area of ​​two bounding boxes, Area union Represents the union area of ​​two bounding boxes, Feature intersection The feature representing the intersection, Feature union Represents the characteristics of a union; α, β, γ and δ are weighting factors of the intersection area, union area, intersection feature and union feature respectively.

Images