Vision-enhanced laser measurement system and measurement method

Abstract

The invention relates to the technical field of airplane horizontal measurement and airborne equipment calibration, in particular to a vision-enhanced laser measurement system and a measurement method. The system comprises an AGV intelligent transport vehicle, a multi-section lifting electric cylinder arranged on the AGV intelligent transport vehicle, a rotary table arranged on the multi-section lifting electric cylinder, a laser radar and a high-resolution telephoto camera which are arranged on the rotary table, and a control terminal connected with the AGV intelligent transport vehicle, the multi-section lifting electric cylinder, the laser radar and the high-resolution telephoto camera through networking equipment. The control terminal is provided with image processing software and a target recognition algorithm. According to the invention, the measurement precision is improved; the measurement efficiency is improved; the adaptability and the flexibility are improved; and the long-term cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of aircraft level measurement and airborne equipment calibration, in particular to a vision-enhanced laser measurement system and measurement method. BACKGROUND

[0002] In the field of modern industrial automation and intelligent manufacturing, especially in the aerospace, automobile manufacturing and other industries, accurate three-dimensional space measurement is crucial to ensure product quality and assembly precision. Traditional manual measurement methods are not only inefficient, but also susceptible to human factors, resulting in unstable measurement results and low precision.

[0003] With the development of non-contact measurement technology, especially the intelligent measurement system combining machine vision and laser measurement technology, it has gradually become the preferred choice in the industry due to its high precision, high efficiency, and intelligence. Therefore, a vision-enhanced laser measurement system and measurement method are proposed to improve efficiency and measurement accuracy and reduce labor costs, providing strong technical support for intelligent manufacturing. SUMMARY

[0004] To solve the above technical problems, the present application proposes a vision-enhanced laser measurement system and measurement method. It can complete the measurement of the whole aircraft level, the calibration of airborne equipment, and the rapid and accurate measurement of large and complex structures without human intervention.

[0005] The technical problems solved by the present application are realized by the following technical solutions:

[0006] A vision-enhanced laser measurement system, comprising:

[0007] AGV intelligent transport vehicle;

[0008] Multi-section lifting electric cylinder, arranged on the AGV intelligent transport vehicle, for meeting different measurement height requirements;

[0009] Laser radar, arranged on the top of the multi-section lifting electric cylinder, for emitting laser signals and receiving reflected signals, and obtaining distance information by processing mixed signals;

[0010] High-resolution long-focus camera, arranged on the top of the multi-section lifting electric cylinder and tracking with the laser radar, for observing the feature points and details to be measured, and providing accurate auxiliary positioning information for the laser radar;

[0011] The control terminal is connected with the AGV intelligent transport vehicle, the multi-section lifting electric cylinder, the laser radar and the high-resolution long-focus camera through the networking equipment adopting the star type topological structure, and is provided with an image processing software and a target recognition algorithm, and is used for controlling the movement of the AGV intelligent transport vehicle and the multi-section lifting electric cylinder, and performing image processing and target recognition on the image data photographed by the laser radar and the high-resolution long-focus camera, and for the long-distance feature points, the positioning information is obtained through the high-resolution long-focus camera, and then the feature point position is measured according to the positioning information by the laser radar, and a three-dimensional coordinate is constructed.

[0012] As a further improvement of the application, the top of the multi-section lifting electric cylinder is provided with a rotary table, and the laser radar and the high-resolution long-focus camera are arranged on the rotary table in parallel, and the rotary table is signal-connected with the control terminal.

[0013] As a further improvement of the application, the control terminal is connected with an internal storage system and a hardware master control through the networking equipment, and the hardware master control is connected with the AGV intelligent transport vehicle and the multi-section lifting electric cylinder.

[0014] As a further improvement of the application, the image processing software includes denoising processing, grayscale and contrast enhancement, and geometric distortion correction, wherein the denoising processing adopts a non-local mean filter, and the noise is suppressed by a weighted average of pixel neighborhood similarity, and the formula is as follows:

[0015]

[0016] In the formula, w(x, y) is the similarity weight of pixel x and neighborhood y, and C(x) is a normalized coefficient.

[0017] As a further improvement of the application, the geometric distortion correction is based on the camera intrinsic matrix K and the distortion coefficients (k1, k2, p1, p2), and the optical distortion is compensated, and the calculation formula of the corrected pixel coordinates is as follows:

[0018]

[0019] In the formula, r2=x2+y2, and (x, y) is the original pixel coordinate.

[0020] As a further improvement of the application, the target recognition algorithm includes a traditional feature algorithm and a deep learning target detection algorithm, wherein the traditional feature algorithm extracts key points through scale-invariant feature, and is suitable for high-texture areas, and the formula is as follows:

[0021] D(x, sigma) = G(x, k sigma) * I(x) - G(x, sigma) * I(x)

[0022] In the formula, G(x, sigma) is a Gaussian kernel, k is a scale factor, and D(x, sigma) is a Gaussian difference (DoG) response value.

[0023] As a further improvement of the present application, the deep learning target detection algorithm detects targets in real time through a convolutional neural network, is suitable for low-texture or structured components, and has a loss function as follows:

[0024]

[0025] In the formula, λ coord and λ class are weight coefficients, 1 ij obj is an object existence indicator function, and outputs the boundary box coordinates (x, y) and the class probability pi(c).

[0026] A visual enhancement laser measurement method applies the above-mentioned visual enhancement laser measurement system and comprises the following steps:

[0027] Step (a), turn on the power of the measurement system:

[0028] The AGV intelligent transport vehicle, the multi-section lifting electric cylinder, the laser radar, the high-resolution long-focus camera, the turntable, the networking equipment, and the control terminal are powered on and work;

[0029] Step (b), the measurement system completes initialization:

[0030] The laser radar emits visible guide laser for positioning and invisible measurement laser for measurement, the guide laser and the measurement laser are coaxial, the measurement software on the control terminal is run, the interface of the measurement software displays the images taken by the built-in camera and the high-resolution long-focus camera on the laser radar, and the guide laser point of the laser radar is displayed in the center of the video image;

[0031] Step (c), measure the whole aircraft horizontally and construct the aircraft coordinate system:

[0032] In the video interface of the laser radar, the mouse is dragged to control the guide laser point of the laser radar to point to the specified feature point, the built-in camera on the laser radar simultaneously points to the feature point, the control terminal runs the follow-up algorithm of the turntable to generate the control instruction of the turntable, the turntable drives the high-resolution long-focus camera to move, and the pointing angle of the built-in camera on the laser radar is followed up; for a long-distance feature point, the feature point and the guide laser point video taken by the built-in camera on the laser radar are blurred, the enhanced image of the high-resolution long-focus camera is switched to be used, the laser radar completes the measurement of all specified feature point coordinate values, and the control terminal automatically completes the construction of the coordinate system according to the feature point coordinate values;

[0033] Step (d), measurement implementation:

[0034] Select the target point that needs to be measured in the software interface of the control terminal, control the AGV intelligent transport vehicle and the multi-section lifting electric cylinder to the appropriate position, move the mouse to control the laser measurement head of the laser radar to point to the target point, make the target measurement point enter the shooting range of the built-in camera and high-resolution long-focus camera on the laser radar, control the terminal to run the image processing software, automatically identify and mark the target point, quickly guide the laser measurement head of the laser radar to accurately point to the target point, so as to implement the coordinate measurement of the target point, and the calibration error can be obtained by comparing and calculating the actual coordinate value of the target point with the theoretical coordinate value.

[0035] The beneficial effects of the present application are:

[0036] The measurement accuracy is improved: the present application can realize nanometer-level measurement accuracy by combining high-definition visual sensors with laser measurement technology, which is significantly better than traditional contact measurement methods, especially in the measurement of complex surfaces and curved structures, and the visual guidance algorithm further optimizes target recognition and positioning accuracy.

[0037] The measurement efficiency is improved: the present application realizes fast scanning and automatic processing, greatly reduces the time cost of manual intervention and manual adjustment, quickly identifies key measurement points with the aid of the aircraft model, and avoids tedious preparation work.

[0038] The adaptability and flexibility are increased: the present application can not only be used for static measurement, but also is suitable for dynamic real-time measurement, can maintain high reliability under vibration and complex light conditions, and at the same time, is compatible with traditional manual target ball guiding mode, enhances the universality and adaptability of the system, and meets diversified measurement requirements.

[0039] The long-term cost is reduced: the present application reduces the dependence on target and other consumables, reduces the use cost of auxiliary tools, and has good economic benefits. BRIEF DESCRIPTION OF DRAWINGS

[0040] The present application will be further described below in combination with the drawings and examples:

[0041] Figure 1 It is a schematic diagram of the hardware structure of the measurement system of the present application;

[0042] Figure 2 It is a network topology schematic diagram of the network equipment of the measurement system of the present application;

[0043] Figure 3 It is a relationship geometry of the triangle of the measurement system of the present application;

[0044] Figure 4 It is a geometry diagram of the triangulation method of the measurement system of the present application;

[0045] Figure 5 It is a collaborative work schematic diagram of the measurement system of the present application.

[0046] In the figure: 1, AGV intelligent transport vehicle; 2, multi-section lifting electric cylinder; 3, laser radar; 4, high-resolution long-focus camera; 5, rotary table; 6, networking equipment; 7, control terminal; 8, internal storage system; 9, hardware total control. DETAILED DESCRIPTION

[0047] In order to make the technical means, creative features, purposes and effects of the present application easy to understand, the present application is further described below in conjunction with the drawings and examples.

[0048] As shown in Figure 1 and Figure 2 , a visual enhancement laser measurement system mainly includes AGV intelligent transport vehicle 1, multi-section lifting electric cylinder 2, laser radar 3, high-resolution long-focus camera 4, rotary table 5, networking equipment 6, control terminal 7, internal storage system 8, and hardware total control 9. The internal storage system 8 is used for storing data.

[0049] Further, the AGV intelligent transport vehicle 1 serves as an omnidirectional mobile platform and can move omnidirectionally around the aircraft to meet the requirements of multi-angle measurement. The multi-section lifting electric cylinder 2 is arranged on the AGV intelligent transport vehicle 1 and can be lifted to a height of seven meters to meet the requirements of different measurement heights. The electric cylinder uses screw transmission or gear transmission inside to reduce the gap and error in the movement process and ensure high precision of measurement. The rotary table 5 is arranged on the top of the multi-section lifting electric cylinder 2 and is signal-connected with the control terminal 7. Through signal control of the control terminal 7, six-degree-of-freedom high-precision rotation can be realized. The laser radar 3 is arranged on the rotary table 5 and is used for emitting laser signals and receiving reflected signals to obtain distance information and the like by processing mixed signals. The laser radar measurement head has powerful six-degree-of-freedom function, supports high-speed dynamic measurement, and is built-in with a camera for shooting features of the measured object. The high-resolution long-focus camera 4 is arranged on the rotary table 5 and is arranged in parallel with the laser radar 3 to provide long-distance high-definition imaging capability and can clearly observe the feature points and details to be measured in the distance without contacting the measured object. It provides accurate auxiliary positioning information for the laser radar.

[0050] The function of the high-resolution long-focus camera 4 is mainly to provide a high-definition image for the measurement position that cannot be clearly observed by the laser radar 3. There is a relative spatial relationship between the high-resolution long-focus camera 4 and the laser radar 3. The ideal direction of the visual enhancement system is calculated according to the distance of the laser radar and the measurement point, the horizontal and vertical rotation angles of the measurement point relative to the instrument, and the relative position relationship between the two devices to ensure that the high-resolution long-focus camera 4 can accurately obtain the position of the laser point indicated by the laser radar 3 and play an auxiliary role in measurement accuracy and progress.

[0051] Because the turntable 5 and the laser radar 3 have a certain front-back relationship and angle conversion in the horizontal direction, and also have a height difference in the vertical direction, it is necessary to use a triangle to solve the conversion between angles. The following calculation mode takes the horizontal direction as an example.

[0052] As shown in the accompanying Figure 3 The β angle point in the triangle is the position of the laser radar 3, the γ angle point is the position of the turntable 5, and the α angle point is the measured point position. Because the laser radar 3 can measure the angle and the distance, β and the distance c are known, and the distance a is the distance between the laser radar 3 and the rotation center of the turntable 5 and is known, so other parameters can be solved according to the relationship of the triangle.

[0053] The distance calculation formula is as follows:

[0054]

[0055] The angle calculation formula is as follows:

[0056]

[0057] γ=π-α-β

[0058] Similarly, the angle in the pitch direction can also be calculated by this method.

[0059] Laser triangulation method: as shown in the accompanying Figure 4 , a geometric measurement method using the relationship of a triangle, by projecting a light point onto the measurement object, the distance between the measurement object is determined according to the angle change of the reflected light.

[0060] The high-resolution long-focus camera 4 and the laser radar 3 have a cooperative working mechanism: as shown in the accompanying Figure 5 , the high-resolution long-focus camera 4 first identifies key feature points, and then the laser radar 3 uses these information to quickly locate and measure these points, and constructs the three-dimensional coordinates of the measured object. This mechanism improves the speed and accuracy of measurement, and is particularly suitable for complex measurement scenarios.

[0061] As Figure 2 shown, the networking device 6 adopts a star-shaped topology Ethernet group structure to ensure stable communication between the laser radar, the high-resolution long-focus camera and the computer.

[0062] The control terminal 7 is a tablet computer based on the Windows system, provided with image processing software and target recognition algorithm, and the control terminal 7 is connected with the AGV intelligent transport vehicle 1, the multi-section lifting electric cylinder 2, the laser radar 3, the high-resolution long-focus camera 4 and the internal storage system 8 through the networking device 6 and the hardware total control 9 respectively. The image processing software is used to improve the image quality and adapt to the subsequent processing requirements.

[0063] The preprocessing includes denoising, grayscale and contrast enhancement, and geometric distortion correction.

[0064] The denoising is performed by non-local mean filtering, which suppresses noise by weighted average of pixel neighborhood similarity, and the formula is as follows:

[0065]

[0066] In the formula, w(x, y) is the similarity weight of pixel x and neighborhood y, and C(x) is the normalized coefficient.

[0067] Grayscale and contrast enhancement: After converting the RGB image to a grayscale image, the local details are enhanced by applying limited contrast adaptive histogram equalization.

[0068] The geometric distortion correction is based on the camera intrinsic matrix K and distortion coefficients (k1, k2, p1, p2) to compensate for optical distortion, and the formula for calculating the corrected pixel coordinates is as follows:

[0069]

[0070] In the formula, r2 = x2 + y2, and (x, y) is the original pixel coordinate.

[0071] The preprocessed image is used as the input for subsequent feature extraction to ensure the clarity and geometric accuracy of key features.

[0072] The target recognition algorithm includes traditional feature algorithms and deep learning target detection algorithms.

[0073] The traditional feature algorithm extracts key points by scale-invariant feature extraction and is suitable for high-texture areas, and the formula is as follows:

[0074] D(x, σ) = G(x, kσ) * I(x) - G(x, σ) * I(x)

[0075] In the formula, G(x, σ) is the Gaussian kernel, k is the scale factor, and D(x, σ) is the Gaussian difference (DoG) response value.

[0076] Feature descriptor: For each key point, a 16x16 pixel region gradient direction histogram is extracted to generate a 128-dimensional vector descriptor, achieving rotation and scaling invariance.

[0077] The deep learning target detection algorithm detects targets in real time through a convolutional neural network and is suitable for low-texture or structured components, and the loss function is as follows:

[0078]

[0079] In the formula, λ coord and λ classis a weight coefficient, 1 ij obj is an object existence indicator function, outputting the bounding box coordinates (x, y) and class probability pi(c).

[0080] A visual enhanced laser measurement method, applying the above-mentioned visual enhanced laser measurement system, comprising the following steps:

[0081] Step (a), turn on the power of the measurement system:

[0082] The AGV intelligent transport vehicle 1, the multi-section lifting electric cylinder 2, the laser radar 3, the high-resolution long-focus camera 4, the turntable 5, the networking equipment 6 and the control terminal 7 are powered on and work;

[0083] Step (b), the measurement system completes initialization:

[0084] The laser radar 3 emits visible guide laser for positioning and invisible measurement laser for measurement, the guide laser and the measurement laser are coaxial, the measurement software on the control terminal 7 is run, the interface of the measurement software displays the images taken by the built-in camera on the laser radar 3 and the high-resolution long-focus camera 4, and the guide laser point of the laser radar 3 is displayed in the center of the video image;

[0085] Step (c), measure the whole aircraft horizontally and construct the aircraft coordinate system:

[0086] In the video interface of the laser radar 3, the mouse is dragged to control the guide laser point of the laser radar 3 to point to the specified feature point, the built-in camera on the laser radar 3 simultaneously points to the feature point, the control terminal 7 runs the follow-up algorithm of the turntable 5 to generate the control instruction of the turntable 5, the turntable 5 drives the high-resolution long-focus camera 4 to move, and the pointing angle of the built-in camera on the laser radar 3 is followed, for the long-distance feature point, the feature point and the guide laser point video taken by the built-in camera on the laser radar 3 are blurred, the enhanced image of the high-resolution long-focus camera 4 is switched to use, the laser radar 3 completes the measurement of all specified feature point coordinate values, and the control terminal 7 automatically completes the construction of the coordinate system according to the feature point coordinate values;

[0087] Step (d), measurement implementation:

[0088] In the software interface of the control terminal 7, the target point to be measured is selected, the AGV intelligent transport vehicle 1 and the multi-section lifting electric cylinder 2 are controlled to the appropriate position, the mouse is moved to control the laser measurement head of the laser radar 3 to point to the target point, so that the target measurement point enters the shooting range of the built-in camera on the laser radar 3 and the high-resolution long-focus camera 4, the control terminal 7 runs the image processing software to automatically identify and mark the target point, quickly guides the laser measurement head of the laser radar 3 to accurately point to the target point, so as to implement the coordinate measurement of the target point, and the calibration error can be obtained by comparing and calculating the actual coordinate value of the target point with the theoretical coordinate value.

[0089] The foregoing is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and practice described. Accordingly, all such variations and modifications are intended to be included within the scope of the application as defined in the following claims and the equivalents thereof.

Claims

1. A vision-augmented laser measurement system, characterized by: include: AGV intelligent transport vehicle (1); A multi-section lifting electric cylinder (2) is installed on the AGV intelligent transport vehicle (1) to meet different measurement height requirements; The lidar (3) is set on the top of the multi-section lifting electric cylinder (2) to emit laser signals and receive reflected signals, and obtain distance information by processing the mixed signals; A high-resolution telephoto camera (4) is set on top of the multi-section lifting electric cylinder (2) and moves with the lidar (3) to observe the feature points and details to be measured, and to provide the lidar (3) with accurate auxiliary positioning information. The control terminal (7) is connected to the AGV intelligent transport vehicle (1), the multi-section lifting electric cylinder (2), the lidar (3) and the high-resolution telephoto camera (4) respectively through the networking equipment (6) with a star topology. The control terminal (7) is equipped with image processing software and target recognition algorithm to control the movement of the AGV intelligent transport vehicle (1) and the multi-section lifting electric cylinder (2), and to perform image processing and target recognition on the image data captured by the lidar (3) and the high-resolution telephoto camera (4). For distant feature points, the high-resolution telephoto camera (4) is used to identify and obtain positioning information first, and then the lidar (3) is used to measure the feature point position and construct three-dimensional coordinates based on the positioning information.

2. A vision-enhanced laser measurement system according to claim 1, wherein: A turntable (5) is provided on the top of the multi-section lifting electric cylinder (2). A laser radar (3) and a high-resolution telephoto camera (4) are arranged in parallel on the turntable (5). The turntable (5) is connected to the control terminal (7) via signal.

3. A vision-enhanced laser measurement system according to claim 1, wherein: The control terminal (7) is connected to the internal storage system (8) and the hardware control unit (9) through the networking equipment (6). The hardware control unit (9) is connected to the AGV intelligent transport vehicle (1) and the multi-section lifting electric cylinder (2).

4. A vision-enhanced laser measurement system according to claim 1, wherein: Image processing software includes denoising, grayscale conversion and contrast enhancement, and geometric distortion correction. The denoising process employs non-local mean filtering, which suppresses noise through a weighted average based on pixel neighborhood similarity, as shown in the following formula: In the formula, w(x,y) is the similarity weight between pixel x and its neighbor y, and C(x) is the normalization coefficient.

5. A vision-enhanced laser measurement system according to claim 4, wherein: Geometric distortion correction compensates for optical distortion based on the camera intrinsic parameter matrix K and distortion coefficients (k1,k2,p1,p2). The formula for calculating the corrected pixel coordinates is as follows: In the formula, r2 = x2 + y2, and (x, y) are the original pixel coordinates.

6. A vision-enhanced laser measurement system according to claim 1, wherein: Target recognition algorithms include traditional feature extraction algorithms and deep learning-based target detection algorithms. Traditional feature extraction algorithms extract key points using scale-invariant features and are suitable for highly textured regions. The formula is as follows: D(x,σ)=G((x,kσ)*I(x)-G(x,σ)*I(x) In the formula, G(x,σ) is the Gaussian kernel, k is the scaling factor, and D(x,σ) is the difference in Gaussians (DoG) response value.

7. A vision-enhanced laser measurement system according to claim 6, wherein: Deep learning object detection algorithms detect objects in real time using convolutional neural networks. They are suitable for low-texture or structured parts, and the loss function is: In the formula, λ coord and λ class are weight coefficients, 1 ij obj is an object existence indication function, and outputs the bounding box coordinates (x, y) and the class probability pi(c).

8. A method of vision-enhanced laser measurement, characterized by: The application of a vision-enhanced laser measurement system according to any one of claims 1 to 7 includes the following steps: Step (a) Turn on the power to the measurement system: AGV intelligent transport vehicle (1), multi-section lifting electric cylinder (2), laser radar (3), high-resolution long-focus camera (4), rotary table (5), networking equipment (6) and control terminal (7) are powered on and work; Step (b), the measurement system completes initialization: The laser radar (3) emits visible guide laser for positioning and invisible measurement laser for measurement, the guide laser and the measurement laser are coaxial, the measurement software on the control terminal (7) is run, the interface of the measurement software displays the images taken by the built-in camera on the laser radar (3) and the high-resolution long-focus camera (4), and the guide laser point of the laser radar (3) is displayed in the center of the video image; Step (c), measure the whole aircraft horizontally and construct the aircraft coordinate system: In the video interface of the laser radar (3), the mouse is dragged to control the guide laser point of the laser radar (3) to point to the specified feature point, the built-in camera on the laser radar (3) points to the feature point at the same time, the control terminal (7) runs the follow-up algorithm of the rotary table (5) to generate the control instruction of the rotary table (5), the rotary table (5) drives the high-resolution long-focus camera (4) to move, and the pointing angle of the built-in camera on the laser radar (3) is followed, for the long-distance feature point, the feature point and the guide laser point video taken by the built-in camera on the laser radar (3) are blurred, the enhanced image of the high-resolution long-focus camera (4) is switched to use, the laser radar (3) completes the measurement of all specified feature point coordinate values, and the control terminal (7) automatically completes the construction of the coordinate system according to the feature point coordinate values; Step (d), measurement implementation: In the software interface of the control terminal (7), the target point to be measured is selected, the AGV intelligent transport vehicle (1) and the multi-section lifting electric cylinder (2) are controlled to the appropriate position, the mouse is moved to control the laser measurement head of the laser radar (3) to point to the target point, so that the target measurement point enters the shooting range of the built-in camera on the laser radar (3) and the high-resolution long-focus camera (4), the control terminal (7) runs the image processing software to automatically identify and mark the target point, quickly guides the laser measurement head of the laser radar (3) to accurately point to the target point, so as to implement the coordinate measurement of the target point, and the calibration error can be obtained by comparing and calculating the actual coordinate value of the target point with the theoretical coordinate value.

Images