Large-space three-dimensional feature point autonomous photoelectric measurement system and method, electronic equipment and storage medium

Abstract

The invention discloses a large-space three-dimensional feature point autonomous photoelectric measurement system and method, electronic equipment and a storage medium. The method comprises the following steps: step 1, building the large-space three-dimensional feature point autonomous photoelectric measurement system; 2, initializing a system, and starting data acquisition; 3, acquiring and processing a space key point image and extracting a central point; and 4, guiding the laser tracker to measure space key points. According to the invention, a high-resolution camera is built through a rotary table and is used for searching and collecting key point images of a component, feature extraction and key point identification are carried out on image data through a controller by using a deep learning technology, and accurate extraction of the center of a circular light spot is carried out through a Hessian matrix method. The laser tracker is guided to adjust the pitching angle and the yaw angle of the laser tracker through a PID algorithm, the center of a feature point subjected to visual recognition coincides with the center of a light spot projected by the laser tracker, and automatic, intelligent and high-precision measurement of key points on the surface of a large component can be achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of photoelectric measurement, and specifically relates to an autonomous photoelectric measurement system and method for three-dimensional feature points in a large space, an electronic device and a storage medium. Background Art

[0002] High-precision measurement of key points of large components is essential to ensure product quality, improve production efficiency and ensure safe operation. In the field of high-end equipment manufacturing, such as satellites and high-speed railways, accuracy directly affects the reliability and performance of products. For example, the relative attitude angle measurement accuracy of satellite antennas, star sensors and other loads is crucial for on-orbit operation, and the precise measurement of high-speed railway bogies is an important means to ensure and inspect the assembly quality of high-speed railways. In addition, the installation posture measurement accuracy of large components is directly related to the safe operation of equipment. For example, the wheelset of a high-speed railway bogie has strict requirements on neutrality and parallelism after being assembled to the bogie to ensure safe operation.

[0003] Photoelectric measurement technology uses light waves as a length reference, has sub-nanometer resolution and nanometer accuracy, and can achieve fast and accurate measurement. It also has the characteristics of high directionality, high monochromaticity, and high coherence. It is widely used in the field of ultra-precision machining and measurement, has good stability, and strong resistance to electromagnetic interference, and is suitable for measurement in complex electromagnetic environments. Photoelectric measurement is non-contact, which can avoid damage to the object being measured and reduce errors caused by contact. At present, photoelectric measurement technology has been widely used in industrial automation, security monitoring, robot navigation and other fields. It has high reliability and strong adaptability, and can achieve coordinated measurement of the position and posture of multiple components in a large range. However, there is little research on how to apply it to the high-precision measurement of key points of large components, and there is a technical gap. Summary of the invention

[0004] In order to solve the technical problems existing in the prior art, the purpose of the present invention is to provide a system and method for autonomous photoelectric measurement of three-dimensional feature points in a large space, an electronic device and a storage medium.

[0005] In order to achieve the above purpose and the above technical effect, the technical solution adopted by the present invention is:

[0006] A method for autonomous photoelectric measurement of three-dimensional feature points in a large space, comprising the following steps:

[0007] Step 1: Build an autonomous photoelectric measurement system for large-space three-dimensional feature points

[0008] The large-space three-dimensional feature point autonomous photoelectric measurement system includes a controller, a movable AGV vehicle and a camera and a laser tracker arranged thereon, wherein the controller is connected to the camera and the laser tracker;

[0009] Step 2: Initialize the system and start data collection;

[0010] Step 3: spatial key point image acquisition, processing and center point extraction;

[0011] Step 4: Guide the laser tracker to measure key points in space.

[0012] Furthermore, in step one, the large-space three-dimensional feature point autonomous photoelectric measurement system also includes a power supply module, which is connected to the controller, the camera and the laser tracker and supplies power to the controller, the camera and the laser tracker.

[0013] Furthermore, the camera is installed on a turntable, and the turntable is mounted on an AGV vehicle.

[0014] Furthermore, in step 3, the steps of spatial key point image acquisition, processing and center point extraction include:

[0015] 31) The AGV moves according to the preset path planning. During the movement, the turntable drives the camera to rotate. The camera's field of view covers different areas in the large space. The camera captures the key point images in the space in real time and transmits them to the controller;

[0016] 32) The controller uses deep learning technology to extract features and identify key points of the image data collected by the camera to obtain the coordinates of the key points;

[0017] 33) The center of the circular light spot is accurately extracted using the Hessian matrix method.

[0018] Furthermore, in step 3), the step of accurately extracting the center of the circular light spot by using the Hessian matrix method includes:

[0019] 331) First, the original image is preprocessed, including grayscale conversion and noise removal, to improve the accuracy of subsequent processing;

[0020] 332) For each pixel in the image, the Hessian matrix is ​​constructed by calculating the second-order partial derivative of its gray value; the determinant of the Hessian matrix is ​​used to detect feature points. For a circular light spot, a specific pattern will be shown in the eigenvalue analysis of the Hessian matrix. The position of the center of the light spot can be determined by finding the maximum value point of the determinant. In order to further improve the extraction accuracy of the center point, sub-pixel level refinement is performed near the center point detected by the Hessian matrix;

[0021] Through the above steps, the Hessian matrix method can accurately extract the center of the circular light spot and maintain high stability and accuracy even in the presence of noise interference or undesirable lighting conditions.

[0022] Furthermore, in step 4, the step of guiding the laser tracker to measure the key points in space includes:

[0023] The PID algorithm is used to guide the laser tracker to adjust its pitch and yaw angles, so that the center of the feature point recognized by vision coincides with the center of the light spot projected by the laser tracker, thereby realizing automatic measurement of key points in space.

[0024] Furthermore, the steps of guiding the laser tracker to adjust its pitch and yaw angles by the PID algorithm include:

[0025] 411) respectively extracting the coordinates of the center of the feature point and the center of the light spot projected by the laser tracker;

[0026] 412) There is a positional deviation between the center of the feature point and the center of the light spot projected by the laser tracker, which can be determined by calculating the coordinate difference between the two, that is, the error; the calculated error is used as the input of the controller to guide the adjustment of the laser tracker;

[0027] 413) The calculated control quantity is applied to the laser tracker to adjust its pitch and yaw angles so that the center of the light spot moves toward the center of the feature point. The controller continuously monitors the position deviation between the center of the light spot and the center of the feature point, and adjusts the laser tracker in real time until the two coincide, thereby achieving dynamic stable tracking and automatic measurement of key points in space.

[0028] The present invention also discloses an autonomous photoelectric measurement system for three-dimensional feature points in a large space, including a controller, a camera, a laser tracker, a power supply module, an AGV trolley and a turntable mounted thereon, wherein the camera is installed on the turntable, the laser tracker is mounted on the AGV trolley, and the AGV trolley can move according to a preset path planning, the power supply module is connected to the controller, the camera, and the laser tracker and supplies power to the controller, the camera, and the laser tracker, the controller is connected to the camera and the laser tracker, the camera captures images of key points in the space in real time and transmits them to the controller for analysis and processing, and the controller controls the laser tracker to automatically aim and accurately measure the key points in the space according to the position information provided by the camera.

[0029] The present invention also discloses an electronic device, comprising:

[0030] at least one processor;

[0031] a memory communicatively coupled to the at least one processor;

[0032] The memory stores a computer program executable by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the above-mentioned autonomous photoelectric measurement method of three-dimensional feature points in a large space.

[0033] The present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned autonomous photoelectric measurement method for three-dimensional feature points in a large space.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1) The present invention discloses an autonomous photoelectric measurement system and method for three-dimensional feature points in a large space, an electronic device and a storage medium. Based on the principles of visual detection and photoelectric testing, the system can realize the functions of automatic identification, positioning, aiming and three-dimensional measurement of key points of large spatial components through image acquisition, intelligent recognition and automatic aiming, assisted by a high-precision laser tracker and a controller, and can realize automatic, intelligent and high-precision measurement of key points on the surface of large components. The system has certain versatility and can realize the measurement requirements of components of different sizes by carrying AGV carts of different sizes, using cameras of different focal lengths, etc., and using different control devices and processes;

[0036] 2) The present invention uses a high-precision two-dimensional turntable equipped with a high-resolution camera to search and identify key points of components; an AGV trolley is used to carry a turntable and a laser tracker, and the controller is used to trigger data collection, which can realize the automatic collection of three-dimensional coordinates of key points on the surface of large-sized components;

[0037] 3) The present invention proposes a method for realizing spatial key point recognition and precise positioning based on a combination of a turntable and a high-resolution camera. The spatial key points are detected by a target detection algorithm based on deep learning, and the key point center is obtained by using a feature point center extraction algorithm, providing a basis for subsequent automatic recognition;

[0038] 4) The present invention proposes a method for measuring key points in space based on a camera-guided laser tracker. The camera is used to simultaneously identify the laser point and the center of the key point, and the PID control method is used to adjust the rotation of the laser tracker. Finally, the laser point emitted by the laser tracker coincides with the center of the key point in space, and the laser tracker is triggered to collect data, thereby completing the collection of three-dimensional data of the key point in space. The transfer method based on the laser tracker can realize the splicing of multiple sets of data and the overall measurement of large components. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the present invention;

[0040] Figure 2 It is a structural schematic diagram of the present invention;

[0041] Figure 3 It is a schematic diagram of the key point image and the laser spot image of the present invention. DETAILED DESCRIPTION

[0042] The present invention is described in detail below so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0043] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0044] like Figure 1-3 As shown, the present invention discloses an autonomous photoelectric measurement system for three-dimensional feature points in a large space, including: a flexible AGV trolley 1, a high-resolution camera 2, a laser tracker 3, a controller (preferably a PLC), and a stable power supply module, wherein the AGV trolley 1 can move according to a preset path planning, the camera 2 and the laser tracker 3 are respectively mounted on the AGV trolley 1, specifically, the camera 2 is installed on a turntable 4, and the turntable 4 carried on the AGV trolley 1 is a mobile platform, which allows the camera 2 to move flexibly in a large space and cover a wider measurement area. The power supply module is connected to the controller, the camera 2, and the laser tracker 3 and supplies power to the controller, the camera 2, and the laser tracker 3. The controller is connected to the camera 2 and the laser tracker 3. The key point images in the space are captured in real time by the camera 2 and transmitted to the controller for analysis and processing. The laser tracker 3 is controlled by the controller to automatically aim and accurately measure the key points in the space according to the position information provided by the camera 2.

[0045] The present invention also discloses a method for autonomous photoelectric measurement of three-dimensional feature points in a large space, comprising the following steps:

[0046] Step 1: Construction of autonomous photoelectric measurement system for large space three-dimensional feature points

[0047] In the process of building an autonomous photoelectric measurement system for three-dimensional feature points in a large space, an integrated solution is adopted to achieve high-precision measurement of key points in space. The core components of the system include a flexible AGV trolley 1, a high-resolution camera 2, a laser tracker 3, a controller (preferably a PLC), and a stable power supply module. The imaging and detection of key points in space by camera 2 is the basis of the system. It uses advanced image processing technology to capture and analyze the image information of key points to achieve accurate positioning of the target. The power supply module provides stable and reliable power support for the entire system, ensuring the performance stability of camera 2 and other electronic equipment during long-term operation. The turntable 4 carried on the AGV trolley 1 is the mobile platform of the system, which allows camera 2 to move flexibly in a large space and cover a wider measurement area. Through precise control, camera 2 can align and track key points in space to achieve all-round data collection. In addition, the system also integrates a laser tracker 3. This innovative design enables the laser tracker 3 to automatically aim and accurately measure key points in space based on the position information provided by camera 2. The application of this photoelectric measurement technology not only improves the level of measurement automation, but also significantly improves the measurement accuracy and efficiency, providing an efficient and reliable solution for large-space three-dimensional measurement.

[0048] Step 2: System initialization and data collection start

[0049] With the initialization and startup of the AGV 1, the data collection process begins. First, after receiving the startup command from the controller, the AGV 1 will perform a self-check procedure to ensure that the electronic equipment such as the camera 2 on it is in normal working condition. Then, the AGV 1 slowly starts according to the preset path planning and begins to move in the measurement area. During the movement, the camera 2 mounted on the turntable 4 starts working and captures the key point images in the space in real time. These image data contain the location information of the key points, providing a basis for subsequent measurement and analysis.

[0050] Step 3: Spatial key point image acquisition, processing and center point extraction

[0051] The key is the coordinated work of the camera 2, the turntable 4 and the AGV 1. The following is a detailed description of the process:

[0052] 31) Camera imaging detects key points in space. Turntable 4 drives camera 2 to rotate and cover different areas in the large space. Camera 2 captures images of key points in space. This process involves image acquisition and preprocessing, including denoising and enhancement, to improve the accuracy of subsequent key point detection;

[0053] 32) In the recognition process of deep learning methods, the controller uses deep learning technology, especially convolutional neural network (CNN), to extract features and identify key points of the collected images. By learning a large amount of image data, the deep learning model can identify key points in the image, such as edges, corners, etc. The heatmap method can be used. The convolutional neural network outputs heat maps of multiple channels. Each channel represents a type of key point. The high-value points on the heat map represent the possible positions of the key points. By extracting the maximum value points on the heat map, the coordinates of the key points can be obtained. In addition, the regression method can also be used to directly regress the coordinates of the key points. This method directly outputs the coordinates of the key points after processing through the fully connected layer;

[0054] 33) Center point extraction: The accurate extraction of the center of the circular light spot can be achieved by the Hessian matrix method, which is widely used in the field of computer vision, especially in high-precision three-dimensional measurement systems. The following is the detailed process of extracting the center of the circular light spot using the Hessian matrix method:

[0055] 331) First, the original image is preprocessed, including grayscale and noise removal, to improve the accuracy of subsequent processing. This step is to reduce the interference noise in the image, thereby improving the accuracy of spot center extraction;

[0056] 332) The Hessian matrix is ​​a second-order derivative matrix that describes the curvature of grayscale changes in an image. For each pixel in the image, the Hessian matrix is ​​constructed by calculating the second-order partial derivative of its grayscale value. The determinant of the Hessian matrix is ​​used to detect feature points. For a circular light spot, it will show a specific pattern in the eigenvalue analysis of the Hessian matrix. By finding the maximum value point of the determinant, the position of the center of the light spot can be determined. In order to further improve the extraction accuracy of the center point, sub-pixel level refinement can be performed near the center point detected by the Hessian matrix. This usually involves Taylor series expansion or high-order fitting to determine a more accurate center position. Through Taylor expansion, the sub-pixel coordinates of the light spot can be determined by minimizing the error in the neighborhood of the center point of the light spot;

[0057] Through the above steps, the Hessian matrix method can accurately extract the center of the circular light spot and maintain high stability and accuracy even in the presence of noise interference or undesirable lighting conditions.

[0058] Step 4: Guide the laser tracker to measure key points

[0059] To achieve the coincidence of the center of the feature point of visual recognition with the center of the light spot projected by the laser tracker 3, the laser tracker is guided to adjust its pitch and yaw angles through the PID algorithm, which can be divided into the following three steps:

[0060] 411) extracting the coordinates of the center of the feature point and the center of the light spot projected by the laser tracker 3 respectively by the above method;

[0061] 412) There is a position deviation between the center of the feature point identified by the visual system and the center of the light spot projected by the laser tracker 3. This deviation can be determined by calculating the coordinate difference between the two, that is, the error; the calculated error is used as the input of the PID controller to guide the adjustment of the laser tracker 3;

[0062] 413) Apply the calculated control amount to the laser tracker 3, adjust its pitch and yaw angles, and move the center of the light spot toward the center of the feature point. The system continuously monitors the position deviation between the center of the light spot and the center of the feature point, and adjusts the laser tracker 3 in real time until the two coincide, achieving dynamic stable tracking;

[0063] Through the above three steps, the PID algorithm can be effectively used to guide the laser tracker 3 to adjust its own pitch and yaw angles, so as to achieve the coincidence of the center of the light spot with the center of the spatial feature point, and realize the automatic measurement of the key points in space.

[0064] Through the above four steps, autonomous photoelectric measurement of three-dimensional feature points in a large space can be carried out, and the key points of large components can be accurately identified and measured. This method has certain versatility and can be extended to the inspection fields of trains, airplanes, ships, etc.

[0065] Parts or structures not specifically described in the present invention may adopt existing technologies or existing products and will not be described in detail here.

[0066] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for autonomous photoelectric measurement of three-dimensional feature points in a large space, characterized in that: The following steps are involved: Step 1: Build an autonomous photoelectric measurement system for large-space three-dimensional feature points The large-space three-dimensional feature point autonomous photoelectric measurement system includes a controller, a movable AGV vehicle and a camera and a laser tracker arranged thereon, wherein the controller is connected to the camera and the laser tracker; Step 2: Initialize the system and start data collection; Step 3: spatial key point image acquisition, processing and center point extraction; Step 4: Guide the laser tracker to measure key points in space.

2. The method for autonomous photoelectric measurement of three-dimensional feature points in a large space according to claim 1, characterized in that: In step one, the large-space three-dimensional feature point autonomous photoelectric measurement system further includes a power supply module, which is connected to the controller, the camera and the laser tracker and supplies power to the controller, the camera and the laser tracker.

3. The method for autonomous photoelectric measurement of three-dimensional feature points in a large space according to claim 1, characterized in that: The camera is installed on a turntable, and the turntable is mounted on an AGV vehicle.

4. The method for autonomous photoelectric measurement of three-dimensional feature points in a large space according to claim 1, characterized in that: In step 3, the steps of spatial key point image acquisition, processing and center point extraction include: 31) The AGV moves according to the preset path planning. During the movement, the turntable drives the camera to rotate. The camera's field of view covers different areas in the large space. The camera captures the key point images in the space in real time and transmits them to the controller; 32) The controller uses deep learning technology to extract features and identify key points of the image data collected by the camera to obtain the coordinates of the key points; 33) The center of the circular light spot is accurately extracted using the Hessian matrix method.

5. The method for autonomous photoelectric measurement of three-dimensional feature points in a large space according to claim 4, characterized in that: In step 3), the step of accurately extracting the center of the circular light spot by using the Hessian matrix method includes: 331) First, the original image is preprocessed, including grayscale conversion and noise removal, to improve the accuracy of subsequent processing; 332) For each pixel in the image, the Hessian matrix is ​​constructed by calculating the second-order partial derivative of its gray value; the determinant of the Hessian matrix is ​​used to detect feature points. For a circular light spot, a specific pattern will be shown in the eigenvalue analysis of the Hessian matrix. The position of the center of the light spot can be determined by finding the maximum value point of the determinant. In order to further improve the extraction accuracy of the center point, sub-pixel level refinement is performed near the center point detected by the Hessian matrix; Through the above steps, the Hessian matrix method can accurately extract the center of the circular light spot and maintain high stability and accuracy even in the presence of noise interference or undesirable lighting conditions.

6. The method for autonomous photoelectric measurement of three-dimensional feature points in a large space according to claim 1, characterized in that: In step 4, the steps of guiding the laser tracker to measure key points in space include: The PID algorithm is used to guide the laser tracker to adjust its pitch and yaw angles, so that the center of the feature point recognized by vision coincides with the center of the light spot projected by the laser tracker, thereby realizing automatic measurement of key points in space.

7. The method for autonomous photoelectric measurement of three-dimensional feature points in a large space according to claim 6, characterized in that: The steps to guide the laser tracker to adjust its pitch and yaw angles through the PID algorithm include: 411) respectively extracting the coordinates of the center of the feature point and the center of the light spot projected by the laser tracker; 412) There is a positional deviation between the center of the feature point and the center of the light spot projected by the laser tracker, which can be determined by calculating the coordinate difference between the two, that is, the error; the calculated error is used as the input of the controller to guide the adjustment of the laser tracker; 413) The calculated control quantity is applied to the laser tracker to adjust its pitch and yaw angles so that the center of the light spot moves toward the center of the feature point. The controller continuously monitors the position deviation between the center of the light spot and the center of the feature point, and adjusts the laser tracker in real time until the two coincide, thereby achieving dynamic stable tracking and automatic measurement of key points in space.

8. An autonomous photoelectric measurement system for three-dimensional feature points in a large space, characterized in that: It includes a controller, a camera, a laser tracker, a power supply module, an AGV trolley and a turntable mounted thereon, wherein the camera is installed on the turntable, the laser tracker is mounted on the AGV trolley, and the AGV trolley can move according to a preset path planning. The power supply module is connected to the controller, the camera, and the laser tracker and supplies power to the controller, the camera, and the laser tracker. The controller is connected to the camera and the laser tracker. The camera captures images of key points in the space in real time and transmits them to the controller for analysis and processing. The controller controls the laser tracker to automatically aim and accurately measure key points in the space according to the position information provided by the camera.

9. An electronic device, characterized in that: include: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the autonomous photoelectric measurement method of three-dimensional feature points in a large space as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for autonomous photoelectric measurement of three-dimensional feature points in a large space as described in any one of claims 1 to 7 is implemented.

Images