Wall marking device and method based on visual tracking

Abstract

A wall marking device and method based on visual tracking. The calibration board is used to calibrate the monitoring camera to obtain the corresponding relationship between the camera calibration coefficient and the spatial coordinates; the obtained calibration result is stored to the control system; the control system is used to control the monitoring camera to complete the automatic focusing of the probe; the image captured by the monitoring camera is collected and image recognition is performed, the z coordinate position of the probe is calculated according to the calibration result, and the z coordinate is the position of the wall surface where the probe is located; the z coordinate position is used to control the movement of the sliding table; the sheet light source installed on the sliding table is controlled to work; the sheet light source is used to emit sheet light to the wall surface to generate a laser line to mark the wall surface position; the monitoring camera is used to monitor the probe position, and the captured probe image, probe and laser line image are sent to the control system; the control system calculates the height distance between the probe and the laser line according to the probe and the laser line image, and moves the probe to the target position according to the height distance.

Description

Technical Field

[0001] This invention develops a wall marking device and method based on visual tracking, belonging to the field of measurement and control technology. Background Technology

[0002] Accurate probe positioning near walls is crucial. For example, in fluid measurement, due to the significant velocity gradient of fluids near walls, when using a hot-wire anemometer to measure the velocity of fluid near a wall, the distance between the hot wire and the wall must be known to accurately obtain the near-wall flow state. In non-destructive testing of objects, accurate positioning ensures that the probe can perform tests without damaging the structure, which is essential for protecting the integrity of the structure and extending its service life. Ensuring accurate probe positioning near walls requires obtaining the relative position between the probe and the wall. In real-world environments, walls often have complex three-dimensional shapes, such as curved surfaces, inclined surfaces, and uneven surfaces. The local wall position of the probe must be determined in real time to complete the probe positioning, which increases the difficulty of positioning the probe near the wall. Determining the wall position is key to accurate probe positioning. Summary of the Invention

[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a wall marking device and method based on vision tracking.

[0004] The solution of the present invention is: a wall marking device based on visual tracking, comprising: a monitoring camera, a sheet light source, a slide, a calibration plate, and a control system;

[0005] The calibration board is used to calibrate the monitoring camera and obtain the correspondence between the camera calibration coefficients and spatial coordinates; the obtained calibration results are stored in the control system.

[0006] The control system is used to control the monitoring camera to automatically focus on the probe; acquire images captured by the monitoring camera and perform image recognition; calculate the z-coordinate position of the probe based on the calibration results; control the slide to move according to the z-coordinate position; control the sheet light source mounted on the slide to work; define the z-axis as the direction of the camera axis, the x-axis as the horizontal direction of the camera image, and the y-axis as the vertical direction of the camera image, where the x and y coordinates are the image pixel coordinates, and the z-coordinate is the actual position coordinate;

[0007] A sheet light source is used to emit sheet light onto the wall surface to generate laser lines, which are then used to mark the wall surface positions.

[0008] The monitoring camera is used to monitor the position of the probe and sends the captured images of the probe and the laser line to the control system. The control system performs image recognition based on the images of the probe and the laser line to calculate the height distance between the probe and the laser line, and moves the probe to the target position based on the height distance.

[0009] Preferably, the calibration plate is arrayed with multiple circular calibration points.

[0010] Preferably, the distance between the centers of adjacent calibration points is maintained at 1.2-1.8 times the diameter of the calibration circle. To ensure calibration accuracy, the number of calibration points should be as large as possible, with a minimum of 16, provided that the image size of the calibration circle diameter is greater than 20 pixels. The distribution of calibration points should cover the entire camera image.

[0011] Preferably, the calibration plate should be perpendicular to the camera axis, and the center of the calibration plate should be located on the camera axis.

[0012] Preferably, the camera is calibrated in the following manner:

[0013] Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z...). K Place the calibration plate sequentially on (z1, z2, ..., z). K During the calibration process, the camera's focal length remains constant. Each time the calibration board is moved in the z-direction, the camera needs to be refocused to ensure the calibration board is clear. At each calibration position, the monitoring camera is used to image the calibration board.

[0014] Calculate the center (x) of each circular calibration point i ,y j ,z k The pixel dimensions of the circle diameter on the monitoring camera, where i = 1, 2, ..., I; j = 1, 2, ..., J; k = 1, 2, ..., K; combined with the actual diameter dimensions of the circular calibration points, the center (x) of each circular calibration point is obtained. i ,y j ,z k The corresponding camera calibration factor c (mm / pixel);

[0015] The camera calibration coefficient c(x,y,z) at any point (x,y,z) in space is obtained by linear interpolation; then, given the camera calibration coefficient c and the coordinates x and y, the z coordinate position is obtained, i.e., z = g(x,y,c).

[0016] A marking method implemented by the vision-tracking-based wall marking device includes:

[0017] The monitoring camera was calibrated using a calibration board to obtain the correspondence between the camera calibration coefficients and spatial coordinates;

[0018] Install the monitoring camera, sheet light source, and slide table as described in claim 1, ensuring that the focal length of the monitoring camera is the same as that during calibration, and that the laser surface generated by the sheet light source is perpendicular to the axis of the monitoring camera; ensure that the axial direction of the probe is located in the y-axis direction;

[0019] Move the probe to the position to be worked within the camera's field of view, monitor the camera to automatically focus on the probe position to make the image clear, and take a picture;

[0020] Based on the captured image, select the characteristic size of the probe and calculate the calibration coefficient c0 of the local position (x0, y0) of the characteristic size. Then, calculate the probe's z-axis position z0 = g(x0, y0, c0) based on the calibration result.

[0021] The slide table is used to move the sheet light source to position z0. The laser line generated by the sheet light source marks the wall position at the probe, and the monitoring camera is used to image it.

[0022] The height distance between the probe tip and the laser line is calculated based on the image with laser line captured by the monitoring camera. The probe is then adjusted to the target position based on this height distance to complete the operation.

[0023] Preferably, the camera is calibrated in the following manner:

[0024] Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z...). K Place the calibration plate sequentially on (z1, z2, ..., z). K During the calibration process, the camera's focal length remains constant. Each time the calibration board is moved in the z-direction, the camera needs to be refocused to ensure the calibration board is clear. At each calibration position, the monitoring camera is used to image the calibration board.

[0025] Calculate the center (x) of each circular calibration point i ,y j ,z k The pixel dimensions of the circle diameter on the monitoring camera, where i = 1, 2, ..., I; j = 1, 2, ..., J; k = 1, 2, ..., K; combined with the actual diameter dimensions of the circular calibration points, the center (x) of each circular calibration point is obtained. i ,y j ,z k The corresponding camera calibration factor c (mm / pixel);

[0026] The camera calibration coefficient c (mm / pixel) at any point in space is obtained by linear interpolation; then, given the camera calibration coefficient c and coordinates x and y, the z coordinate position is obtained, i.e., z = g(x, y, c).

[0027] Preferably, the probe includes a hot-wire anemometer and a cutting tool.

[0028] Preferably, the calibration coefficient c0 of the local location (x0, y0) of the feature size is the actual feature size of the probe divided by the pixel size of the probe feature size in the image.

[0029] Preferably, the following formula is used to calculate the height distance d between the probe tip and the laser line: y1 is the y coordinate of the intersection of the probe axis and the laser line, y2 is the y coordinate of the probe tip, c(x2,y,z0) is the camera calibration coefficient, x2 is the x coordinate of the probe tip, and z0 is the z coordinate of the plane where the camera image is located.

[0030] The beneficial effects of this invention compared to the prior art are:

[0031] When using existing image recognition technologies to identify the location of complex-shaped walls, there are problems such as unclear wall images and low resolution, which affect the accuracy of wall location identification. This invention can accurately track the wall at the local position of the probe, and use laser lines to mark the wall to accurately obtain the distance between the probe and the wall, thus achieving precise positioning. Attached Figure Description

[0032] Figure 1 A schematic diagram of a vision-tracking-based wall marking device provided in an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the monitoring camera calibration provided in an embodiment of the present invention.

[0034] In the picture:

[0035] 1. Monitoring camera; 2. Sheet light source (2-1 laser line); 3. Slide stage; 4. Probe; 5. Calibration plate. Detailed Implementation

[0036] The present invention will be further described below with reference to the embodiments.

[0037] The basic idea of ​​this invention is to use a sheet light source to generate a bright laser line on the local wall surface of the probe to mark the wall position. The key to the marking process is to ensure that the laser line coincides as closely as possible with the local wall surface of the probe. (See attached diagram.) Figure 1 With appendix Figure 2 This invention utilizes a monitoring camera to monitor the probe's position, calculates the camera's focal plane position (i.e., the probe's z-coordinate) using image recognition methods, and then uses a sliding table to move the light source, positioning the laser line within the camera's focal plane to complete the wall marking. (See attached document.) Figure 1 With appendix Figure 2 The device mainly consists of the following parts.

[0038] 1) Monitoring camera

[0039] Monitor probe position. Send the captured probe images and probe and laser line images to the control system.

[0040] 2) Sheet light source

[0041] The light emitted from the plate is directed onto the wall surface, generating laser lines to mark the wall's position.

[0042] 3)Slide

[0043] This allows the sheet light source to move along the z-direction, moving it to the z-coordinate position where the probe is located.

[0044] 4) Calibration plate

[0045] This is used to calibrate monitoring cameras and obtain the correspondence between camera calibration coefficients and spatial coordinates. Multiple circular calibration points are arrayed on the calibration plate, with the center-to-center distance between adjacent calibration points maintained at approximately 1.5 times the diameter of the calibration circle. To ensure calibration accuracy, the number of calibration points should be as large as possible (minimum 16), provided that the image size of the calibration circle diameter is greater than 20 pixels, and the distribution of calibration points should cover the entire camera image.

[0046] 5) Control System

[0047] a) Used to control the monitoring camera to automatically focus on the probe; b) Acquire images captured by the monitoring camera, perform image recognition, and calculate the z-coordinate position of the probe based on the calibration results, wherein the z-coordinate is the position of the probe on the wall surface; c) Control the slide to move according to the z-coordinate position; d) Control the sheet light source mounted on the slide to work; e) Calculate the height distance between the probe and the laser line based on image recognition of the probe and laser line images.

[0048] The implementation process of this invention is described in detail below, with reference to the appendix. Figure 1 , 2 The preferred specific implementation steps are as follows:

[0049] 1) Calibrate the camera, such as... Figure 2 As shown, set the coordinates of the calibration points on the z-axis (z1, z2, ..., z...). K Place the calibration plate sequentially on (z1, z2, ..., z). K Position. During the calibration process, keep the camera's focal length constant. Each time the calibration board is moved in the z-direction, the camera needs to be refocused to ensure the calibration board is sharp.

[0050] Calculate each circular calibration point (x) i ,y j ,z k Let f(x,y,z) be the diameter (in pixels) of the circle on the camera (where i = 1,2,…,I; j = 1,2,…,J; k = 1,2,…,K). The camera calibration coefficient c (mm / pixel) at any point in space can be obtained through linear interpolation, i.e., c = f(x,y,z). Similarly, given the camera calibration coefficient c and the coordinates x and y, the z-coordinate can be obtained, i.e., z = g(x,y,c).

[0051] The z-axis is defined as the direction of the camera axis, the x-axis as the horizontal direction of the camera image, and the y-axis as the vertical direction of the camera image. The x and y coordinates are the image pixel coordinates, and the z coordinate is the actual position coordinate. The origin of the coordinate axis in this invention is set at the top of the camera head (or can be given manually).

[0052] 2) Install the test system, such as Figure 1 As shown, it includes a monitoring camera, a sheet light source, and a slide table to ensure that the camera focal length is the same as the calibration time and that the laser surface generated by the sheet light source is perpendicular to the camera axis.

[0053] 3) Move the probe (such as a hot-wire anemometer, cutting tool, etc.) to the position to be worked within the camera's field of view, such as... Figure 1 As shown. The monitoring camera automatically focuses on the probe position to ensure a clear image.

[0054] 4) Select the feature size of the probe, calculate the calibration coefficient c0 of the local position (x0, y0) of the feature size (the actual feature size of the probe divided by the pixel size of the probe feature size on the image), and calculate the position of the probe in the z-axis.

[0055] z0 = g(x0, y0, c0).

[0056] 5) Move the light source to position z0. The generated laser line marks the wall position at the probe and forms a clear image in the monitoring camera.

[0057] 6) Calculate the height distance between the probe and the laser line, and move the probe to the target height position according to the height distance to complete the task.

[0058] The following formula is used to calculate the height distance d between the probe tip and the laser line:

[0059] y1 is the y coordinate of the intersection of the probe axis and the laser line, y2 is the y coordinate of the probe tip, c(x2,y,z0) is the camera calibration coefficient, x2 is the x coordinate of the probe tip, and z0 is the z coordinate of the plane where the camera image is located.

[0060] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0061] The parts of this invention not described in detail are common knowledge to those skilled in the art.

Claims

1. A wall marking device based on visual tracking, characterized by The utility model relates to a kind of automatic focusing system of probe, comprising: Monitoring camera, sheet light source, sliding table, calibration board, control system; Calibration board is used to calibrate monitoring camera, obtain the corresponding relationship between camera calibration coefficient and space coordinates;The calibration result obtained is stored to control system; Control system is used to control monitoring camera to complete the automatic focusing of probe;Image recognition is carried out to the image captured by monitoring camera, and the z coordinate position of probe is calculated according to the calibration result, and the sliding table is controlled to move according to the z coordinate position;Control sheet light source installed on sliding table to work;Define z axis as the direction of camera axis, x axis as the horizontal direction of camera image, y axis as the vertical direction of camera image, wherein x and y coordinates are image pixel coordinates, and z coordinate is actual position coordinate; Sheet light source is used to emit sheet light to wall surface, generate laser line, to mark wall surface position; Monitoring camera is used to monitor probe position, and the image of probe, probe and laser line image photographed are sent to control system;Control system carries out image recognition according to probe and laser line image, calculates the height distance between probe and laser line, and moves probe to target position according to height distance, to complete work.

2. The apparatus of claim 1, wherein: A plurality of circular calibration points are arranged on the calibration board.

3. The apparatus of claim 2, wherein: The distance between the centers of adjacent calibration points is kept at 1.2-1.8 times the diameter of the calibration circle. To ensure calibration accuracy, the image size of the calibration circle should be greater than 20 pixels, and the number of calibration points should be as large as possible, with a lower limit of 16. The distribution of calibration points should cover the entire camera image.

4. The apparatus of claim 1, wherein: The calibration board should be perpendicular to the camera axis, and the center of the calibration board should be located on the camera axis.

5. The apparatus of claim 2, wherein: The camera is calibrated by the following method: Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z...). K Place the calibration plate sequentially on (z1, z2, ..., z). K During the calibration process, the camera's focal length remains constant. Each time the calibration board is moved in the z-direction, the camera needs to be refocused to ensure the calibration board is clear. At each calibration position, the monitoring camera is used to image the calibration board. Calculate the circle diameter pixel size of each circular calibration point center (x i ,y j ,z k ) on the monitoring camera, wherein i=1, 2,…,I; j=1, 2,…,J; k=1, 2,…,K; combined with the actual diameter size of the circular calibration point, the camera calibration coefficient c (mm / pixel) corresponding to each circular calibration point center (x i ,y j ,z k ) is obtained; The camera calibration coefficient c(x, y, z) at any point (x, y, z) in space is obtained by linear interpolation. Then, given the camera calibration coefficient c and the coordinates x and y, the z coordinate position is obtained, i.e., z = g(x, y, c).

6. A marking method using the wall marking device based on visual tracking according to claim 1, characterized in that The utility model relates to a kind of automatic focusing system of probe, comprising: Calibration board is used to calibrate monitoring camera, obtain the corresponding relationship between camera calibration coefficient and space coordinates; Install the monitoring camera, sheet light source and sliding table of claim 1, ensure that the focal length of monitoring camera is the same as during calibration, and the laser plane generated by sheet light source is perpendicular to the axis of monitoring camera;Ensure that the axis direction of probe is in y axis direction; Move probe to the position to be worked in the field of view of camera, and automatically focus monitoring camera to probe position to make image clear, and take a photo; According to the image photographed, select the feature size of probe to calculate the calibration coefficient c0 of the local position (x0, y0) of feature size, and then calculate the z direction position z0 of probe according to the calibration result, i.e., z0 = g(x0, y0, c0); Move sheet light source to z0 position using sliding table, and mark the wall surface position at probe using the laser line generated by sheet light source, and image using monitoring camera; Calculate the height distance between the tip of probe and laser line according to the image with laser line photographed by monitoring camera, and adjust probe to target position according to the height distance to complete work.

7. The method of claim 6, wherein: The camera is calibrated by the following method: The calibration point coordinates of the z-axis are set as (z1, z2, …, zN). K The calibration board is placed in the positions (z1, z2, …, zN) in sequence, the focal length of the camera is kept unchanged during the calibration process, the camera is refocused every time the calibration board is moved in the z direction, the calibration board is ensured to be clear, and the calibration board is imaged by using the monitoring camera at each calibration position. K The calibration point coordinates of the z-axis are set as (z1, z2, …, zN). K The calibration board is placed in the positions (z1, z2, …, zN) in sequence, the focal length of the camera is kept unchanged during the calibration process, the camera is refocused every time the calibration board is moved in the z direction, the calibration board is ensured to be clear, and the calibration board is imaged by using the monitoring camera Calculate the circle diameter pixel size of each circular calibration point center (x i ,y j ,z k ) on the monitoring camera, wherein i=1, 2,…,I; j=1, 2,…,J; k=1, 2,…,K; combined with the actual diameter size of the circular calibration point, the camera calibration coefficient c(mm / pixel) corresponding to each circular calibration point center (x i ,y j ,z k ) is obtained; The camera calibration coefficient c (mm / pixel) at any point in space is obtained by linear interpolation; then, the z coordinate position is obtained, i.e. z = g(x, y, c), with the known camera calibration coefficient c and the coordinates x and y.

8. The method of claim 6, wherein: The probe comprises a hot-wire anemometer or a cutting tool.

9. The method of claim 8, wherein: Characteristic dimension The calibration coefficient c0 of the local position (x0, y0) is the actual characteristic dimension of the probe divided by the pixel size of the characteristic dimension of the probe on the image.

10. The method of claim 6, wherein: In calculating the height distance d between the probe tip and the laser line, the following equation is used: y1 is the y coordinate of the intersection of the probe axis and the laser line, y2 is the y coordinate of the probe tip, c(x2, y, z0) is the camera calibration coefficient, x2 is the x coordinate of the probe tip, and z0 is the z coordinate of the plane in which the camera image lies.

Images