Wall surface marking device and method based on visual tracking

Abstract

The invention discloses a wall surface marking device and method based on visual tracking. The calibration plate is used for calibrating the monitoring camera to obtain a corresponding relation between a camera calibration coefficient and a space coordinate; the obtained calibration result is stored in a control system; the control system is used for controlling the monitoring camera to complete automatic focusing of the probe; images shot by the monitoring camera are collected, image recognition is carried out, the z coordinate position of the probe is calculated according to the calibration result, and the z coordinate is the position of the wall face where the probe is located; controlling a sliding table to move according to the z coordinate position; controlling a sheet light source mounted on the sliding table to work; the sheet light source is used for emitting sheet light to the wall surface and generating laser rays to mark the position of the wall surface; the monitoring camera is used for monitoring the position of the probe and sending a shot probe image and a shot probe and laser ray image to the control system; and the control system performs image recognition according to the probe and the laser line image to calculate the height distance between the probe and the laser line, and moves the probe to a target position according to the height distance.

Description

Technical Field

[0001] The invention develops a wall marking device and method based on visual tracking, belonging to the field of measurement and control technology. Background Art

[0002] The accurate positioning of the probe near the wall is of great significance. For example, in the field of fluid measurement, due to the large velocity gradient of the fluid near the wall, when using a hot wire anemometer to measure the movement velocity of the fluid near the wall, the distance between the hot wire and the wall needs to be known in order to accurately obtain the flow state near the wall; in non-destructive testing of objects, accurate positioning can ensure that the probe can be tested without damaging the structure, which is of great significance for protecting the integrity of the structure and extending its service life. To ensure the accurate positioning of the probe near the wall, the relative position between the probe and the wall needs to be obtained. In the actual environment, the wall often has a complex three-dimensional morphology, such as curved surfaces, inclined surfaces, concave and convex surfaces, etc. The local wall position of the probe needs to be determined in real time to complete the positioning of the probe, which increases the difficulty of positioning the probe near the wall. The determination of the wall position is the key to the accurate positioning of the probe. Summary of the invention

[0003] The technical problem solved by the present invention is to overcome the deficiencies of the prior art and provide a wall marking device and method based on visual tracking.

[0004] The technical solution of the present invention is: a wall marking device based on visual tracking, including: a monitoring camera, a light source, a slide, a calibration plate, and a control system;

[0005] The calibration board is used to calibrate the monitoring camera to obtain the corresponding relationship between the camera calibration coefficient and the spatial coordinates; the obtained calibration results are stored in the control system;

[0006] The control system is used to control the monitoring camera to complete the automatic focusing of the probe; collect the images taken by the monitoring camera, perform image recognition, calculate the z coordinate position of the probe according to the calibration results, and control the slide to move according to the z coordinate position; control the light source installed on the slide to work; define the z axis as the camera axis direction, the x axis as the horizontal direction of the camera image, and the y axis as the vertical direction of the camera image, where the x and y coordinates are the image pixel coordinates, and the z coordinate is the actual position coordinate;

[0007] The sheet light source is used to emit a sheet of light to the wall surface to generate a laser line to mark the position of the wall surface;

[0008] The monitoring camera is used to monitor the position of the probe and send the captured probe image and probe and laser line image to the control system; the control system performs image recognition based on the probe and laser line images to calculate the height distance between the probe and the laser line, and moves the probe to the target position based on the height distance.

[0009] Preferably, a plurality of circular calibration points are arranged in an array on the calibration plate.

[0010] Preferably, the distance between the centers of adjacent calibration points is maintained at 1.2-1.8 times the diameter of the calibration circle. To ensure calibration accuracy, the number of calibration points should be as large as possible, with a lower limit of 16, while ensuring that the image size of the calibration circle diameter is greater than 20 pixels. The distribution of calibration points should traverse the entire camera image.

[0011] Preferably, the calibration plate should be perpendicular to the camera axis, and the center of the calibration plate should be located on the camera axis.

[0012] Preferably, the camera is calibrated by:

[0013] Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z K ), and place the calibration plate at (z1,z2,…,z K ) position, keep the focal length of the camera unchanged during the calibration process, and refocus the camera every time the calibration plate is moved in the z direction to ensure that the calibration plate is clear. Use the monitoring camera to image the calibration plate at each calibration position;

[0014] Calculate the center of each circular calibration point (x i ,y j ,z k ) is the circle diameter pixel size on the monitoring camera, where i = 1, 2, ..., I; j = 1, 2, ..., J; k = 1, 2, ..., K; combined with the actual diameter size of the circular calibration point, the center of each circular calibration point (x i ,y j ,z k ) corresponds to the camera calibration coefficient c (mm / pixel);

[0015] The camera calibration coefficient c(x, y, z) at any point (x, y, z) in space is obtained through linear interpolation; then, given the camera calibration coefficient c and coordinates x and y, the z coordinate position is obtained, that is, z = g(x, y, c).

[0016] A marking method implemented by the wall marking device based on visual tracking comprises:

[0017] Use the calibration board to calibrate the monitoring camera and obtain the corresponding relationship between the camera calibration coefficient and the spatial coordinates;

[0018] Install the monitoring camera, light source sheet, and slide stage described in claim 1, ensure that the focal length of the monitoring camera is the same as that during calibration, and that the laser surface generated by the light source sheet is perpendicular to the axis of the monitoring camera; ensure that the axis direction of the probe is in the y-axis direction;

[0019] Move the probe to the position to be operated within the camera's field of view, and the monitoring camera automatically focuses on the probe position to make the image clear and take a photo;

[0020] According to the captured image, the characteristic dimension of the probe is selected to calculate the calibration coefficient c0 of the local position (x0, y0) of the characteristic dimension, and then the z-position of the probe z0=g(x0, y0, c0) is calculated according to the calibration result;

[0021] The light sheet is moved to the z0 position using a slide, and the laser line generated by the light sheet marks the wall position at the probe, and an image is formed using a monitoring camera;

[0022] The height distance between the probe tip and the laser line is calculated based on the image with the laser line taken by the monitoring camera, and the probe is adjusted to the target position according to the height distance to complete the operation.

[0023] Preferably, the camera is calibrated by:

[0024] Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z K ), and place the calibration plate at (z1,z2,…,z K ) position, keep the focal length of the camera unchanged during the calibration process, and refocus the camera every time the calibration plate is moved in the z direction to ensure that the calibration plate is clear. Use the monitoring camera to image the calibration plate at each calibration position;

[0025] Calculate the center of each circular calibration point (x i ,y j ,z k ) is the circle diameter pixel size on the monitoring camera, where i = 1, 2, ..., I; j = 1, 2, ..., J; k = 1, 2, ..., K; combined with the actual diameter size of the circular calibration point, the center of each circular calibration point (x i ,y j ,z k ) corresponds to the camera calibration coefficient c (mm / pixel);

[0026] The camera calibration coefficient c (mm / pixel) at any point in space is obtained through linear interpolation; then, given the camera calibration coefficient c and coordinates x and y, the z coordinate position is obtained, that is, z = g(x, y, c).

[0027] Preferably, the probe includes a hot wire anemometer and a cutting tool.

[0028] Preferably, the calibration coefficient c0 of the local position (x0, y0) of the characteristic size is the actual characteristic size of the probe divided by the pixel size of the characteristic size of the probe on the image.

[0029] Preferably, when calculating the height distance d between the probe tip and the laser line, the following formula is used for calculation: y1 is the y coordinate of the intersection of the probe axis and the laser line, y2 is the y coordinate of the probe tip, c(x2,y,z0) is the camera calibration coefficient, x2 is the x coordinate of the probe tip, and z0 is the z coordinate of the plane where the camera image is located.

[0030] The beneficial effects of the present invention compared with the prior art are:

[0031] When using existing image recognition technology to identify the position of complex wall surfaces, there are problems such as unclear wall image imaging and low resolution, which affects the recognition accuracy of the wall position. The present invention can accurately track the wall surface at the local position of the probe, mark the wall surface with a laser line, accurately obtain the distance between the probe and the wall surface, and achieve precise positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of a wall marking device based on visual tracking provided by an embodiment of the present invention

[0033] Figure 2 This is a schematic diagram of the monitoring camera calibration provided by an embodiment of the present invention.

[0034] In the figure:

[0035] 1. Monitoring camera, 2. Light source (2-1 laser line), 3. Slide table, 4. Probe, 5. Calibration plate DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the embodiments.

[0037] The basic idea of ​​the present invention is to use a sheet light source to generate a laser bright line on the local wall surface of the probe to mark the wall surface position. The key point in the marking process is to ensure that the laser line coincides with the local wall surface of the probe as much as possible. Figure 1 With attached Figure 2 The present invention uses a monitoring camera to monitor the probe position, uses image recognition to calculate the camera focal plane position (i.e. the probe's z coordinate), and then uses a slide to move the light source so that the laser line is located in the camera focal plane to complete the wall marking. Figure 1 With attached Figure 2 , the device mainly consists of the following parts.

[0038] 1) Monitoring camera

[0039] Monitor the probe position. Send the captured probe image and probe and laser line image to the control system.

[0040] 2) Sheet light source

[0041] A sheet of light is emitted to the wall, generating a laser line to mark the position of the wall.

[0042] 3)Slide

[0043] The light sheet is enabled to move along the z direction, and is moved to the z coordinate position where the probe is located.

[0044] 4) Calibration board

[0045] Used to calibrate monitoring cameras and obtain the correspondence between camera calibration coefficients and spatial coordinates. Multiple circular calibration points are arranged in an array on the calibration board, and the distance between the centers of adjacent calibration points is kept at about 1.5 times the diameter of the calibration circle. To ensure the calibration accuracy, the number of calibration points should be as large as possible (with a lower limit of 16) while ensuring that the image size of the calibration circle diameter is greater than 20 pixels, and the distribution of calibration points should traverse the entire camera image.

[0046] 5) Control system

[0047] a) used to control the monitoring camera to complete the automatic focus of the probe; b) collect the images taken by the monitoring camera, perform image recognition, and calculate the z coordinate position of the probe according to the calibration results, wherein the z coordinate is the position of the wall where the probe is located; c) control the movement of the slide according to the z coordinate position; d) control the operation of the light source installed on the slide; e) perform image recognition based on the image of the probe and the laser line to calculate the height distance between the probe and the laser line.

[0048] The implementation process of the present invention is described in detail with reference to the attached Figure 1 , 2 , the better specific implementation steps are:

[0049] 1) Calibrate the camera, such as Figure 2 As shown, set the coordinates of the calibration points on the z-axis (z1, z2, ..., z K ), and place the calibration plate at (z1,z2,…,z K ) position. Keep the focal length of the camera unchanged during the calibration process. Each time the calibration plate is moved in the z direction, the camera needs to be refocused to ensure that the calibration plate is clear.

[0050] Calculate each circular calibration point (x i ,y j ,z k )(where i=1,2,…,I;j=1,2,…,J;k=1,2,…,K) is the circle diameter pixel size on the camera. The camera calibration coefficient c(mm / pixel) at any point in space can be obtained by linear interpolation, that is, c=f(x,y,z). Similarly, given the camera calibration coefficient c and coordinates x and y, the z coordinate position can be obtained, that is, z=g(x,y,c).

[0051] Define the z-axis as the camera axis direction, the x-axis as the horizontal direction of the camera image, and the y-axis as the vertical direction of the camera image, wherein the x and y coordinates are the image pixel coordinates, and the z coordinate is the actual position coordinate; the origin of the coordinate axis of the present invention is set at the vertex of the camera head (can also be given artificially).

[0052] 2) Install the test system, such as Figure 1 As shown, it includes a monitoring camera, a light sheet, and a slide table, which ensures that the focal length of the camera is the same as that during calibration and that the laser surface generated by the light sheet is perpendicular to the camera axis.

[0053] 3) Move the probe (such as hot wire anemometer, cutting tool, etc.) to the position to be operated within the camera's field of view, such as Figure 1 The monitoring camera automatically focuses on the probe position to make the image clear.

[0054] 4) Select the characteristic size of the probe to calculate the calibration coefficient c0 of the characteristic size at the local position (x0, y0) (the actual characteristic size of the probe divided by the pixel size of the characteristic size of the probe on the image), and calculate the z-position of the probe

[0055] z0=g(x0,y0,c0).

[0056] 5) Move the light sheet to the z0 position. The generated laser line marks the wall position at the probe and is clearly imaged in the monitoring camera.

[0057] 6) Calculate the height distance between the probe and the laser line, and move the probe to the target height position according to the height distance to complete the task.

[0058] When calculating the height distance d between the probe tip and the laser line, use the following formula:

[0059] y1 is the y coordinate of the intersection of the probe axis and the laser line, y2 is the y coordinate of the probe tip, c(x2,y,z0) is the camera calibration coefficient, x2 is the x coordinate of the probe tip, and z0 is the z coordinate of the plane where the camera image is located.

[0060] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

[0061] Parts not described in detail in the present invention belong to common knowledge of those skilled in the art.

Claims

1. A wall marking device based on visual tracking, characterized in that include: Monitoring camera, light source, slide, calibration plate, control system; The calibration board is used to calibrate the monitoring camera to obtain the corresponding relationship between the camera calibration coefficient and the spatial coordinates; the obtained calibration results are stored in the control system; The control system is used to control the monitoring camera to complete the automatic focusing of the probe; collect the images taken by the monitoring camera, perform image recognition, calculate the z coordinate position of the probe according to the calibration results, and control the slide to move according to the z coordinate position; control the light source installed on the slide to work; define the z axis as the camera axis direction, the x axis as the horizontal direction of the camera image, and the y axis as the vertical direction of the camera image, where the x and y coordinates are the image pixel coordinates, and the z coordinate is the actual position coordinate; The sheet light source is used to emit a sheet of light to the wall surface to generate a laser line to mark the position of the wall surface; The monitoring camera is used to monitor the position of the probe and send the captured probe image and probe and laser line image to the control system; the control system performs image recognition based on the probe and laser line images to calculate the height distance between the probe and the laser line, and moves the probe to the target position based on the height distance.

2. The device according to claim 1, characterized in that: A plurality of circular calibration points are arranged in an array on the calibration plate.

3. The device according to claim 2, characterized in that: The distance between the centers of adjacent calibration points is kept at 1.2-1.8 times the diameter of the calibration circle. To ensure the calibration accuracy, the number of calibration points should be as large as possible, with a lower limit of 16, while ensuring that the image size of the calibration circle diameter is greater than 20 pixels. The distribution of the calibration points should traverse the entire camera image.

4. The device according to claim 1, characterized in that: The calibration plate should be perpendicular to the camera axis, and the center of the calibration plate should be located on the camera axis.

5. The device according to claim 2, characterized in that: The camera is calibrated in the following way: Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z K ), and place the calibration plate at (z1,z2,…,z K ) position, keep the focal length of the camera unchanged during the calibration process, and refocus the camera every time the calibration plate is moved in the z direction to ensure that the calibration plate is clear. Use the monitoring camera to image the calibration plate at each calibration position; Calculate the center of each circular calibration point (x i ,y j ,z k ) is the circle diameter pixel size on the monitoring camera, where i = 1, 2, ..., I; j = 1, 2, ..., J; k = 1, 2, ..., K; combined with the actual diameter size of the circular calibration point, the center of each circular calibration point (x i ,y j ,z k ) corresponds to the camera calibration coefficient c (mm / pixel); The camera calibration coefficient c(x, y, z) at any point (x, y, z) in space is obtained through linear interpolation; then, given the camera calibration coefficient c and coordinates x and y, the z coordinate position is obtained, that is, z = g(x, y, c).

6. A marking method implemented by the wall marking device based on visual tracking as claimed in claim 1, characterized in that include: Use the calibration board to calibrate the monitoring camera and obtain the corresponding relationship between the camera calibration coefficient and the spatial coordinates; Install the monitoring camera, light source sheet, and slide stage described in claim 1, ensure that the focal length of the monitoring camera is the same as that during calibration, and that the laser surface generated by the light source sheet is perpendicular to the axis of the monitoring camera; ensure that the axis direction of the probe is in the y-axis direction; Move the probe to the position to be operated within the camera's field of view, and the monitoring camera automatically focuses on the probe position to make the image clear and take a photo; According to the captured image, the characteristic dimension of the probe is selected to calculate the calibration coefficient c0 of the local position (x0, y0) of the characteristic dimension, and then the z-position of the probe z0=g(x0, y0, c0) is calculated according to the calibration result; The light sheet is moved to the z0 position using a slide, and the laser line generated by the light sheet marks the wall position at the probe, and an image is formed using a monitoring camera; The height distance between the probe tip and the laser line is calculated based on the image with the laser line taken by the monitoring camera, and the probe is adjusted to the target position according to the height distance to complete the operation.

7. The method according to claim 6, characterized in that: The camera is calibrated in the following way: Set the coordinates of the calibration points on the z-axis (z1, z2, ..., z K ), and place the calibration plate at (z1,z2,…,z K ) position, keep the focal length of the camera unchanged during the calibration process, and refocus the camera every time the calibration plate is moved in the z direction to ensure that the calibration plate is clear. Use the monitoring camera to image the calibration plate at each calibration position; Calculate the center of each circular calibration point (x i ,y j ,z k ) is the circle diameter pixel size on the monitoring camera, where i = 1, 2, ..., I; j = 1, 2, ..., J; k = 1, 2, ..., K; combined with the actual diameter size of the circular calibration point, the center of each circular calibration point (x i ,y j ,z k ) corresponds to the camera calibration coefficient c (mm / pixel); The camera calibration coefficient c (mm / pixel) at any point in space is obtained through linear interpolation; then, given the camera calibration coefficient c and coordinates x and y, the z coordinate position is obtained, that is, z = g(x, y, c).

8. The method according to claim 6, characterized in that: The probe comprises a hot wire anemometer and a cutting tool.

9. The method according to claim 8, characterized in that: Feature size The calibration coefficient c0 at the local position (x0, y0) is the actual feature size of the probe divided by the pixel size of the probe feature size on the image.

10. The method according to claim 6, characterized in that: When calculating the height distance d between the probe tip and the laser line, use the following formula: y1 is the y coordinate of the intersection of the probe axis and the laser line, y2 is the y coordinate of the probe tip, c(x2,y,z0) is the camera calibration coefficient, x2 is the x coordinate of the probe tip, and z0 is the z coordinate of the plane where the camera image is located.

Images