Positioning Method and System for a Vision-Automated Image Acquisition Cart Based on a High-Speed Camera

Abstract

The positioning method and system of an automated image acquisition trolley based on high-speed camera vision disclosed by the present invention include: installing positioning tags on the front and rear sides of the AGV trolley; setting a camera above the AGV trolley, calibrating the camera to obtain the mapping relationship between the pixel position and the actual position of the fiducial points in the images captured by the camera; using the camera to capture the environment to obtain images containing the positioning tags; performing binary processing on the images; analyzing the positioning tags in the binary images to identify the positions, angles, and contents of the positioning tags; according to the position and angle relationships between the center point of the trolley and the positioning tags and corresponding to the contents of the positioning tags, solving the relative position relationship between the center point of the trolley and the positioning tags; combining the mapping relationship calibrated by the camera and through coordinate transformation, obtaining the pose information of the trolley in the environment. The present invention realizes the positioning of the pose parameters of the trolley in the environment, and the amount of calculation in the recognition process is small, and both the positioning accuracy and speed can meet the application requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of AGV car positioning, and particularly to a positioning method and system for an automated image acquisition car based on high-speed camera vision. Background Art

[0002] An AGV (Automated Guided Vehicle), usually also referred to as an AGV car, refers to a transport vehicle equipped with an automatic navigation device that can travel along a specified navigation path and has safety protection and various load transfer functions.

[0003] According to the navigation method during the automatic driving process of the AGV, the AGV is divided into the following types: electromagnetic induction guided AGV, laser guided AGV, vision guided AGV, etc. Among them, the vision guided AGV is equipped with a camera and sensors. During the driving process of the AGV, the camera dynamically acquires image information of the vehicle surrounding environment and compares it with the image database, thereby determining the current position and making a decision on the next driving step. Since there is no requirement for manually setting any physical path, the vision guided AGV theoretically has the best guiding flexibility, and with the rapid development of computer image acquisition, storage, and processing technologies, its practicality is becoming stronger and stronger.

[0004] Currently, the common navigation method based on vision guided AGV is to set path or marker information on the road surface, such as pasting continuous identification lines as the path or setting a QR code marker area, etc. The camera captures the image of the marker information, and after the image is processed by digital image processing technology, the coordinate values of the feature points in the computer image coordinate system are obtained, and then the relationship between the computer image coordinate system and the world coordinate system is used for conversion to obtain the deviation of the AGV attitude, which is used as feedback to control the AGV. However, the identification of identification lines and QR code images is difficult, the entire vision analysis process is relatively cumbersome, the required energy consumption is high, and the calculation amount is large, and it is impossible to ensure the motion trajectory accuracy of the AGV car. Summary of the Invention

[0005] Object of the Invention: The object of the present invention is to provide a positioning method and system for an automated image acquisition car based on high-speed camera vision in view of the deficiencies of the prior art, to realize the positioning of the pose parameters of the car in the environment, and the amount of calculation in the recognition process is small, and both the positioning accuracy and speed can meet the requirements of AGV navigation in indoor acquisition.

[0006] Technical solution: The positioning method of the automatic image acquisition trolley based on high-speed camera vision of the present invention includes the following steps: S1. Design positioning labels: Install a total of 4 positioning labels with different contents on the front and rear sides of the AGV trolley; S2. Digitalize the environment: Set a camera above the AGV trolley, calibrate the camera, and obtain the mapping relationship between the pixel position and the actual position of the fiducial points in the image captured by the camera; S3. Obtain pictures: Use the camera to take pictures of the environment to obtain images containing positioning labels; S4. Binarize the pictures: Perform binarization processing on the images; S5. Identify the positioning labels: Analyze the positioning labels in the binarized image, and identify the position, angle, and content of the positioning labels; S6. Coordinate mapping: According to the position and angle relationship between the center point of the AGV trolley and the positioning labels, and corresponding to the content of the positioning labels, solve the relative position relationship between the center point of the AGV trolley and the positioning labels; S7. Locate the pose of the trolley: Combine the mapping relationship calibrated by the camera, and through coordinate transformation, obtain the pose information of the AGV trolley in the environment.

[0007] To further improve the above technical solution, it further includes S8. Trolley movement navigation: S81. Select a measured point in the environment, and determine the position information of the measured point through the mapping relationship calibrated by the camera; S82. According to step S7, obtain the pose information of the AGV trolley at the current position in the environment, including the orientation of the trolley and the center point of the trolley; S83. Calculate the distance and angle information between the center point of the AGV trolley at the current position and the center point of the AGV trolley at the target position, and the distance and angle information between the center point of the AGV trolley at the target position and the measured point; S84. Determine the path for the AGV trolley to move from the current position to the target position according to the information calculated in step S83.

[0008] Further, the calibration process of the camera in step S2 includes: The camera captures a standard calibration image; Based on the squares in the standard calibration image, evenly select fiducial points, and record the pixel positions and actual positions of each fiducial point in the standard calibration image; According to the pixel positions and actual positions of the fiducial points, establish a mapping relationship.

[0009] Further, in step S5, select the two positioning labels with the highest recognition degree for recognition calculation. If the calculation passes, the recognition is correct. If the calculation fails, select the third positioning label for recognition calculation. If the calculation of the third positioning label still fails, randomly rotate the angle and perform recognition again.

[0010] Further, according to the positions of the positioning labels on the AGV trolley, they are divided into right front, left front, left rear, and right rear. Each positioning label includes a square border area and a data area provided in the middle of the border area. The data area is a circular icon, and the colors of the border area and the data area are different; The circular icons of the right front, left front, left rear, and right rear are different in shape, and the colors of the border area and the data area are also different.

[0011] Further, the installation positions of the right front, left front, left rear, and right rear of the vehicle head on the AGV vehicle have the characteristics of translational and rotational invariance.

[0012] Further, the installation position of the camera above the AGV vehicle can ensure that the AGV vehicle can be photographed by the camera when moving in the environment, and the photographed image is a front view.

[0013] Further, the camera is an industrial camera with a full-frame exposure method, which has the functions of adjusting frame rate and exposure time. The camera lens is a wide-angle lens with a field of view angle of 135°.

[0014] A positioning system for an automated image acquisition vehicle based on a high-speed camera vision for implementing the above positioning method includes a host computer, an AGV vehicle, and a camera. The AGV vehicle moves in the environment, and the camera is set above the AGV vehicle to photograph the AGV vehicle in the environment. Both the AGV vehicle and the camera are connected to the host computer. Four positioning labels with different contents are installed at the front and rear positions of the AGV vehicle. The host computer is provided with a camera calibration program, a function module for calibrating the measured points, a function module for calibrating the pose of the vehicle, and a function module for calibrating the movement. The camera photographs and obtains pictures containing the positioning labels and transmits them to the host computer. The function module for calibrating the measured points is used to select the measured points and calibrate the measured points through the camera calibration program. The function module for calibrating the pose of the vehicle is used to identify the positioning labels, utilize the relative position relationship between the center point of the vehicle and the positioning labels, and calibrate the pose of the vehicle at the current position through the camera calibration program. The function module for calibrating the movement is used to set the target position, calculate the path information for the AGV vehicle to move from the current position to the target position, and send it to the AGV vehicle.

[0015] Further, the functions of the camera calibration program include: the camera photographs a standard calibration image; uniformly selects landmark points based on the squares in the standard calibration image, and records the pixel positions and actual positions of each landmark point in the standard calibration image; establishes a mapping relationship according to the pixel positions and actual positions of the landmark points. The function module for calibrating the pose of the vehicle includes identifying the positions, angles, and contents of the positioning labels, and solving the relative position relationship between the center point of the AGV vehicle and the positioning labels according to the position and angle relationships of the center point of the AGV vehicle between the positioning labels and the corresponding contents of the positioning labels. The function module for calibrating the movement includes calculating the distance and angle information between the center point of the AGV vehicle at the current position and the center point of the AGV vehicle at the target position, as well as the distance and angle information between the center point of the AGV vehicle at the target position and the measured point, and determining the path for the AGV vehicle to move from the current position to the target position according to the calculated information.

[0016] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows: Four positioning labels with different contents are arranged on the front and rear sides of the AGV cart. The four corners of the AGV cart are digitized by images. A high-speed camera is arranged above the AGV cart to obtain an image containing the positioning labels. By processing the image, the position, content, and deflection angle of the positioning labels are identified. According to the position and angle relationship between the center point of the cart and the positioning labels, relevant coordinate transformations are performed using the camera calibration relationship to convert them into the position and attitude information of the cart in the environment, realizing the positioning of the cart in the environment. Compared with the existing machine vision positioning methods, the recognition process of the present invention has a small amount of computation, and both the positioning accuracy and speed can meet the navigation requirements of the AGV in indoor collection. The high-speed camera works stably and has a low cost, and image processing is performed through the upper computer. This not only solves the problems of low efficiency of traditional indoor manual image collection, high human resources and costs, but also improves the positioning accuracy and speed of the recognition process of the entire positioning system. At the same time, it reflects the characteristics of a small amount of computation, high positioning accuracy, and fast running speed in the whole process.

[0017] The 4 positioning labels set on the AGV cart are clearly distinguished by the different shapes and colors of the border area and the data area, which is convenient for being quickly and safely captured by the camera and can be better recognized and positioned. The camera uses an industrial camera with a full-frame exposure method to obtain a larger image shooting range as much as possible, and the installation requirements for the camera ensure the shooting effect within the moving range of the cart. Brief Description of the Drawings

[0018] Figure 1 is the overall structural schematic diagram of the present invention;

[0019] Figure 2 is the left-side three-dimensional structural schematic diagram of the AGV cart in the present invention;

[0020] Figure 3 is the right-side three-dimensional structural schematic diagram of the AGV cart in the present invention;

[0021] Figure 4 is the front-side structural schematic diagram of the AGV cart in the present invention;

[0022] Figure 5 is the top-view structural schematic diagram of the AGV cart in the present invention;

[0023] Figure 6 is the structural schematic diagram of the positioning label of the AGV cart in the present invention;

[0024] Figure 7 is the working flow chart of the present invention;

[0025] Figure 8 is the image processing flow chart of the present invention;

[0026] Figure 9 This is a schematic diagram of the process of the AGV vehicle in the present invention moving from the current position to the target position. Detailed implementation manners

[0027] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the described embodiments.

[0028] The positioning system of the automated image acquisition vehicle based on high-speed camera vision provided by the present invention includes a host computer, an AGV vehicle (self-developed), and a Hikvision MVS camera. The model of the host computer is: Inter i7 2.90GHZ CPU + 32GB memory + 1TB hard disk + Windows 10 Professional Edition system. In order to ensure that the AGV vehicle can obtain clear images during movement, the camera uses an industrial camera with a full-frame exposure method to facilitate setting the frame rate and exposure time; in order to ensure that the image acquisition range is as large as possible, a wide-angle lens is used, and the field of view angle is about 135°. The wide-angle lens causes relatively large image distortion, so there are relatively high requirements for the installation of the camera and the vision pictures.

[0029] As Figure 1 shown in the installation schematic diagram, the upper part is the position and orientation of the high-speed camera, and the lower part is the operating area area of the AGV vehicle in the environment. A Hikvision MVS wide-angle camera is fixed above the AGV vehicle. When the lens of this camera is about 2.45m from the ground, the field of view range can reach 5m * 3.8m. In order to realize the effective movement of the AGV vehicle within a limited area, 4 positioning tags need to be installed on both the front and rear sides of the AGV vehicle to be quickly, safely and very clearly acquired by the upper camera, and then compared with the calibration icons in the host computer software.

[0030] As Figures 2 to 5 shown, a total of 4 positioning tags are arranged on both the front and rear sides of the AGV vehicle for positioning. The positioning tags are inlaid on a square with a size of 56mm * 56mm. The 4 positioning tags are divided into: the first on the right side of the vehicle head, the second on the left side of the vehicle head, the third on the left side of the vehicle tail, and the fourth on the right side of the vehicle tail according to their positions on the AGV vehicle.

[0031] As Figure 6 shown, 4 different pictures are used for the 4 positioning tags to facilitate machine vision recognition. The positioning tags are divided into two areas: a border area and a data area. The colors of the outer border area and the central data area need to be distinguished. The data area is mainly composed of circular icons, and it is ensured that the colors of all icons are different, so that machine vision can better identify and locate under the camera.

[0032] Digitize the indoor environment by designing positioning tags on both the front and rear sides of the AGV cart. A high-speed camera captures an image containing 4 positioning tags, and a certain algorithm is used to process the image and identify the position, orientation, and content of the positioning tags. Based on the position and angular relationship of the center point of the AGV cart between the positioning tags, combined with the calibration relationship table of the high-speed camera and relevant coordinate transformations, obtain the position and attitude information of the object to be tested in the environment, and achieve the positioning of the cart in the indoor environment.

[0033] Camera installation: To ensure the normal positioning of the AGV cart, there are certain requirements for the installation of the camera. The camera needs to be installed above the AGV cart. It is best that the installation position enables the cart to be photographed within the driving range. That is to say, it is best that the farthest position where the cart is expected to travel is close to the center point of the camera, and the displayed pictures need to be parallel with basically no inclination. The scene displayed under the camera lens needs to be a positive image, consistent with the scene seen by the human eye in real life.

[0034] Configuration: To better reflect the visual effect, the MVS application software of the Hikvision camera needs to be installed on the upper computer. The installation process of the Hikvision camera application software is relatively convenient and simple. After installation, open the MVS software and perform standard configuration.

[0035] Calibration: Perform calibration and positioning of the 4 positioning tags at the four corners of the AGV cart under machine vision. The calibration icons under vision should be as clear and distinct as possible. After the camera captures the 4 positioning tags at the four corners of the AGV cart, select two with the highest recognition rate for calculation. If the calculation passes, it is considered correctly recognized. If not, the third one will be selected. If it still fails, the angle will be randomly rotated and recognized again. Otherwise, the AGV cart cannot be accurately positioned. Therefore, the requirements for the placement of the overhead camera, the performance of the camera, and the clarity of the pictures at the four corners of the AGV are very high.

[0036] After the hardware installation is completed, camera calibration needs to be carried out through the upper computer software, and corresponding operations need to be performed by selecting the function modules for calibrating the measured points to be calibrated, calibrating the pose of the cart, and calibrating the movement function module.

[0037] The working process and the image processing process are as Figure 7 、 Figure 8 shown:

[0038] S1. Design positioning tags, and install a total of 4 positioning tags with different contents on both the front and rear sides of the AGV cart. The positioning tags have the characteristics of translational and rotational invariance, and can achieve good positioning effects in the surrounding environment;

[0039] S2: Digital Environment: Install a camera above the AGV cart, calibrate the camera, take a standard calibration image, uniformly select fiducial points based on the grids in the standard calibration image, and record the pixel positions and actual positions of each fiducial point in the standard calibration image; establish a mapping relationship according to the linear functional relationship between the pixel positions and actual positions of the fiducial points. The slope of this linear function will change according to different positions.

[0040] S3: Obtain Picture: Use the camera to take pictures of the surrounding environment to obtain an image containing positioning tags.

[0041] S4: Binarize Picture: Perform binarization processing on the image.

[0042] S5: Positioning Tag Recognition: Analyze the image to obtain the positions of the positioning tags in the image, and calculate the lengths and directions of the lines connecting different tags to the image.

[0043] S6: Coordinate Mapping: According to the position and angle relationships of the center point of the AGV cart between the positioning tags, and corresponding to the content of the positioning tags, solve the relative position relationship between the center point of the cart and the positioning tags.

[0044] S7: Cart Pose Positioning: Combine the mapping relationship obtained from camera calibration, and through coordinate transformation, obtain the pose of the center point of the cart in the actual environment.

[0045] As Figure 9 shown in the AGV cart movement navigation process, including:

[0046] S81: Select the measured point 5 in the environment, and determine the position information of the measured point through the mapping relationship obtained from camera calibration.

[0047] S82: According to the pose information of the AGV cart at the current position obtained in step S7, including the orientation of the cart and the center point of the cart.

[0048] S83: Calculate the distance L and included angle A information between the center point of the AGV cart at the current position and the center point of the AGV cart at the target position, as well as the distance and included angle B information between the center point of the AGV cart at the target position and the measured point.

[0049] S84: Determine the path for the AGV cart to move from the current position to the target position according to the information calculated in step S83.

[0050] The positioning system of the automated image acquisition cart for high-speed camera vision provided by the present invention utilizes the characteristics of stable operation and low cost of high-speed cameras, and applies advanced image processing algorithms currently. It not only solves the problems of low efficiency, high human resources and costs of traditional indoor manual image acquisition, but also improves the positioning accuracy and speed of the entire positioning system recognition process. At the same time, it reflects the characteristics of small computational workload, high positioning accuracy and fast operation speed in the whole process.

[0051] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as a limitation of the present invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. A positioning method for an automated image acquisition trolley based on high-speed camera vision, characterized in that, The steps are as follows: S1. Design positioning labels: Install a total of 4 positioning labels with different contents on the front and rear sides of the AGV vehicle; divide them into right front, left front, left rear, and right rear according to their positions on the AGV vehicle. Each positioning label includes a square border area and a data area located in the middle of the border area. The data area is a circular icon, and the colors of the border area and the data area are different; the circular icons of the right front, left front, left rear, and right rear are different in shape, and the colors of the border areas and the data areas are also different; S2. Digital environment: Set a camera above the AGV vehicle, calibrate the camera to obtain the mapping relationship between the pixel positions and the actual positions of the fiducial points in the images captured by the camera; the calibration process of the camera includes: the camera captures a standard calibration image; evenly select fiducial points based on the grids in the standard calibration image, record the pixel positions and the actual positions of each fiducial point in the standard calibration image; establish the mapping relationship according to the pixel positions and the actual positions of the fiducial points; S3. Obtain pictures: Use the camera to take pictures of the environment to obtain images containing positioning labels; S4. Binarize the pictures: Perform binarization processing on the images; S5. Identify positioning labels: Analyze the positioning labels in the binarized images to identify the positions, angles, and contents of the positioning labels; S6. Coordinate mapping: According to the position and angle relationship of the center point of the AGV vehicle between the positioning labels, corresponding to the contents of the positioning labels, solve the relative position relationship between the center point of the AGV vehicle and the positioning labels; S7. Locate the pose of the vehicle: Combine the mapping relationship calibrated by the camera, and through coordinate transformation, obtain the pose information of the AGV vehicle in the environment; S8. Navigate the vehicle to move: S81. Select a measured point in the environment and determine the position information of the measured point through the mapping relationship calibrated by the camera; S82. According to the pose information of the AGV vehicle at the current position obtained in step S7, including the orientation of the vehicle and the center point of the vehicle; S83. Calculate the distance and angle information between the center point of the AGV vehicle at the current position and the center point of the AGV vehicle at the target position, as well as the distance and angle information between the center point of the AGV vehicle at the target position and the measured point; S84. Determine the path for the AGV vehicle to move from the current position to the target position according to the information calculated in step S83.

2. The positioning method of the automated image acquisition trolley based on high-speed camera vision according to claim 1, characterized in that: In step S5, select the two positioning labels with the highest recognition degree for recognition calculation. If the calculation passes, the recognition is correct. If the calculation fails, select the third positioning label for recognition calculation. If the calculation of the third positioning label still fails, randomly rotate the angle and perform recognition again.

3. The positioning method of the vision automation image acquisition trolley based on a high-speed camera according to claim 2, characterized in that: The installation positions of the right front, left front, left rear, and right rear on the AGV vehicle have the characteristics of translational and rotational invariance.

4. The positioning method of the automated image acquisition trolley based on high-speed camera vision according to claim 1, characterized in that: The installation position of the camera above the AGV vehicle can ensure that the AGV vehicle can be captured by the camera when moving in the environment and the captured image is a front view.

5. The positioning method of the automated image acquisition trolley based on high-speed camera vision according to claim 4, characterized in that: The camera is an industrial camera with a full-frame exposure method, which has the functions of adjusting the frame rate and exposure time. The camera lens is a wide-angle lens with a field of view angle of 135°.

6. A positioning system for an automated image acquisition trolley based on high-speed camera vision, which is used to execute the positioning method described in claim 1, characterized in that: The positioning system includes a host computer, an AGV vehicle, and a camera. The AGV vehicle moves in the environment, and the camera is installed above the AGV vehicle to photograph the AGV vehicle in the environment. Both the AGV vehicle and the camera are connected to the host computer. Four positioning tags with different contents are installed at the front and rear positions of the AGV vehicle. The host computer is provided with a camera calibration program, a function module for calibrating the measured points, a function module for calibrating the pose of the vehicle, and a function module for calibrating the movement. The camera photographs and obtains an image containing the positioning tags and transmits it to the host computer. The function module for calibrating the measured points is used to select the measured points and calibrate the measured points through the camera calibration program. The function module for calibrating the pose of the vehicle is used to identify the positioning tags, utilize the relative position relationship between the center point of the vehicle and the positioning tags, and calibrate the pose of the vehicle at the current position through the camera calibration program. The function module for calibrating the movement is used to set the target position, calculate the path information for the AGV vehicle to move from the current position to the target position, and send it to the AGV vehicle. The functions of the camera calibration program include: the camera photographs a standard calibration image; Based on the squares in the standard calibration image, fiducial points are evenly selected, and the pixel positions and actual positions of each fiducial point in the standard calibration image are recorded. According to the pixel positions and actual positions of the fiducial points, a mapping relationship is established. The function module for calibrating the pose of the vehicle includes identifying the positions, angles, and contents of the positioning tags, and solving the relative position relationship between the center point of the AGV vehicle and the positioning tags according to the position and angle relationships between the center point of the AGV vehicle and the positioning tags and the corresponding contents of the positioning tags. The function module for calibrating the movement includes calculating the distance and included angle information between the center point of the AGV vehicle at the current position and the center point of the AGV vehicle at the target position, as well as the distance and included angle information between the center point of the AGV vehicle at the target position and the measured point, and determining the path for the AGV vehicle to move from the current position to the target position according to the calculated information.

Images