Monocular vision robot automatic station searching method

Abstract

The invention provides an automatic station searching method for a monocular vision robot, the robot is provided with a camera and an excitation light source, the camera is provided with an optical filter, a workstation is provided with an activatable identifier, and the method comprises the following steps: the robot starts to rotate after navigating to a first position; exciting light is emitted through the exciting light source, the activatable identifier can be activated by the exciting light and emit activation light, and the activation light can pass through the optical filter; the camera receives the activation light and recognizes the activatable identifier, and then rotation is stopped; and the robot can be automatically docked with the work station through the position of the activatable identifier by visual localization. According to the automatic station searching method for the monocular vision robot, on one hand, the robot can identify the activatable identification target more easily, the anti-interference capability is improved, on the other hand, the contrast ratio during imaging is enhanced, the contour and internal details of the activatable identification are clearer, and then the precision of the robot in the monocular vision positioning and docking process is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of robot automatic station search, and more specifically, to a monocular vision robot automatic station search method. Background Art

[0002] All the actions of the robot require energy, and this energy mainly comes from the power storage unit inside the machine. Ensuring the power is the key to satisfying the basic functional behaviors of the robot. In order to make the robot more automated and intelligent, it is necessary to be able to autonomously search for a workstation in the environment after receiving a signal such as charging or water storage, and through precise positioning, the interface on the robot is perfectly matched with the interface of the workstation to achieve logistical purposes such as charging or water storage.

[0003] There is a method in the prior art that sets a QR code on the workstation, relies on the robot's camera to scan the QR code to identify the workstation, and then combines vision with laser radar to achieve the docking of the robot and the workstation. However, the area of ​​the QR code in this method is limited, and there is a problem of inaccurate recognition and scanning at a long distance. At the same time, the camera is also affected by ambient light, which further affects the recognition accuracy of the QR code and the positioning accuracy during the automatic station search process. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a monocular vision robot automatic station search method to solve the technical problem of insufficient positioning accuracy in the automatic station search process in the prior art.

[0005] To achieve the above purpose, the technical solution adopted in this application is: to provide a monocular vision robot automatic station search method, which is applied to a robot and a workstation, wherein the robot is provided with a camera and an excitation light source, the camera is provided with a filter, and the workstation is provided with an activatable mark, and the method comprises the steps of: The robot navigates to the first position and then starts to rotate; The excitation light source emits an excitation light, wherein the activatable mark can be activated by the excitation light and emits the activation light, and the activation light can pass through the filter; After receiving the activation light through the camera and recognizing the activatable mark, the rotation stops; The robot automatically docks with the workstation by visually locating the position of the activatable mark.

[0006] Preferably, the activatable mark may be a fluorescent material layer or a photochromic material layer.

[0007] Preferably, after the robot navigates to the first position and before it starts to rotate, the robot further comprises the following steps: Obtain the camera's in-camera distortion parameters through calibration; The transformation relationship between the pixel coordinate system and the camera coordinate system is obtained through the camera intrinsic parameters, and the distortion relationship of the calibration image is obtained through the distortion parameters.

[0008] Preferably, after the robot navigates to the first position and before it starts to rotate, the robot further comprises the following steps: measuring a first relative position relationship between the activatable marker and the workstation docking interface; The offset between the optical center of the camera and the workstation interface in the horizontal direction is calculated at different positions to obtain a second relative position relationship between the camera and the workstation interface.

[0009] Preferably, the method for automatically docking a robot with a workstation by visually locating the position of an activatable marker comprises the steps of: Based on the first relative position relationship and the second relative position relationship, the solvePnP function is used to solve the rotation vector and translation vector information of the camera in the world coordinate system with respect to the activatable marker as the center; The rotation vector is converted into a rotation matrix through the Rodrigues matrix, and the pitch angle, yaw angle, and roll angle of the camera in the world coordinate system are obtained according to the rotation matrix; The robot moves a distance d1 in a direction parallel to the front of the workstation and arrives directly in front of the workstation; Control the robot to adjust the docking posture according to the pitch angle, yaw angle and roll angle; The robot moves a distance d2 toward the workstation to complete the docking.

[0010] Preferably, after the robot adjusts the docking posture and before moving a distance d2 toward the workstation, it also includes the steps of: continuing to identify the activatable mark, and if it is judged to be within the error, maintaining the current position; if it is outside the error, continuing to move in a direction parallel to the front of the workstation until it reaches the front of the workstation.

[0011] Preferably, the fluorescent material layer is invisible infrared fluorescent ink, the excitation light is ultraviolet light with a wavelength of 200nm-400nm, the filter is a bandpass filter of the corresponding band, and the camera is an infrared camera.

[0012] Preferably, the activatable mark includes a first positioning mark, a second positioning mark and a coding area, the first positioning mark and the second positioning mark are located at the same horizontal height, and the coding area is compiled with workstation parameter information.

[0013] Preferably, the activatable band of the first positioning mark is the first band, the activatable band of the second positioning mark is the second band, the activatable band of the coding area is the third band, and the excitation light emitted by the excitation light source during the rotation of the robot is a mixed laser containing the first band, the second band and the third band; after the robot stops rotating, the steps further include: The excitation light source emits excitation light of a first wavelength band, excitation light of a second wavelength band, and excitation light of a third wavelength band in sequence; The camera correspondingly acquires the first image, the second image, and the third image in sequence; Detecting a first positioning mark from the first image, detecting a second positioning mark from the second image, and detecting a third positioning mark from the third image; The first image, the second image, and the third image are mapped into the same image.

[0014] Preferably, a third positioning mark is arranged at the center position of the first positioning mark, a fourth positioning mark is arranged at the center position of the second positioning mark, the activatable band of the third positioning mark is the second band, and the activatable band of the fourth positioning mark is the first band; during the automatic docking process between the robot and the workstation, if the distance between the robot and the workstation is less than the first threshold range, visual positioning is performed by detecting the third positioning mark and the fourth positioning mark to complete the automatic docking of the robot and the workstation.

[0015] The beneficial effect of the monocular vision robot automatic station search method provided in the present application is that: compared with the prior art, the robot is provided with a camera and an excitation light source, a filter is provided on the camera, and an activatable mark is provided on the workstation. During the self-rotation process, the robot illuminates the surroundings through the excitation light source to emit excitation light. The nearby activatable marks are activated and emit activation light after being irradiated by the excitation light. The activation light passes through the filter and enters the camera, while the ambient light is filtered and cannot enter the camera. In this way, only the activatable mark is shown in the imaging image of the camera. On the one hand, it makes it easier for the robot to identify the activatable mark target and improves the anti-interference ability. On the other hand, it enhances the contrast during imaging, making the outline and internal details of the activatable mark clearer, thereby improving the positioning accuracy of the robot in the automatic pile search process based on monocular vision. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 A schematic diagram of the process of the automatic station search method of the monocular vision robot provided in the embodiment of the present application; Figure 2 A schematic diagram of a robot and a workstation provided in an embodiment of the present application; Figure 3 A schematic diagram of an activatable mark activated by an excitation light source provided in an embodiment of the present application; Figure 4 A schematic diagram of automatic docking between a robot and a workstation provided in an embodiment of the present application; Figure 5 A schematic diagram of the positional relationship between the pitch angle, yaw angle, and roll angle in the world coordinate system provided in an embodiment of the present application; Figure 6 A schematic plan view of an activatable logo provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0020] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0021] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0022] Please also read Figures 1 to 3, the monocular vision robot automatic station search method provided by the embodiment of the present application is now described. The monocular vision robot automatic station search method is applied to a robot and a workstation, the robot is provided with a camera and an excitation light source, the camera is provided with a filter, and the workstation is provided with an activatable mark, and the method comprises the steps of: Step S1, the robot navigates to the first position and starts to rotate; Step S2, emitting excitation light through an excitation light source, wherein the activatable mark can be activated by the excitation light and emit activation light, and the activation light can pass through the filter; Step S3, stopping the rotation after receiving the activation light through the camera and recognizing the activatable mark; Step S4, the robot automatically docks with the workstation by visually locating the position of the activatable mark.

[0023] It is understandable that the activatable mark can be a fluorescent material layer or a photochromic material layer. The characteristics of this type of material are: it has its own color or is colorless in the natural environment, and can be activated after being irradiated with excitation light in a certain wavelength range, and then emits light of another wavelength.

[0024] The display style of the activatable logo can be a marker code called Aruco, which is a binary square reference marker widely used in camera posture estimation. The main advantages are simple and fast detection and high robustness. There is a ready-made detection method in the image processing library opencv for detection, and no special detection model needs to be trained. In other embodiments, it can also be other display styles, such as QR code, bar code, etc.

[0025] The function of the filter itself is to allow only light of a certain wavelength to pass through, that is, in this embodiment, only the activation light is allowed to pass through, thereby filtering out all ambient light and scattered excitation light. In this way, in step S3, the camera can directly recognize the activatable mark after receiving the activation light, and the information recorded in the activatable mark (such as workstation parameter information) can be read after the robot approaches the workstation, thereby improving the recognition speed. Traditional robots need to read the information content of the identification code, and then determine whether it is a workstation positioning identification code based on its information content, in order to eliminate the interference identification codes in the environment.

[0026] It is understandable that, for example, when the robot suddenly needs to be charged or watered during the execution of a task, the robot can automatically navigate to the first position, which can be a fuzzy position close to the workstation, and the robot can find the precise position of the workstation by self-rotation. During the self-rotation process, the excitation light source emits excitation light to illuminate the surroundings, and the nearby activatable markers are activated and emit activation light after being illuminated by the excitation light. The activation light passes through the filter and enters the camera, while the ambient light is filtered and cannot enter the camera. In this way, only the activatable markers are included in the imaging image of the camera, and for example, the approximate interference markers and background images in the environment will not be imaged. In addition, whether it is a dark or bright reflective environment, it will not be affected. Therefore, on the one hand, it makes it easier for the robot to identify the activatable marker target and improve the anti-interference ability. On the other hand, it enhances the contrast during imaging, making the outline and internal details of the activatable marker clearer, thereby improving the positioning accuracy of the robot in the automatic pile-finding process based on monocular vision.

[0027] In a specific embodiment, before step S1, that is, before the robot starts to rotate after navigating to the first position, the following steps are also included: Obtain the camera's in-camera distortion parameters through calibration; The transformation relationship between the pixel coordinate system and the camera coordinate system is obtained through the camera intrinsic parameters, and the distortion relationship of the calibration image is obtained through the distortion parameters.

[0028] It is understandable that the calibration method can use the existing Zhang Zhengyou calibration method, which is mainly to obtain the camera's intrinsic distortion parameters. The transformation relationship between the pixel coordinate system and the camera coordinate system is obtained through the camera intrinsic parameters. Since the camera is located on the robot, the camera coordinate system can be converted into the robot's map coordinate system. By calibrating the image distortion relationship, the error caused by the camera image distortion can be reduced and the target positioning accuracy can be improved.

[0029] Furthermore, before step S1, that is, before the robot starts to rotate after navigating to the first position, the following steps are also included: measuring a first relative position relationship between the activatable marker and the workstation docking interface; The offset between the optical center of the camera and the workstation interface in the horizontal direction is calculated at different positions to obtain a second relative position relationship between the camera and the workstation interface.

[0030] It is understandable that since the positions of the activatable marker and the workstation interface are inconsistent, the camera actually locates the position of the activatable marker, which can be converted into the position of the workstation interface by measuring the first relative position relationship between the activatable marker and the workstation interface.

[0031] In addition, unlike 3D vision, the monocular vision of the present application itself does not have depth information, so it is necessary to calculate the horizontal offset between the optical center of the camera and the workstation interface at different positions, that is, it is necessary to obtain the second relative position relationship between the camera and the workstation interface based on the pixel level, and convert the offset based on the pixel level into the actual plane distance.

[0032] For further information, please also refer to Figure 4 and Figure 5 In step S4, the method of automatically docking the robot with the workstation by visually locating the position of the activatable mark includes the steps of: Based on the first relative position relationship and the second relative position relationship, the solvePnP function is used to solve the rotation vector and translation vector information of the camera in the world coordinate system with respect to the activatable marker as the center; The rotation vector is converted into a rotation matrix through the Rodrigues matrix, and the pitch angle, yaw angle, and roll angle of the camera in the world coordinate system are obtained according to the rotation matrix; The robot moves a distance d1 in a direction parallel to the front of the workstation and arrives directly in front of the workstation; Control the robot to adjust the docking posture according to the pitch angle, yaw angle and roll angle; The robot moves a distance d2 toward the workstation to complete the docking.

[0033] It can be understood that since the ground can be approximately considered flat, the robot and the workstation are both on the plane, so the yaw angle is the angle that the robot needs to adjust, and the translation direction only needs to consider the horizontal offset distance.

[0034] In addition, after the robot adjusts the docking posture and before moving a distance d2 toward the workstation, it also includes the following steps: continue to identify the activatable mark, and if it is judged to be within the error, maintain the current position; if it is outside the error, continue to move in a direction parallel to the front of the workstation until it reaches the front of the workstation.

[0035] In a specific embodiment, the fluorescent material layer is invisible infrared fluorescent ink, the excitation light is ultraviolet light with a wavelength of 200nm-400nm, the filter is a bandpass filter of the corresponding band, and the camera is an infrared camera.

[0036] It is understandable that since the invisible infrared fluorescent ink is invisible, i.e. colorless, it is invisible to the naked eye when printed on the workstation. On the one hand, it can improve the system security and prevent the robot from being cracked, and on the other hand, it will not affect the aesthetics of the workstation. At the same time, ultraviolet light with a wavelength of 200nm-400nm is also invisible light, and the robot will not cause visual interference to people even in public places. In addition, the invisible infrared fluorescent ink can emit an activation light with a wavelength of 750nm to 1000nm under the irradiation of the ultraviolet light. The activation light with a wavelength of 750nm to 1000nm is in the infrared light band, so the activation light is also invisible light.

[0037] In this way, the entire process of steps S1-S4 is completed under the "invisible" effect, with a high degree of anti-cracking ability, and will not cause visual interference to people regardless of whether it is in a bright or dark environment.

[0038] In a specific embodiment, please refer to Figure 6 The activatable mark includes a first positioning mark, a second positioning mark and a coding area, the first positioning mark and the second positioning mark are located at the same horizontal height, and the coding area is compiled with workstation parameter information.

[0039] It is understandable that, since the initial distance and yaw angle between the robot and the workstation are unknown, when the yaw angle or the initial distance is large, there is a situation where the camera cannot clearly obtain the activatable mark, but the positioning can be achieved by first detecting the positions of the first positioning mark and the second positioning mark. For example, the distance is calculated by calculating the pixel size of the first positioning mark and the second positioning mark in the image through a visual algorithm, and the yaw angle is calculated by calculating the relative position of the first positioning mark and the second positioning mark in the image through a visual algorithm. In this way, after the robot locates the relative position relationship with the workstation, it approaches the workstation and identifies the coding area during the docking process.

[0040] For further information, please also refer to Figure 6 , the activatable band of the first positioning mark is the first band, the activatable band of the second positioning mark is the second band, the activatable band of the coding area is the third band, and the excitation light emitted by the excitation light source during the rotation of the robot is a mixed laser containing the first band, the second band and the third band; after the robot stops rotating, the step further includes: The excitation light source emits excitation light of a first wavelength band, excitation light of a second wavelength band, and excitation light of a third wavelength band in sequence; The camera correspondingly acquires the first image, the second image, and the third image in sequence; Detecting a first positioning mark from the first image, detecting a second positioning mark from the second image, and detecting a third positioning mark from the third image; The first image, the second image, and the third image are mapped into the same image.

[0041] It is understandable that, since the area of ​​the activatable mark is small and the positions of the first positioning mark and the second positioning mark are close, although it is easier to find the position of the activatable mark under the mixed laser, the first positioning mark and the second positioning mark detected at this time will interfere with each other during imaging. The main reasons are as follows: 1. Overlap of light signals: The light signals of two positioning marks will overlap on the image, making it difficult to accurately distinguish the positions of the two positioning marks. This overlap will cause errors in the image processing algorithm when extracting and matching features.

[0042] 2. Uneven lighting: When two positioning marks are activated at the same time, the brightness will be uneven, especially when the two positioning marks are close to each other or the light intensity is very different. This uneven light source will affect the image quality and reduce the positioning accuracy.

[0043] 3. Camera exposure problem: Activating two positioning markers at the same time causes the camera to overexpose or underexpose, thus affecting the clarity and contrast of the image. This will also reduce the positioning accuracy.

[0044] Thus, in this embodiment, after the robot stops rotating, the method of activating two positioning marks and detecting their positions respectively has a higher positioning accuracy in the image vision positioning process. This method can avoid overlapping of activation light, improve the uniformity of activation light and reduce camera exposure problems, thereby ensuring that the image processing algorithm can extract and match features more accurately, thereby improving positioning accuracy.

[0045] Similarly, the recognition rate can also be improved when identifying the workstation parameter information in the coding area at a long distance.

[0046] For further information, please also refer to Figure 6 , a third positioning mark is set at the center position of the first positioning mark, a fourth positioning mark is set at the center position of the second positioning mark, the activatable band of the third positioning mark is the second band, and the activatable band of the fourth positioning mark is the first band; during the automatic docking process between the robot and the workstation, if the distance between the robot and the workstation is less than the first threshold range, the third positioning mark and the fourth positioning mark are detected for visual positioning to complete the automatic docking of the robot and the workstation.

[0047] It is understandable that there will inevitably be errors in positioning at long distances. However, after the robot is guided to a close range through coarse positioning, the error will be magnified, and the camera needs more precise visual positioning to complete docking. In this way, at close distances, positioning is performed by switching between the third positioning mark and the third positioning mark. Close-range precise positioning can more accurately correct these errors, reduce the impact of cumulative errors on the final positioning results, and improve the accuracy of automatic docking.

[0048] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A monocular vision robot automatic station search method, applied to robots and workstations, characterized in that: The robot is provided with a camera and an excitation light source, the camera is provided with a filter, and the workstation is provided with an activatable mark. The method comprises the steps of: The robot navigates to the first position and then starts to rotate; The excitation light source emits an excitation light, wherein the activatable mark can be activated by the excitation light and emits the activation light, and the activation light can pass through the filter; After receiving the activation light through the camera and recognizing the activatable mark, the rotation stops; The robot automatically docks with the workstation by visually locating the position of the activatable mark.

2. The monocular vision robot automatic station search method according to claim 1, characterized in that: After the robot navigates to the first position and before it starts to rotate, the steps further include: Obtain the camera's in-camera distortion parameters through calibration; The transformation relationship between the pixel coordinate system and the camera coordinate system is obtained through the camera intrinsic parameters, and the distortion relationship of the calibration image is obtained through the distortion parameters.

3. The monocular vision robot automatic station search method as claimed in claim 2, characterized in that: After the robot navigates to the first position and before it starts to rotate, the steps further include: measuring a first relative position relationship between the activatable marker and the workstation docking interface; The offset between the optical center of the camera and the workstation interface in the horizontal direction is calculated at different positions to obtain a second relative position relationship between the camera and the workstation interface.

4. The monocular vision robot automatic station search method as claimed in claim 3, characterized in that: A method for automatically docking a robot with a workstation by visually locating a position of an activatable marker, comprising the steps of: Based on the first relative position relationship and the second relative position relationship, the solvePnP function is used to solve the rotation vector and translation vector information of the camera in the world coordinate system with respect to the activatable marker as the center; The rotation vector is converted into a rotation matrix through the Rodrigues matrix, and the pitch angle, yaw angle, and roll angle of the camera in the world coordinate system are obtained according to the rotation matrix; The robot moves a distance d1 in a direction parallel to the front of the workstation and arrives directly in front of the workstation; Control the robot to adjust the docking posture according to the pitch angle, yaw angle and roll angle; The robot moves a distance d2 toward the workstation to complete the docking.

5. The monocular vision robot automatic station search method as claimed in claim 4, characterized in that: After the robot adjusts the docking posture and before it moves a distance d2 toward the workstation, it also includes the following steps: continue to identify the activatable mark, and if it is judged to be within the error, maintain the current position; if it is outside the error, continue to move in a direction parallel to the front of the workstation until it reaches the front of the workstation.

6. The monocular vision robot automatic station search method according to claim 1, characterized in that: The activatable mark may be a fluorescent material layer or a photochromic material layer.

7. The monocular vision robot automatic station search method according to claim 6, characterized in that: The fluorescent material layer is invisible infrared fluorescent ink, the excitation light is ultraviolet light with a wavelength of 200nm-400nm, the filter is a bandpass filter of the corresponding band, and the camera is an infrared camera.

8. The monocular vision robot automatic station search method according to claim 1, characterized in that: The activatable mark includes a first positioning mark, a second positioning mark and a coding area, the first positioning mark and the second positioning mark are located at the same horizontal height, and the coding area is compiled with workstation parameter information.

9. The monocular vision robot automatic station search method as claimed in claim 8, characterized in that: The activatable waveband of the first positioning mark is the first waveband, the activatable waveband of the second positioning mark is the second waveband, the activatable waveband of the coding area is the third waveband, and the excitation light emitted by the excitation light source during the rotation of the robot is a mixed laser including the first waveband, the second waveband and the third waveband; after the robot stops rotating, the steps further include: The excitation light source successively emits excitation light of a first wavelength band, excitation light of a second wavelength band, and excitation light of a third wavelength band; The camera correspondingly acquires the first image, the second image, and the third image in sequence; Detecting a first positioning mark from the first image, detecting a second positioning mark from the second image, and detecting a third positioning mark from the third image; The first image, the second image, and the third image are mapped into the same image.

10. The monocular vision robot automatic station search method according to claim 9, characterized in that: A third positioning mark is set at the center position of the first positioning mark, and a fourth positioning mark is set at the center position of the second positioning mark. The activatable band of the third positioning mark is the second band, and the activatable band of the fourth positioning mark is the first band. During the automatic docking process between the robot and the workstation, if the distance between the robot and the workstation is less than the first threshold range, visual positioning is performed by detecting the third positioning mark and the fourth positioning mark to complete the automatic docking of the robot and the workstation.

Images