Laser positioning method based on bright frame image

Abstract

The application discloses a laser positioning method based on a bright frame image. An execution subject of the laser positioning method is a robot equipped with a structured light module. The structured light module comprises a line laser emitter and a camera. The laser positioning method comprises the following steps: the robot searches for a line laser position from the bright frame image by executing an inter-frame tracking algorithm, and sets a coordinate of the line laser position as a positioning coordinate of the line laser emitted by the line laser emitter in a current frame image.

Description

Technical Field

[0001] This invention relates to the technical field of laser data screening, and more particularly to a laser positioning method based on bright frame images. Background Technology

[0002] Structured light modules generally refer to any laser module that includes a line laser emitter and a camera module. In a structured light module, the line laser emitter emits a line laser beam. This beam can be positioned in front of the robot. The camera module can capture environmental images and receive reflected light from objects struck by the line laser beam. The line laser beam emitted by the emitter is within the camera module's field of view. The line laser helps detect the contours, height, and / or width of objects in the robot's direction of travel; this information is collectively referred to as the laser's position information. When the camera module captures images of the reflected light from the line laser emitter on obstacle surfaces, the selected laser line positioning area is relatively coarse and incomplete. In scenarios with high ambient light intensity, this can easily introduce interference points, leading to misjudgments and affecting the obstacle avoidance performance of the robot with the structured light module. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention discloses a laser positioning method based on bright frame images, the specific technical solution of which is as follows:

[0004] The laser positioning method based on bright frame images is executed by a robot equipped with a structured light module, which includes a line laser emitter and a camera. The laser positioning method includes: the robot searching for the position of the line laser in the bright frame image by executing an inter-frame tracking algorithm, and then setting the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image.

[0005] Furthermore, the method by which the robot searches for the position of the line laser in a bright frame image by executing an inter-frame tracking algorithm includes: Step 1, the robot traverses the current frame image column by column and obtains the initial pixel position in the corresponding column of the current frame image. At the same time, it excludes pixels in the current frame image that do not have a line laser position based on the pixels in the corresponding column that conform to the preset brightness distribution characteristics. The line laser position is used to represent the reflection position of the line laser on the surface of the object to be measured; Step 2, except for the column containing the pixels where the line laser position does not exist, in the current column of the current frame image, the robot sets the initial pixel position in the current column as the search center, and then searches upwards along the current column from the search center within a search radius. The system searches for pixels, starting from the search center and moving downwards along the current column within a search radius. Then, based on the difference in brightness values ​​between the upward and downward searched pixels at the search centers determined in two consecutive search states, and the inter-frame matching relationship formed by the same type of pixel values ​​in the same column of the current frame image relative to the reference frame image, it filters out the convex hull center pixels in the current column to update the previously determined convex hull center pixels in the current column of the current frame image. The reference frame image is configured as the bright frame image where the robot most recently found the line laser position before acquiring the current frame image. Each time the search center in the current column is updated, the convex hull values ​​set in the current column are... The center pixel is also updated once; Step 3: Based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel in the current frame image, interference points are removed from the already selected convex hull center pixels; After the robot has traversed all the convex hull center pixels in all columns of pixels in the current frame image to remove all interference points, the coordinates of the remaining convex hull center pixels are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. Then the robot searches for the determined line laser position in each column in the current frame image to connect the line laser emitted by the line laser emitter to the object under test. The laser line segments formed on the surface are determined, and the robot has searched for the laser line positions in the current frame image by executing an inter-frame tracking algorithm. The laser line positions determined in the same column are the positions of the last updated convex hull center pixels in the same column after the robot has traversed all pixels in that column. The coordinates of a laser line position are represented using the corresponding positioning coordinates. The selected convex hull center pixels are the last updated convex hull center pixels in each column of convex hull center pixels in the current frame image. The selected convex hull center pixels are the convex hull center pixels in each column of convex hull center pixels in the current frame image that are closest to the origin of the coordinate system of the current frame image.The current frame image is a bright frame image of the light reflected back from the surface of the object being measured by a line laser emitted by a line laser emitter, captured by the robot-controlled camera.

[0006] Further, in step 2, whenever a convex hull center pixel is selected for a search center, the adjacent pixel searched upwards or downwards along the current column from the search center is updated as the search center, and step 2 is executed again to obtain a new convex hull center pixel and update it as the new convex hull center pixel; each search center is within the coverage area of ​​a search radius relative to the initial pixel position, wherein the search radius is set to a first preset pixel distance; wherein the robot sets the set of pixels conforming to the convex hull feature in the current column of the current frame image as the brightness value, starting from the convex hull center and moving upwards along the current column respectively. A convex hull is formed by a set of pixels that decrease in brightness from top to bottom and the center of the convex hull. The center of the convex hull is the pixel with the highest brightness value in the set of pixels, and the pixel at the center of the convex hull is set as belonging to the pixel at the center of the convex hull. Within the set of pixels that meet the characteristics of the convex hull, starting from the center of the convex hull and moving upward along the same column, the brightness value of the pixel decreases upward along the current column and generates a first gradient value between the brightness values ​​of two adjacent pixels. Furthermore, starting from the center of the convex hull and moving downward along the same column, the brightness value of the pixel decreases downward along the current column and generates a second gradient value between the brightness values ​​of two adjacent pixels, so that the center of the convex hull belongs to the search center.

[0007] Further, in step 2, the method of filtering out convex hull center pixels based on the difference between the brightness values ​​of upward-searched pixels and downward-searched pixels in the search states corresponding to two adjacent determined search centers, and the inter-frame matching relationship formed by the same type of values ​​of pixels in the same column of the current frame image relative to the reference frame image, includes: in the current column of the current frame image, controlling the brightness value of the search center to compare with the brightness value of the convex hull center pixel in the same column of the previously searched image; the convex hull center pixel in the same column of the previously searched image is the convex hull center pixel filtered out in the same column of the current frame image based on the previously determined search center, the previously determined search center is a pixel that is either downward or upward adjacent to the currently determined search center in the current column of the current frame image, and the column order of the pixels in the same column of the current frame image is the same as that of the previously determined search center. The column order of the current column in the current frame image is equal. If the brightness value of the currently determined search center is greater than the brightness value of the convex hull center pixel found in the same column in the previous search, then in the current column of the current frame image, pixels are searched upwards from the search center, and the number of pixels whose brightness values ​​decrease according to the first gradient value is counted until the upward counting stop condition is met. Then, the number of pixels whose brightness values ​​decrease according to the first gradient value is marked as the upward gradient descent quantity, and the upward search for pixels stops until the search center is updated next time. Furthermore, pixels are searched downwards from the search center, and the number of pixels whose brightness values ​​decrease according to the second gradient value is counted until the downward counting stop condition is met. Then, the number of pixels whose brightness values ​​decrease according to the second gradient value is marked as the downward gradient descent quantity, and the downward search for pixels stops until the search center is updated next time.When the robot determines that the number of upward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents required to find the center pixel of the convex hull in the same column in the previous search, and / or determines that the number of downward gradient descents counted in the current column of the current frame image is greater than or equal to the number of downward gradient descents required to find the center pixel of the convex hull in the same column in the previous search, if the robot detects that among the pixels traversed in the current column of the current frame image, neither the first gradient value nor the second gradient value is equal to the first preset gradient parameter, and the absolute value of the difference between the first gradient value and the second gradient value is less than the second preset gradient parameter, then... Furthermore, if the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found by searching upwards along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between similar brightness values ​​formed by searching upwards along the same column of pixels in the reference frame image, and the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found by searching downwards along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between similar brightness values ​​formed by searching downwards along the same column of pixels in the reference frame image, then the robot marks the currently determined search center as the convex hull center pixel; wherein, the first preset gradient parameter is less than the second preset gradient parameter.

[0008] Further, the absolute value of the difference between the same type of brightness values ​​formed by searching upwards among pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found by searching upwards from the search center in the same column as the current column, and the brightness value of the pixel at the search center in the same column. The distance between the pixel with the smallest brightness value found by searching upwards and the search center in the same column is less than or equal to the search radius. Similarly, the absolute value of the difference between the same type of brightness values ​​formed by searching downwards among pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found by searching downwards from the search center in the same column as the current column, and the brightness value of the pixel at the search center in the same column. The distance between the pixel with the smallest brightness value found by searching downwards and the search center in the same column is less than or equal to the search radius.

[0009] Furthermore, for a currently determined search center, step 2 further includes: if the brightness value of a pixel at the search center is greater than the brightness value of the pixel at the center of the convex hull in the same column found in the previous search, then in the current column of the current frame image, a search is performed upwards from the search center and downwards from the search center for pixels; if the robot detects that the brightness value of a pixel is not decreasing according to the first gradient value during the upward search from the search center, then the pre-set upward gradient anomaly count is performed once, and then the robot determines whether it has searched all the pixels covered within the search radius upwards along the current column of the current frame image; if so, the robot stops. The robot stops searching for pixels upwards along the current column of the current frame image and determines that the upward counting stop condition has been met. Otherwise, if the upward gradient anomaly frequency is greater than a first preset error count, the robot stops searching for pixels upwards along the current column of the current frame image and determines that the upward counting stop condition has been met. Furthermore, if during the downward search from the search center, it is detected that the brightness value of a pixel does not decrease according to the second gradient value, then the preset downward gradient anomaly count is counted once. Then, the robot determines whether it has searched all the pixels covered within the search radius downwards along the current column of the current frame image. If yes, the robot stops searching for pixels downwards along the current column of the current frame image and determines that the upward counting stop condition has been met. The counting stops if the upward gradient anomaly frequency exceeds the second preset error count. Otherwise, the robot stops searching for pixels downwards along the current column of the current frame image and determines that the downward counting stop condition is met. Alternatively, during the upward search from the search center, the robot counts the pixels with a brightness value of 255 that are adjacent to each other along the current column of the current frame image, and marks the number of pixels with a brightness value of 255 that are adjacent to each other as the upward overexposure count. When the robot detects that the upward overexposure count exceeds the third preset error count, and / or when the number of pixels covered within the search radius has been counted upwards along the current column of the current frame image, the robot stops. The robot stops searching for pixels upwards along the current column of the current frame image and determines that the upward counting stop condition is met. During the downward search from the search center, the robot counts pixels with a brightness value of 255 that are adjacent to each other along the current column of the current frame image, and marks the number of pixels with a brightness value of 255 that are adjacent to each other as the downward overexposure count. When the robot detects that the upward overexposure count is greater than the fourth preset error count, and / or when the number of pixels covered within the search radius is counted downwards along the current column of the current frame image, the robot stops searching for pixels downwards along the current column of the current frame image and determines that the downward counting stop condition is met.

[0010] Further, in step 3, the method of removing interference points from the already selected convex hull center pixels based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel in the current frame image includes: the robot traverses all columns of pixels in the current frame image and obtains the latest convex hull center pixel in each column, and saves the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image, for each of the current frame images... The convex hull center pixel is defined as follows: within a circular region centered on the location of the line laser emitted by the line laser emitter in the previous dark frame image, and with a radius equal to the distance of the detection pixel, if the robot determines that the brightness value of at least one pixel within this circular region is greater than the brightness value of the convex hull center pixel in the current frame image that has the same coordinates as the center of the circle by a preset ambient light brightness threshold, then the robot determines that the convex hull center pixel in the current frame image that has the same coordinates as the center of the circle is an interference point. The robot cannot find the line laser position at the interference point and removes the interference point from the current frame image.

[0011] Further, in step 1, the method of excluding pixels in the current frame image that do not have a line laser position based on pixels in the corresponding column that conform to the preset brightness distribution characteristics includes: if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round by a first preset brightness threshold, or if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round by a second preset brightness threshold, then starting from a position one reference pixel distance upward along the current column of the current frame image, searching for pixels downward along the current column of the current frame image; if it is detected that the brightness value of the currently searched pixel is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round... If the brightness value of a pixel is greater than a first preset brightness threshold, or if the brightness value of a currently searched pixel is equal to 255, the error position counter is counted once, and the currently searched pixel is determined to be a pixel that conforms to the preset brightness distribution characteristics. When the robot detects that the error position counter count is greater than the reference pixel count threshold, it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels without line laser positions, and the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. The reference pixel distance is represented by the number of pixels so that the reference pixel count threshold is equal to the reference pixel distance. The line laser position found in the same column in the previous round is the position of the convex hull center pixel finally determined in the same column of the reference frame image.

[0012] Further, in step 1, the method of excluding pixels in the current frame image that do not have a line laser position based on pixels in the corresponding column that conform to the preset brightness distribution characteristics includes: taking the initial pixel position in the current column of the current frame image as the center of a ring, in the current column of the current frame image, marking the pixels covered by the annular area located below the center of the ring, with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius, as the first test pixel; then calculating the average brightness value of the first test pixel; if the average brightness value of the first test pixel is greater than the brightness value of the pixel located at the line laser position in the same column found in the previous round, then determining that the first test pixel conforms to the preset brightness distribution characteristics, and determining that there is no line laser position in the current column of the current frame image, then setting the pixels in the current column of the current frame image as pixels without a line laser position, and then excluding the pixels in the current column of the current frame image from the pixel search range in step 2; wherein, the first positioning radius is smaller than the second positioning radius, and the line laser position located in the same column found in the previous round is a reference. The location of the convex hull center pixel finally determined in the same column of the frame image; or, taking the initial pixel position in the current column of the current frame image as the center of a ring, in the current column of the current frame image, the pixels covered by the ring area above the center of the ring, with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius, are marked as the second test pixels. Then, the average value of the brightness value of the second test pixels is calculated. If the average value of the brightness value of the second test pixels is greater than the brightness value of the pixel at the line laser position found in the previous round in the same column, then the second test pixel is determined to be a pixel that conforms to the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels without line laser positions, and the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. Wherein, the first positioning radius is smaller than the second positioning radius, and the line laser position found in the previous round in the same column belongs to the location of the convex hull center pixel finally determined in the same column of the reference frame image.

[0013] Furthermore, the initial pixel position is the position of the original pixel in the image captured by the camera after the line laser emitted by the line laser emitter is reflected back to the camera's field of view on the robot's traveling plane, assuming there are no obstacles in front of the robot. Each original pixel is a reflection position on the robot's traveling plane, used to represent the search starting point for searching the line laser position in each column of the same frame image. The reference frame image is a bright frame image configured to contain the line laser position most recently found by the robot before the current frame image is captured. The line laser position most recently found by the robot originates from the convex hull center pixel in the corresponding column of the reference frame image.

[0014] Further, in step 1, if the initial pixel position cannot be obtained in the current column of the current frame image, the line laser position found in the same column in the previous round is updated to the initial pixel position, and the second preset pixel distance is updated to the search radius. Then, step 2 is repeated to search for the convex hull center pixel in the corresponding column. The line laser position found in the same column in the previous round is the position of the convex hull center pixel finally determined in the same column of the reference frame image or the initial pixel position in the same column of the first bright frame image. If the robot cannot find the convex hull center pixel in the same column during the repeated execution of step 2, it is determined that the robot cannot find the line laser position in the same column.

[0015] Furthermore, the image sequence formed by the light reflected back from the surface of the object being measured by the line laser emitted by the line laser emitter is configured to alternate between bright frame images and dark frame images, such that: when the current frame image captured by the camera is a bright frame image, the next frame image captured by the camera is a dark frame image; during the time interval between the camera capturing the current bright frame image and the camera capturing the next bright frame image, the camera captures the current dark frame image; after the camera captures the next bright frame image, the camera captures the next dark frame image; wherein, during the execution of the laser positioning method, the first frame image of the image sequence is a bright frame image.

[0016] Furthermore, if the installation distance between the camera and the line laser module is greater, then in the image captured by the camera, the coordinate offset of the pixel representing the reflection position of the line laser on the surface of the obstacle relative to the center of the camera will increase.

[0017] Furthermore, the emission angle of the line laser emitter and the receiving angle of the camera are set as follows: the line laser emitter emits a line laser to a preset detection position in front of the robot body, and the line laser is reflected back to the camera at the preset detection position. The length of the laser line segment formed by the line laser at the preset detection position is greater than the width of the robot body. Whenever the robot travels a preset distance along the direction from the current position to the preset detection position, the horizontal distance between the preset detection position and the robot decreases, and the coordinate offset of the pixel in the image captured by the camera that represents the same reflection position of the line laser at the preset detection position increases relative to the center of the camera.

[0018] The technical effect of this invention is as follows: During the laser positioning method for tracking the reflected light of a line laser, when the robot detects that the current frame image captured by the camera is a bright frame image, it selects to input the current frame image into the processing rule model corresponding to the inter-frame tracking algorithm to output an effective laser position. This effectively filters out various ambient light interferences in scenarios where the camera is not too close to obstacles, reducing reliance on infrared filters. Specifically, in pixels conforming to convex hull features, the invention considers the numerical relationship between the brightness gradient generated in upward-searched pixels and the brightness gradient generated in downward-searched pixels, the difference in the search state corresponding to two adjacent determined search centers, the relationship between the brightness value of the currently searched pixel and the brightness value of the convex hull center pixel determined in the same column of the same frame image, and the formation of the same type of values ​​in the same column of pixels relative to the reference frame image. The system uses inter-frame matching to filter out the convex hull center pixels of the current column. It then traverses every pixel within the search radius relative to the initial pixel position in the current column and updates the convex hull center pixels. After eliminating interference from pixels where there are no line laser positions, the final convex hull pixels of the current column are determined. This method can more accurately and efficiently determine areas rich in line laser image information and reduce misjudgment of interference points in strong ambient light. It searches for more accurate convex hull center pixels by tracking the number and brightness values ​​of pixels that conform to the convex hull characteristics of the reference frame image. After eliminating all interference points, the coordinates of the remaining convex hull center pixels are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. This allows for more accurate tracking of the reflected light from the laser on the obstacle surface within a relatively comprehensive image area, making it suitable for robot navigation and walking scenarios to achieve the effect of robot obstacle localization. Attached Figure Description

[0019] Figure 1 This is a flowchart of a laser positioning method based on bright frame images, as disclosed in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. To further illustrate the embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0021] This invention discloses a laser positioning method based on bright frame images. Specifically, it locates the reflection position of laser light on the surface to be tested. Moreover, it adaptively selects representative laser line position information based on the changes in the brightness values ​​of pixels in two related frames of images acquired by a camera (corresponding to changes in ambient light intensity), thereby overcoming the interference of ambient light and improving obstacle detection accuracy and obstacle avoidance efficiency of the robot. The laser positioning method disclosed in this invention is implemented by a robot that relies on structured light navigation and positioning. This robot is equipped with a structured light module, which includes a line laser emitter and a camera. Optionally, if the camera does not have an infrared filter, the image captured by the camera retains both infrared and visible light imaging information. The line laser emitted by the line laser emitter is within the field of view of the camera. The line laser emitted by the line laser sensor can be projected onto the surface of an obstacle. The field of view of the camera covers all or part of the obstacle's outline. In general navigation and positioning scenarios, during indoor and outdoor movement, the structured light module on the robot can detect whether there are obstacles in front of the robot's direction of travel. As the robot moves towards an obstacle, the laser positioning method is executed to influence the detection accuracy of the reflection position on the obstacle's surface, thereby improving positioning and obstacle avoidance accuracy.

[0022] It should be noted that the structured light module used in this application refers to any sensor module that includes a line laser emitter and a camera. In the structured light module, the line laser emitter is used to emit line lasers outward. The line laser emitted by the line laser emitter can be located within the effective detection area in front of the robot. The camera can sequentially acquire multiple frames of images under various ambient light conditions, including infrared and visible light imaging information. The visible light imaging information can be directly used to build a map and mark the positions of obstacles in the map. In this embodiment, the focus is mainly on receiving the image of the reflected light returned from the object being measured by the line laser. It is necessary to overcome the interference of ambient light of different wavelengths (introducing interference point information). The line laser emitted by the line laser emitter is within the field of view of the camera and forms a laser line segment on the surface of the object being measured or on a horizontal ground. The line laser can help detect the contour, height, and / or width of objects in the robot's direction of travel. This embodiment mainly extracts the height information of the object to meet the needs of the inter-frame tracking algorithm. Compared to image sensor-based sensing solutions, line laser emitters can provide cameras with more accurate pixel height and orientation information, reducing the complexity of sensing operations and improving real-time performance.

[0023] Specifically, the working principle of the structured light module is as follows: a line laser emitter emits a line laser beam. After reaching the surface of an obstacle, a portion of the emitted line laser beam is reflected back and forms pixels on the image via the optical imaging system in the camera. Because the distance from the object surface to the return point varies, the flight time of the reflected light differs. By measuring the flight time of the reflected light, each pixel can obtain independent distance and direction information. Then, using triangulation, height and width information are obtained and marked as the coordinate information of the pixel on the image, collectively referred to as position information. During the robot's movement, on the one hand, the line laser emitter in the structured light module can be controlled to emit line laser beams. When the line laser encounters an obstacle in the path, it will be reflected back, at least covering the ground medium and low obstacles on the ground. On the other hand, the camera in the structured light module can be controlled to capture environmental images of the area in front. During this period, if the line laser detects an obstacle in the path, it will form a laser line segment on the object's surface. This laser line segment can be captured by the camera; that is, the image captured by the camera will contain the laser line segment formed by the line laser emitted by the line laser emitter encountering the object. The angle between the laser line segment formed by the line laser on the object's surface and the horizontal plane is not limited; for example, it can be parallel or perpendicular to the horizontal plane, or at any angle, depending on the application requirements. Each laser line segment contains multiple pixels, and each pixel corresponds to a point on the obstacle surface. The pixels on the laser line segments in a large number of environmental images, representing points on the obstacle surface, can form obstacle point cloud data. The coordinate system used for this obstacle point cloud data can be the robot's coordinate system. The robot can then use the transformation relationship between the camera's image coordinate system and the robot's coordinate system to transform the pixel coordinates on the laser line segment to the robot's coordinate system to obtain the obstacle point cloud data. Alternatively, the coordinate system used for this obstacle point cloud data can also be the world coordinate system. The robot can then use the transformation relationship between the camera's coordinate system, the robot's coordinate system, and the world coordinate system to transform the pixel coordinates on the laser line segment to the robot's coordinate system to obtain the obstacle point cloud data. The obstacle point cloud data can include, but is not limited to, the three-dimensional coordinate information, color information, reflection intensity information, etc. After obtaining obstacle point cloud data, the height and width information of the obstacles are acquired, and the type of obstacle can be identified based on the obstacle point cloud data. In this embodiment, the identification of obstacle type and its occupied area is not limited to using obstacle point cloud data. For example, obstacle point cloud data can be input into a deep learning model to identify obstacle type. Alternatively, obstacles can be depicted based on obstacle point cloud data to obtain obstacle points and obstacle outlines, and the obstacle type can be determined based on the obstacle outlines. Alternatively, obstacle point clustering analysis, threshold filtering, and confidence level judgment can be used.

[0024] Specifically, the implementation form of the line laser emitter is not limited; it can be any device / product capable of emitting line lasers. For example, a line laser emitter can be, but is not limited to, a laser tube. Similarly, the implementation form of the camera is not limited; any vision device capable of acquiring environmental images is applicable to the embodiments of this application. For example, a camera can include, but is not limited to, a monocular camera, a binocular camera, etc. In the embodiments of this application, the wavelength of the line laser emitted by the line laser emitter can be limited to the wavelength of infrared light, such as an infrared laser. Of course, in some embodiments, the installation position, installation angle, etc., of the line laser emitter, as well as the installation positional relationship between the line laser emitter and the camera module, are not limited. In the embodiments of this application, the number of line laser emitters is not limited; for example, it can be one, two, or more. Similarly, the number of cameras is not limited; for example, it can be one, two, or more.

[0025] In some embodiments, the field of view of the camera includes a vertical field of view and a horizontal field of view. In this embodiment, a camera with a suitable field of view can be selected according to application requirements, as long as the line laser emitted by the line laser emitter is within the field of view of the camera. The angle between the laser line segment formed by the line laser on the object surface and the horizontal plane is not limited. For example, it can be parallel or perpendicular to the horizontal plane, or it can be at any angle to the horizontal plane, depending on the application requirements.

[0026] In some embodiments, the installation height of the structured light module, consisting of a line laser emitter and a camera, needs to be determined based on the size of the obstacle to be detected. A higher installation height of the structured light module within the robot results in a larger longitudinal space covered in front of the robot, which reduces the detection accuracy for smaller obstacles. Conversely, a lower installation height results in a smaller longitudinal space covered in front of the robot, improving the detection accuracy for smaller obstacles. Preferably, the line laser emitter is installed above the camera without an infrared filter, and the centerline of the line laser emitter intersects the centerline of the camera at a single point.

[0027] See Figure 1As can be seen, the laser positioning method based on bright frame images disclosed in this invention basically includes: the robot searches for the position of the line laser from the bright frame image by executing an inter-frame tracking algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. In this system, the robot controls a camera to capture images of the light reflected from the surface of an object by a line laser emitted by a line laser emitter. The robot can then detect the brightness type of the captured images, specifically distinguishing between bright and dark frames. In some embodiments, as the robot moves towards a predetermined target location, it controls the line laser emitter to emit line laser light and controls the camera to capture images of the light reflected from the surface of the object. The structured light module operates in a specific manner, with the line laser emitter emitting line laser light according to a preset modulation period and emission power level. The camera periodically captures images, obtaining a sequence of images. Each image sequence includes data from at least one frame, and each frame contains a laser line segment formed by the line laser hitting the object's surface or the ground. Each laser line segment contains multiple coordinate data points, and the coordinate data from the laser line segments in a large number of environmental images can form point cloud data.

[0028] Specifically, the method for detecting whether the image captured by the camera is a bright frame image or a dark frame image includes: controlling the line laser emitter to emit line laser according to a preset modulation period. When the line laser is an infrared laser modulation signal, the infrared laser modulation signal outputs a first level (corresponding to a logic high level) in the first modulation sub-cycle. After being reflected by the object under test, it is captured by the camera to form a bright frame image. The infrared laser modulation signal outputs a second level (corresponding to a logic low level) in the first modulation sub-cycle. After being reflected by the object under test, it is captured by the camera to form a dark frame image. Thus, within one sampling cycle, the imaging plane reflected by the camera consists of one bright frame image and one dark frame image. The robot sets an image structure (the structure information of the image data) for each frame image in the images captured by the camera, caches it, and marks the brightness and darkness attributes of the line laser accordingly. Each frame image can be saved as the previous frame image for tracking and matching. Under the robot's configuration, the image sequence formed by the light reflected back from the surface of the object being measured by the line laser emitted by the line laser emitter and in the imaging plane of the camera is configured to alternate between bright frame images and dark frame images, so that: when the current frame image captured by the camera is a bright frame image, the next frame image captured by the camera is a dark frame image; during the time interval between the camera capturing the current bright frame image and the camera capturing the next bright frame image, the camera captures the current dark frame image, and this time interval is equal to one sampling period of the camera; after the camera captures the next bright frame image, the camera captures the next dark frame image. In the laser positioning method, the first frame image of the light reflected back from the surface of the object being tested by the line laser, captured by the camera, is a bright frame image, denoted as the first frame image captured after the line laser emitter emits the line laser, or the first bright frame image. Then, the second frame image of the light reflected back from the surface of the object being tested by the camera is a dark frame image, denoted as the second frame image after the line laser emitter emits the line laser, or the first dark frame image. Next, the third frame image of the light reflected back from the surface of the object being tested by the camera is a bright frame image, denoted as the third frame image after the line laser emitter emits the line laser, or the second bright frame image. Then, the next frame image captured by the camera is a dark frame image. The robot, based on the above alternating generation method, sequentially distinguishes and marks the bright and dark frame images from a sequence of images captured by the camera according to the camera's sampling period.

[0029] Preferably, the distinction between bright and dark frames can be achieved based on the average grayscale value of the images. Specifically, for each frame acquired, all pixels in the frame are traversed, and the sum of the brightness values ​​of all pixels is calculated. Then, the quotient of the sum of brightness values ​​and the number of pixels is taken as the average brightness value of the frame. When the robot detects that the brightness values ​​of a preset threshold number of pixels in the frame are all greater than the average brightness value, the frame is designated as a bright frame, which can be used to search for pixels that meet the optimal convex hull condition in positioning scenarios with strong ambient light, allowing for increased camera exposure. When the robot detects that the brightness values ​​of a preset threshold number of pixels in the frame are all less than the average brightness value, the frame is designated as a dark frame, which can be used to find pixels that meet the optimal convex hull condition in positioning scenarios with weak ambient light, allowing for decreased camera exposure, thus improving the adaptability of the laser positioning method to ambient light intensity. The preset threshold number is greater than or equal to the total number of pixels in the frame.

[0030] In this embodiment, when the robot detects that the current frame image captured by the camera is a bright frame image, the robot executes an inter-frame tracking algorithm to search for the line laser position in the current frame image, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image, so as to realize the positioning of the pixel position of the line laser; wherein, the robot sequentially inputs each frame image captured by the camera into the processing rule model corresponding to the inter-frame tracking algorithm, and outputs the line laser position in the corresponding bright frame image to obtain the laser line segment formed by connecting the various line laser positions, which facilitates the positioning of the object under test; input inter-frame tracking The image in the processing rule model corresponding to the tracking algorithm can be divided into the current frame image and the previous frame image, or the current frame image and the next frame image. The matching relationship between the previous bright frame image and the current bright frame image, and / or the matching relationship between the previous dark frame image and the current bright frame image can be used to track the reflection position of the line laser. In scenarios where the camera is not too close to the obstacle, various ambient light interferences, especially strong ambient light interferences, are effectively filtered out. The algorithm overcomes the influence of the vertical jump of the pixel caused by the change in distance between the camera and the object under test, tends to obtain a more accurate line laser position, and reduces the dependence on infrared filters.

[0031] As one embodiment, the method by which the robot searches for the location of the outgoing laser line in a bright frame image by executing an inter-frame tracking algorithm includes:

[0032] Step 1: The robot traverses the current frame image column by column and obtains the initial pixel position in the current column of the current frame image. Generally, an initial pixel position can be obtained in each column of the current frame image. An initial pixel position serves as the search starting point for searching for pixels that meet the optimal convex hull condition in its column. It should be noted that the initial pixel position is the position of the original pixel in the image captured by the camera after the line laser emitted by the line laser emitter is reflected back to the camera's field of view from the robot's travel plane (generally the ground) when there are no obstacles in front of the robot (or obstacles within the camera's field of view). At this time, the line laser emitter or camera has been calibrated. Preferably, the robot's travel plane can be represented by the surface of the object under test. Each original pixel is a reflection position on the robot's travel plane, used to represent the search starting point for searching the line laser position in each column of the same frame image. The original pixels obtained in the same frame image are preferably located in the same row, which may include adjacent pixels in the same row. The object under test can be an obstacle protruding from the robot's travel plane.

[0033] Simultaneously, pixels that do not have a line laser position in the current frame image are excluded based on the pixels that conform to the preset brightness distribution characteristics in the corresponding column. This is to eliminate the interference of pixels with strong ambient light interference before starting the search for pixels that conform to the convex hull characteristics in the current frame image. The line laser position is used to represent the reflection position of the line laser on the surface of the object to be tested, which is searched by the robot in the current frame image. In this embodiment, pixels that do not have a line laser position in the current frame image are pixels with strong ambient light interference.

[0034] Step 2: In addition to the column containing the pixel where the laser line position is not determined in Step 1, the robot sequentially traverses the relevant columns of the current frame image, specifically traversing the pixels within the column where the initial pixel position exists. In the current column of the current frame image, the robot sets the initial pixel position of the current column as the search center, and then searches for pixels within a search radius from the search center upwards along the current column. Optionally, starting from the initial pixel position of the current column, the robot sets a first preset pixel distance as the search radius, and searches upwards and downwards along the column direction for pixels that conform to the convex hull feature. Here, the current column is the column that the robot is currently traversing, and the search radius is also applicable to defining the coverage area in two opposite column directions of the same column. The two opposite column directions of the same column include the direction of searching upwards along the same column starting from the initial pixel position and the direction of searching downwards along the same column. Then, for a currently determined search center, based on the difference between the brightness values ​​of the upward-searched pixels and the downward-searched pixels in the search states corresponding to the two adjacent determined search centers, and the inter-frame matching relationship formed by the same type of values ​​of the current frame image relative to the reference frame image in the same column of pixels, the convex hull center pixels in the current column are selected. Then, the convex hull center pixels in the current column are updated to replace the convex hull center pixels determined in the current column of the current frame image in the previous time. Whenever the search center in the current column is updated once, the convex hull center pixels set in the current column are also updated once. And every pixel within the search radius relative to the initial pixel position in the current column is traversed and the convex hull center pixels are updated to determine the final convex hull pixels in the current column.

[0035] The reference frame image is configured as a bright frame image containing the latest line laser position found by the robot before the current frame image is acquired. The latest line laser position found by the robot is selected from the convex hull center pixels of the corresponding column. The difference between the brightness values ​​of pixels searched upwards and downwards can be extended to the numerical relationship between the brightness gradients generated in the upward and downward searches. This can be compared with the search states corresponding to two adjacent search centers, which is beneficial for selecting the convex hull center pixels from the searched pixels that meet the convex hull features. Moreover, it is the result of comparing the brightness values ​​of the currently determined search center upwards and downwards with the brightness values ​​of the previously determined search center in the same column. The two adjacent search centers are the currently determined search center and the previously determined search center in the same column. It can be the two adjacent search centers determined by searching the pixels within the search radius in two rounds along the upward direction of the current column from the initial pixel position, and then filtering and updating the convex hull center pixels of the same column; or it can be the two adjacent search centers determined from the initial pixel position. Starting from the current column downwards, the system searches for pixels within the search radius in two consecutive rounds, filtering and updating adjacent search centers determined by the convex hull center pixels of the same column. Each round of searching corresponds to one search center, and also corresponds to a search state within the pixel region of different columns. The update range of the search center is within the coverage area of ​​the search radius relative to the initial pixel position, including the initial pixel position, to facilitate the updating of the convex hull center pixels of the same column, continuously filtering out more accurate convex hull centers to represent the line laser position. The inter-frame matching relationship includes matching the number of pixels and matching the brightness values ​​between two frames of images. These two frames can be adjacent frames or two bright frames separated by one or more frames. Specifically, the matching involved can be based on the changes in brightness values ​​and vertical coordinates of pixels at the same reflection position of the line laser on the surface of the obstacle in images acquired in real time during the robot's movement.

[0036] Based on this, whenever the robot completes step 2 by traversing upwards along the current column from the search center and filtering and updating the convex hull center pixel, and also completes step 2 by traversing downwards along the current column from the search center and filtering and updating the convex hull center pixel, the robot then begins traversing the pixels within the search radius from the initial pixel position of the next column of the current frame image.

[0037] In step 1, when the current frame image is a bright frame image, the previous frame image is a dark frame image. The robot saves the previous frame image. If the robot has already searched for the corresponding line laser position (including the line laser position on the corresponding column or the line laser position on all columns) in the previous frame image before executing the current step 1, then the previous frame image is marked as the reference frame image. Preferably, the reference frame image is configured as a bright frame image where the line laser position found by the robot is located before the current frame image is acquired. The line laser position found by the robot is derived from the convex hull center pixel of the corresponding column. Specifically, within the corresponding column, with the initial pixel position as the center, all pixels within a search radius covering the upward direction of the corresponding column and a search radius covering the downward direction of the corresponding column are sequentially updated to the search center. Then, a new convex hull center pixel is set in step 2.

[0038] Step 3: For the convex hull center pixels that the robot has already selected as belonging to each column of pixels, the robot first determines the convex hull center pixels that belong to interference points based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixels in the current frame image. Then, the interference points are removed from the already selected convex hull center pixels. This can be done by selecting a column-by-column traversal method to remove the interference points existing in the current frame image. After the robot has traversed all columns of pixels in the current frame image to eliminate all interference points, the coordinates of the remaining convex hull center pixels are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. The robot then searches for the determined line laser positions in each column within the current frame image, connecting them to form the laser line segment formed by the line laser emitted by the line laser emitter on the surface of the object under test. This confirms that the robot has successfully searched for the line laser positions in the current frame image by executing an inter-frame tracking algorithm. The line laser positions determined in the same column are those of the last updated convex hull center pixels in that column after the robot has traversed all pixels in that column. The coordinates of a line laser position are represented using the corresponding positioning coordinates. Since the robot obtains the line laser positions in each column by traversing each column in steps 1 to 3, after determining the column number, the coordinates of each line laser position can be represented only by the ordinate value to identify the height information of the line laser's reflection position on the obstacle surface, which can also be used for obstacle avoidance.

[0039] It is worth noting that the selected convex hull center pixel is the last updated convex hull center pixel in each column of convex hull center pixels that exist (can be searched) in the current frame image; the selected convex hull center pixel is the convex hull center pixel in each column of convex hull center pixels that is closest to the origin of the coordinate system of the current frame image. Preferably, the convex hull center pixel may not be updated in every column of the current frame image, so the laser line segment formed at the end is not continuous, which can be used to represent the raised obstacles on the ground where the robot travels.

[0040] In some embodiments, due to multiple reflections of the line laser, multiple pixels conforming to the convex hull feature will be generated in the same column of the current frame image. Two or more convex hull center pixels will be updated simultaneously, and two or more convex hull center pixels will be determined in the current column. Then, the offsets of these convex hull center pixels relative to the origin of the coordinate system of the current frame image are compared, and the convex hull center pixel with the smallest absolute value of the vertical coordinate offset is selected as the final convex hull center pixel determined in the current column. If the vertical coordinate of the origin of the coordinate system of the current frame image represents the vertical coordinate of the robot's travel plane, then the final convex hull center pixel determined in the current column is the convex hull center pixel in the current column of the current frame image that is closest to the ground.

[0041] Specifically, in step 2, except for the column containing pixels where there is no line laser position, the robot sets the initial pixel position obtained in the current column in step 1 as the search center, which is the initial pixel position in step 2 of the aforementioned embodiment. Whenever a convex hull center pixel is selected for a search center, the adjacent pixel searched upwards or downwards along the current column from the search center is updated as the search center, and step 2 is executed again to obtain a new convex hull center pixel and update it as the new convex hull center pixel. Each search center is within a search radius relative to the initial pixel position, limiting the search for convex hull center pixels to the vicinity of the initial pixel position. In this embodiment, the convex hull center pixel is a pixel conforming to the convex hull feature within the search radius. The search radius is set to a first preset pixel distance. Preferably, the first preset pixel distance is less than the maximum pixel distance covered by the current frame image and is within the detection range of the camera. Preferably, the convex hull feature here is a feature used to represent the pattern formed by a line laser hitting the surface of an obstacle. The pixel feature of the pattern's coverage area can be the brightness value of the pixels within the pattern's coverage area, or the brightness value of the pixels within the inscribed circle of the pattern, or the brightness value of the pixels within the circumscribed circle of the pattern.

[0042] In this embodiment, the robot sets the set of pixels conforming to the convex hull feature in the current column of the current frame image as pixels whose brightness values ​​decrease upwards and downwards along the current column starting from the center of the convex hull, and the set of pixels formed by the convex hull center itself, to form a convex hull. This can be regarded as forming a pattern surrounding a laser line segment, used to detect the local effective detection area on the surface of the obstacle where the reflection position of the line laser is located. The convex hull center is the pixel with the largest brightness value in this set of pixels, and the convex hull center pixel is set as belonging to the pixel at the center of the convex hull. Within the set of pixels conforming to the convex hull feature, starting from the center of the convex hull and moving upwards along the same column, the brightness value of the pixel decreases upwards along the current column, generating a first gradient value between the brightness values ​​of two adjacent pixels. Furthermore, starting from the center of the convex hull and moving downwards along the same column, the brightness value of the pixel decreases downwards along the current column, generating a second gradient value between the brightness values ​​of two adjacent pixels, so that the convex hull center belongs to the search center. In this context, within the neighborhood of the convex hull center, there exists at least one pixel whose brightness value is equal to that of the search center. Therefore, within the set of pixels that conform to the convex hull characteristics, starting from the convex hull center, the brightness value decreases upwards along the current column to generate a first gradient value, and the brightness value decreases downwards along the current column to generate a second gradient value. The distance of pixels traversed for the upward decrease from the convex hull center can be less than or equal to the search radius, and the distance of pixels traversed for the downward decrease from the convex hull center can also be less than or equal to the search radius, so as to form a predetermined brightness value gradient change pattern within the convex hull. If the brightness value change pattern of multiple pixels traversed within the same convex hull does not conform to the predetermined brightness value gradient change pattern...

[0043] Preferably, the brightness value of the search center is 255, which is the maximum grayscale value (maximum grayscale level) of the image brightness value divided according to the binarization method. It should be noted that the brightness of a pixel is used to represent the light intensity of light illuminating the surface of the object being measured. When using grayscale values ​​to represent brightness values, the higher the grayscale value, the brighter the image, and the greater the brightness value. The grayscale image formed by image binarization only contains brightness information and does not contain color information, just like a black and white image. The brightness changes continuously from dark to light. Therefore, to represent a grayscale image, the brightness value needs to be quantized, usually divided into 256 levels from 0 to 255. In this embodiment, the value 255 is used to represent a brightness value. When the grayscale value range is 0 to 255, this embodiment also represents the range of pixel brightness values ​​as 0 to 255. Then, each frame of the image captured by the camera can be regarded as converted into a grayscale image, where brightness is grayscale. The larger the grayscale value, the greater the brightness value. The value 0 can represent the darkest pixel, and the value 255 can represent the whitest pixel. The pixel mentioned in this embodiment is an indivisible unit in a frame of image. Each frame of image is composed of many pixels. It exists as a small grid of a single color and can be mapped to a cell (grid) in a raster map. A grayscale image uses one byte to store a pixel.

[0044] As one embodiment, in step 2, for each column of each frame image containing initial pixel positions, excluding columns containing pixels without line laser positions, the method for selecting convex hull center pixels based on the difference between the brightness values ​​of upward-searched pixels and downward-searched pixels in the search states corresponding to two adjacent determined search centers, and the inter-frame matching relationship formed by the current frame image and the same type of values ​​in the same column of pixels in the reference frame image, includes:

[0045] In the current column of the current frame image, as the robot searches upwards or downwards along the column direction for pixels conforming to the convex hull feature, starting from the search center, it compares the brightness value of the search center with the brightness value of the previously searched convex hull center pixel located in the same column. The difference in brightness values ​​can be used to determine the relationship between the two. The previously searched convex hull center pixel located in the same column is the convex hull center pixel selected from the same column of the current frame image based on the previously determined search center. The previously determined search center is a pixel adjacent downwards or upwards to the currently determined search center in the current column of the current frame image. Furthermore, the column order of pixels in the same column of the reference frame image is equal to the column order of the current column of the current frame image. convex hull center pixels in the same column of the reference frame image do not necessarily have the same row order.

[0046] If the robot detects that the brightness value of the search center is greater than the brightness value of the convex hull center pixel in the same column found in the previous search, then in the current column, it searches for pixels upward from the search center and counts the number of pixels whose brightness value decreases according to the first gradient value. That is, whenever a pixel with a decreasing brightness value is found upward, and the brightness value of the currently found pixel is reduced by one of the first gradient values ​​relative to the brightness value of the previously searched pixel (the brightness value of the pixel below the currently found pixel), the count of pixels whose brightness value decreases according to the first gradient value is incremented by one. This can be understood as the robot searching upward along the current column of the current frame image for pixels that conform to the convex hull feature until the upward counting stop condition is met.

[0047] Preferably, the first gradient value adapts to the number of searches. For example, the closer the currently searched pixel is to the upper edge of the convex hull, the larger the first gradient value becomes. Within the same convex hull, the brightness value of pixels closer to the upper edge of the convex hull decreases more drastically, satisfying a predetermined brightness value gradient change law. When the upward counting stop condition is met, the counting of the number of pixels whose brightness value decreases according to the first gradient value stops. At this time, the robot also stops continuing to search for pixels along the column direction, and the number of pixels whose brightness value decreases according to the first gradient value is marked as the upward gradient descent count.

[0048] The robot also searches downwards from the search center for pixels and counts the number of pixels whose brightness values ​​decrease according to the second gradient value. That is, whenever a pixel with a decreasing brightness value is found downwards, and the brightness value of the currently found pixel is reduced by one of the second gradient values ​​relative to the brightness value of the previously searched pixel (the brightness value of the pixel above the currently found pixel), the count of pixels whose brightness values ​​decrease according to the second gradient value is incremented by one. This can be understood as the robot searching downwards along the current column of the current frame image for pixels that conform to the convex hull feature until the downward counting stop condition is met.

[0049] Preferably, the second gradient value adapts to the number of searches. For example, the closer the currently searched pixel is to the lower edge of the convex hull, the larger the second gradient value becomes. Within the same convex hull, the brightness value of pixels closer to the lower edge of the convex hull decreases more drastically, satisfying a predetermined brightness value gradient change law. When the downward counting stop condition is met, the counting of the number of pixels whose brightness value decreases according to the second gradient value is stopped, and the number of pixels whose brightness value decreases according to the second gradient value is marked as the downward gradient descent count.

[0050] After determining to stop searching and counting upwards from the search center along the current column of the current frame image, and determining to stop searching and counting downwards from the search center along the current column of the current frame image, when the robot determines that the number of upward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents counted in the same column of the reference frame image (the column order of the column being compared in the reference frame image is equal to the column order of the current column of the current frame image, so it can also be counted as the current column of the reference frame image), and / or determines that the number of downward gradient descents counted in the current column of the current frame image is greater than or equal to the number of downward gradient descents counted in the same column of the reference frame image (the column order of the column being compared in the reference frame image is equal to the column order of the current column of the current frame image, so it can also be counted as the current column of the reference frame image), it indicates that the robot is approaching the obstacle to be measured, and during this process, the number of pixels conforming to the convex hull feature for characterizing the same local area of ​​the obstacle has increased relative to before the installation height increased, then the number of pixels conforming to the convex hull feature that can be searched in the current frame image is... As the amount of data increases, the robot can detect more detailed parts of obstacles. Based on this, among the pixels traversed in the current column of the current frame image, if it is detected that neither the first gradient value nor the second gradient value is equal to the first preset gradient parameter, and the absolute value of the difference between the first gradient value and the second gradient value is less than the second preset gradient parameter, and the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched upwards along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between the same type of brightness values ​​searched upwards in the same column of the reference frame image, and the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downwards along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between the same type of brightness values ​​searched downwards in the same column of the reference frame image, then the robot marks the currently determined search center as the convex hull center pixel, and determines that the matching relationship formed by the same type of values ​​in the same column of pixels in the current frame image relative to the reference frame image conforms to the expected change in the position of the pixels within the convex hull during the robot's movement.

[0051] Specifically, the absolute value of the difference between the same type of brightness values ​​formed by searching upwards among pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found by searching upwards from the search center in the same column as the current column, and the brightness value of the search center in the same column. The distance between the pixel with the smallest brightness value found by searching upwards and the search center in the same column is less than or equal to the search radius. Furthermore, the absolute value of the difference between the same type of brightness values ​​formed by searching downwards among pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found by searching downwards from the search center in the same column as the current column, and the brightness value of the search center in the same column. The distance between the pixel with the smallest brightness value found by searching downwards and the search center in the same column is less than or equal to the search radius.

[0052] Since the absolute value of the difference between the pixels with the smallest brightness values ​​in opposite directions is increasing, the pixel offset (which can also be understood as the coordinate offset relative to the origin of the coordinate system) of the pixel representing the same reflection position of the line laser increases within the same frame image. This further confirms that as the robot approaches the obstacle to be tested, and during this process, the number of pixels representing the same local area of ​​the obstacle increases relative to before the installation height increases, the number of pixels that can be searched in the current frame image increases, and the robot can detect more details of the obstacle. This proves that the convex hull center pixel among the pixels searched in the current column of the current frame image is a point that relatively accurately represents the laser line segment of the line laser hitting the surface of the obstacle to be tested. This process continues until the convex hull center pixels of all columns in the same frame image are traversed and updated, and the line laser positions in each column are obtained and connected or fitted to represent the line laser, so as to realize the positioning of the obstacle where the laser line segment is located, which facilitates the robot to avoid obstacles in time.

[0053] It should be noted that during step 2, the robot first starts from the search center and searches for pixels sequentially upwards along a column of the current frame image until all pixels within a search radius have been searched upwards along that column. Then, starting from the same search center, it searches for pixels sequentially downwards along a column of the current frame image until all pixels within a search radius have been searched downwards along that column. Alternatively, the robot first starts from the search center and searches for pixels sequentially downwards along a column of the current frame image until all pixels within a search radius have been searched downwards along that column. Then, starting from the same search center, it searches for pixels sequentially downwards along a column of the current frame image until all pixels within a search radius have been searched upwards along a column.

[0054] Preferably, the reference frame is the first frame image of the light reflected back from the surface of the object under test by the line laser, captured by the camera during the execution of the laser positioning method. When the convex hull center pixel in the same column of the reference frame image is located at the initial pixel position in the same column of the reference frame image, the initial pixel position in the same column of the reference frame image is the line laser position in the same column of the reference frame image, and best represents the point of the laser line segment that the line laser hits on the surface of the object under test. Specifically, the first frame image of the light reflected back from the surface of the object under test by the line laser, captured by the camera during the execution of the laser positioning method, is a bright frame image, denoted as the first frame image after the line laser emitter emits the line laser, or simply the first bright frame image.

[0055] It should be noted that the first preset gradient parameter is smaller than the second preset gradient parameter; the first preset gradient parameter is preferably 0 to avoid selecting pixels that conform to the convex hull feature in pixel areas with constant brightness values ​​(such as locally overexposed areas, even though the pixels inside may be the convex hull center); the second preset gradient parameter is preferably 25 to control the coordinate jump of pixels at the same reflection position used to characterize the obstacle within a controllable range, avoiding the introduction of pixels with drastic changes in brightness values, and focusing only on the effective detection area of ​​the obstacle to be tested; this can reduce the search volume and computational load, and also improve the detection accuracy.

[0056] For a currently determined search center, step 2 in the above embodiment further includes stopping conditions for searching pixels along the column direction (the upward counting stopping condition and the downward counting stopping condition), specifically including:

[0057] If the brightness value of the pixel at the search center is greater than the brightness value of the convex hull center pixel in the same column found in the previous search, it indicates that the brightness value of the pixel at the search center (initially the initial pixel position) in the current column of the current frame image is not equal to the brightness value of the reasonable convex hull center found in the previous search, and the difference in brightness values ​​between the two increases as the robot approaches the obstacle. The convex hull center pixel found in the same column in the previous search is the convex hull center pixel selected in the same column of the current frame image based on the previously determined search center. The previously determined search center is a pixel that is adjacent to the currently determined search center in the current column of the current frame image, either downwards or upwards. The robot then begins searching for pixels upwards from the search center in the current column of the current frame image, with the aim of counting pixels that conform to the convex hull feature to filter out the convex hull center pixel in the current column; and searches for pixels downwards from the search center, with the aim of counting pixels that conform to the convex hull feature to filter out the convex hull center pixel in the current column. Specifically, in the current column of the current frame image, the number of pixels whose brightness values ​​decrease according to the first gradient value is counted upwards from the search center. Preferably, starting from the search center, each time a pixel conforming to the convex hull feature is found while searching upwards along the current column, the count is incremented by one to obtain the number of pixels whose brightness values ​​decrease according to the first gradient value. Furthermore, in the current column of the current frame image, the number of pixels whose brightness values ​​decrease according to the second gradient value is counted downwards from the search center. This achieves the goal of searching upwards and downwards along the column direction from the corresponding initial pixel position for pixels conforming to the convex hull feature, respectively. Preferably, starting from the search center, each time a pixel conforming to the convex hull feature is found while searching downwards along the current column, the count is incremented by one to obtain the number of pixels whose brightness values ​​decrease according to the second gradient value.

[0058] In some embodiments, if the robot detects that the brightness value of a pixel does not decrease according to the first gradient value during the upward search from the search center, it counts the preset upward gradient anomaly count once. Then, the robot determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image. If yes, the robot stops searching for pixels along the current column of the current frame image and determines that the upward counting stop condition has been met. Then, it performs step 2 to filter out the convex hull center pixel and starts updating the search center with the adjacent pixel searched from the search center along the current column. Step 2 is then repeated to update the convex hull center pixel. Otherwise, when the upward gradient anomaly frequency is greater than the first preset error count, the robot stops searching for pixels along the current column of the current frame image and determines that the upward counting stop condition has been met. Then, it continues to perform step 2 to filter out the convex hull center pixel and starts updating the search center with the adjacent pixel searched from the search center along the current column. Step 2 is then repeated to update the convex hull center pixel. This process continues until all pixels covered within the search radius have been searched upward relative to the initial pixel position.

[0059] In some embodiments, if the brightness value of a pixel is detected not decreasing according to the second gradient value during the downward search from the search center, the pre-set downward gradient anomaly count is performed once. Then, the robot determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image. If yes, the robot stops searching for pixels along the current column of the current frame image and determines that the downward counting stop condition has been met. Then, according to step 2, based on the difference between the brightness values ​​of the upward-searched pixels and the downward-searched pixels in the search states corresponding to the two adjacent determined search centers, and the inter-frame matching relationship formed by the same type of values ​​of pixels in the same column of the current frame image relative to the reference frame image, the convex hull center pixel is selected. Then, the robot starts searching for the adjacent pixel from the search center along the current column. If the search center is set as the new center, step 2 is repeated to update the convex hull center pixel. Otherwise, if the upward gradient anomaly frequency is greater than the second preset error count, the downward search for pixels along the current column of the current frame image is stopped and the downward counting stop condition is met. Then, according to step 2, based on the difference between the brightness values ​​of the upward-searched pixels and the downward-searched pixels in the search states corresponding to the two adjacent determined search centers, and the inter-frame matching relationship formed by the same type of values ​​of the current frame image and the reference frame image in the same column of pixels, the convex hull center pixel is selected. Then, the adjacent pixel searched upward from the search center along the current column is updated as the search center. Step 2 is repeated to update the convex hull center pixel until all pixels covered within the search radius are searched upward relative to the initial pixel position.

[0060] Regarding the first and second preset error counts, it should be noted that if the pixels searched within the search radius do not conform to the convex hull feature within a certain error allowable range (since the search center is not necessarily the convex hull center), a preset error count needs to be set for judgment. The source of the error is that the reflection position formed by the same emission angle of the line laser collected during robot movement on the same surface of the object under test will change longitudinally, manifested as the pixels representing the same reflection position in different frames shifting upwards along the vertical axis. Therefore, if the robot detects that the brightness value of a pixel does not decrease according to the first gradient value while counting upwards from the search center, and / or detects that the brightness value of a pixel does not decrease according to the second gradient value while counting downwards from the search center, it determines that the gradient value between two adjacent pixels searched along one of the column directions is abnormal, and counts the preset gradient abnormality count once. When the robot detects that the gradient abnormality frequency is greater than the preset error count, and / or has counted all the pixels covered within the search radius, the robot stops counting and stops searching for pixels conforming to the convex hull feature.

[0061] In some embodiments, during the upward search from the search center, the robot counts the pixels with a brightness value of 255 that are adjacent to each other along the current column of the current frame image, and marks the number of pixels with a brightness value of 255 that are adjacent to each other as the upward overexposure count, forming the count of the overexposure area where the pixels at the search center in the upward direction have a continuous brightness value of 255. The robot then determines whether it has searched all the pixels covered within the search radius by moving upwards along the current column of the current frame image. If yes, the robot stops searching for pixels upwards along the current column of the current frame image and confirms that the upward counting stop condition has been met. Then, it executes step 2 to filter out the convex hull center pixel and starts updating the search center with the adjacent pixel found by searching upwards along the current column from the search center. Step 2 is then repeated to update the convex hull center pixel. Otherwise, when the robot detects that the upward overexposure count is greater than the third preset error count, it stops searching for pixels upwards along the current column of the current frame image and confirms that the upward counting stop condition has been met. Then, it continues to execute step 2 to filter out the convex hull center pixel and starts updating the search center with the adjacent pixel found by searching upwards along the current column from the search center. Step 2 is then repeated to update the convex hull center pixel. This process continues until all pixels covered within the search radius have been searched upwards relative to the initial pixel position. Implementation: When the robot detects that the number of upward overexposures is greater than the third preset error count, and / or when the number of pixels covered within the search radius has been counted upwards along the current column of the current frame image, the robot stops searching for pixels upwards along the current column of the current frame image and determines that the upward counting stop condition is met.

[0062] In some embodiments, during the downward search from the search center, the robot counts the pixels with a brightness value of 255 that are adjacent to each other along the current column of the current frame image, and marks the number of pixels with a brightness value of 255 that are adjacent to each other as the number of downward overexposures, forming the count of the number of overexposure areas where the pixels at the search center in the downward direction have a continuous brightness value of 255. The robot then determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image. If yes, the robot stops searching for pixels along the current column of the current frame image and confirms that the downward counting stop condition has been met. Then, it executes step 2 to filter out the convex hull center pixel and starts updating the search center with the adjacent pixel found from the search center along the current column. Step 2 is then repeated to update the convex hull center pixel. Otherwise, when the robot detects that the downward overexposure count is greater than the fourth preset error count, it stops searching for pixels along the current column of the current frame image and confirms that the downward counting stop condition has been met. Then, it continues to execute step 2 to filter out the convex hull center pixel and starts updating the search center with the adjacent pixel found from the search center along the current column. Step 2 is then repeated to update the convex hull center pixel. This process continues until all pixels covered within the search radius have been searched downwards relative to the initial pixel position. Implementation: When the robot detects that the number of upward overexposures exceeds a fourth preset error count, and / or when the number of pixels covered within the search radius has been counted downwards along the current column of the current frame image, the robot stops searching for pixels downwards along the current column of the current frame image and determines that the downward counting stop condition has been met. Therefore, in each round of searching for the convex hull center pixel near the search center, the robot determines the stop search condition by comparing the brightness of pixels searched in opposite directions starting from the search center and counting overexposure and gradient anomalies.

[0063] As one embodiment, in step 3, the method of removing interference points from the already selected convex hull center pixels based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel in the current frame image includes: if the robot has traversed all columns of pixels in the current frame image and obtained the convex hull center pixel from the current frame image, and also has stored the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image, then for each... For each convex hull center pixel, within a circular region centered at the location of the line laser emitted by the line laser emitter in the previous dark image, and with a radius equal to the detection pixel distance, if the robot determines that at least one pixel within this circular region has a brightness value greater than the brightness value of the convex hull center pixel with the same coordinates as the center in the current image by a preset ambient light brightness threshold, then the robot determines that the convex hull center pixel with the same coordinates as the center in the current image is an interference point, and that ambient light interference exists in the vicinity of the convex hull center pixel with the same coordinates as the center in the current image. If the robot cannot find the line laser position at the interference point, then the interference point needs to be removed from the current frame image. The circular region is the effective coverage area corresponding to the positioning coordinates; preferably, the radius of the circular region (detection pixel distance) is not equal to the search radius. During the execution of the inter-frame tracking algorithm, the current frame image is a bright frame image, and the previous frame image is a dark frame image (i.e., the previous dark frame image), and the positioning coordinates in the previous dark frame image (the coordinates of the line laser position determined in one column) are used. In this embodiment, the robot selects the coordinates of the pixel point as the center of the circle in the previous dark frame image. If the coordinates of a convex hull center pixel obtained in the current frame image are equal to the coordinates of the convex hull center pixel with the same coordinates as the center of the circle in the current frame image, then the brightness value can be compared with the brightness value of the convex hull center pixel with the same coordinates as the center of the circle. The preset ambient light brightness threshold is specifically related to the robot's walking speed or rotation speed. Preferably, the greater the robot's walking speed or rotation speed, the more drastic the position jump of the pixel representing the same reflection position in the real-time image acquired by the robot, and the greater the gradient difference between the brightness values ​​of the two pixels. Therefore, the preset ambient light brightness threshold is set to be larger to adapt to the noise reduction accuracy.

[0064] As one embodiment, in step 1, the method of excluding pixels in the current frame image that do not have a line laser position based on pixels in the corresponding column that conform to a preset brightness distribution characteristic includes: if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round by a first preset brightness threshold, or if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round by a second preset brightness threshold, then starting from a position one reference pixel distance upward along the current column of the current frame image from the initial pixel position in the current column of the current frame image, searching for pixels downward along the current column of the current frame image; wherein, the first preset brightness threshold is less than The second preset brightness threshold, the first preset brightness threshold is preferably a value of 10, and the second preset brightness threshold is preferably a value of 235. When the brightness value of the initial pixel in the current column of the current frame image changes little compared to the brightness value generated by the line laser position in the same column found in the previous round, or when the brightness value generated by the line laser position in the same column found in the previous round is large enough to approach the value of 255 (the highest gray level value), the current column may be affected by ambient light. It is necessary to start searching for pixels along the current column of the current frame image from a reference position to exclude pixels with abnormal brightness values ​​or their columns. Moreover, if the sum of the first preset brightness threshold and the second preset brightness threshold is less than the value of 255 (the highest gray level value), then the first preset brightness threshold and the second preset brightness threshold are used as the brightness value judgment conditions for the columns to be excluded by coarse screening.

[0065] Then, during the pixel search process, if the brightness value of the currently searched pixel is detected to be greater than the brightness value of the pixel located at the same line laser position in the previous round by a first preset brightness threshold, or if the brightness value of the currently searched pixel is detected to be equal to the value 255 (the highest grayscale value), then the error position counter is counted once, and it is determined that the currently searched pixel is a pixel that conforms to the preset brightness distribution characteristics. Furthermore, the area near the initial pixel in the current column of the current frame image can have a brightness value greater than the first preset brightness threshold of the pixel located at the same line laser position in the previous round, or it can have a brightness value equal to the value 255 (the highest grayscale value), and is easily affected by ambient light. The reference pixel distance is represented by the number of pixels so that the reference pixel counting threshold is equal to the reference pixel distance.

[0066] When the robot detects that the number of error position gauges is greater than the reference pixel count threshold, it determines that there is no line laser position in the current column of the current frame image. Therefore, the pixels in the current column of the current frame image are set as pixels without line laser positions, and these pixels are excluded from the pixel search range in step 2. Simultaneously, it determines that the ambient light intensity of the robot's environment is greater than a first preset light intensity threshold, indicating strong ambient light interference in the area from a position one reference pixel distance upwards along the current column of the current frame image from the initial pixel position in the current column to the position of the bottom pixel in the current column. The reference pixel count threshold is preferably 25 and is set equal to the reference pixel distance. Therefore, when the number of error position gauges is greater than the position offset of the search starting point of pixels conforming to the preset brightness distribution characteristics relative to the initial pixel position in the same column, it is determined that a line laser position cannot be found in the current column of the current frame image, indicating ambient light interference. This allows for row-by-row comparison of pixels meeting the brightness requirements within the reference test area set in the same column, recording the number of times to determine strong ambient light.

[0067] Preferably, the reference pixel distance is equal to the pixel distance consisting of 25 pixels. Starting from a position 25 pixels above the initial pixel position in the current column of the current frame image, the search for pixels proceeds downwards along the current column of the current frame image until the bottom of the current column is reached. A reference test area is then formed within the current column. Specifically, the area formed by starting from a position one reference pixel above the initial pixel position in the current column of the current frame image and extending downwards along the current column to the bottom of the current column is the reference test area. During the traversal of each pixel in this reference test area, whenever the brightness value of a currently traversed pixel is found to be 10 greater than the brightness value of a pixel located at the same laser position in the same column found in the previous round, or when the brightness value of a currently traversed pixel is found to be equal to 255 (the highest grayscale value), a count is made, and the currently searched pixel is determined to be a pixel conforming to the preset brightness distribution characteristics, until the number of pixels conforming to the preset brightness distribution characteristics exceeds 25. Among them, the sum of the reference pixel distance (or reference pixel count threshold) and the second preset brightness threshold is greater than the value 255 (the highest gray level value), and the sum of the first preset brightness threshold and the second preset brightness threshold is less than the value 255. The comparison shows the change in brightness value of the initial pixel position in the current column of the current frame image relative to the line laser position found in the same column in the previous round, as well as the change in brightness value of the pixel point searched in the current column of the current frame image, reflecting the ambient light intensity of the area corresponding to the current column.

[0068] It should be noted that the line laser position found in the same column in the previous round belongs to the position of the convex hull center pixel finally determined in the same column of pixels of the reference frame image. That is, the line laser position is determined in the same column of pixels of the reference frame image (the position of the convex hull center pixel set after removing interference points in the aforementioned embodiment). The same column of pixels of the reference frame image is the column of pixels in the reference frame image that has the same column order as the current column of the current frame image. Each time the same column of line laser positions is set in a bright frame image, it is recorded as a round of finding the line laser position in the same column of the corresponding image. The search images corresponding to each round are different frame images.

[0069] As one embodiment, in step 1, the method of excluding pixels in the current frame image that do not have a line laser position based on pixels in the corresponding column that conform to the preset brightness distribution characteristics includes:

[0070] Using the initial pixel position in the current column of the current frame image as the center of a ring, the pixels covered by the annular region located below the center of the ring, with an inner diameter of a first positioning radius and an outer diameter of a second positioning radius, are marked as the first pixel to be tested. This is equivalent to: setting a first circle with a first radius equal to the first positioning radius, using the initial pixel position in the current column of the current frame image as the center; and simultaneously setting a second circle with a second radius equal to the second positioning radius, using the initial pixel position in the current column of the current frame image as the center, wherein the first positioning radius is smaller than the second positioning radius; then, below the initial pixel position (in the downward direction of the current column), the pixels covered by the annular region formed by the second circle and the first circle within the current column of the current frame image are marked as the first pixel to be tested.

[0071] Then, the average value of the brightness of the first pixel to be tested is calculated, that is, the ratio of the sum of the brightness values ​​of all the first pixels to be tested in the current column of the current frame image to the total number of the first pixels to be tested in the current column of the current frame image is taken as the average value of the brightness of the first pixel to be tested; wherein, the annular area formed between the first circle and the second circle serves as a transition area for judging changes in light intensity, depending on the setting of the first positioning radius and the second positioning radius. The first positioning radius is preferably 3, and the second positioning radius is preferably 12, which can be expressed by pixel distance, the unit of which is the number of pixels, forming a sufficiently large transition area for judging changes in light intensity.

[0072] If the average brightness value of the first pixel to be tested is greater than the brightness value of the pixel at the same line laser position found in the previous round, then the first pixel to be tested is determined to be a pixel that conforms to the preset brightness distribution characteristics. It is also determined that there is no line laser position in the current column of the current frame image. If there is strong ambient light interference in the reflection area corresponding to the current column of the current frame image, then the pixels in the current column of the current frame image are set as pixels without line laser positions. Then, the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. At the same time, it is determined that the light intensity of the environment where the robot is located is greater than the first preset light intensity threshold. The line laser position found in the same column in the previous round is the position of the convex hull center pixel finally determined in the same column of the reference frame image. Preferably, it is the initial pixel position in the same column of the reference frame image. The same column of the reference frame image is a column with the same column order as the current column of the current frame image.

[0073] Similarly, taking the initial pixel position in the current column of the current frame image as the center of the annulus, in the current column of the current frame image, the pixels covered by the annular region located above the center of the annulus, with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius, are marked as the second pixel to be tested; this is equivalent to: taking the initial pixel position in the current column of the current frame image as the center, setting a first circle with a first radius of the first positioning radius; and simultaneously taking the initial pixel position in the current column of the current frame image as the center, setting a second circle with a second radius of the second positioning radius, wherein the first positioning radius is smaller than the second positioning radius; then, above the initial pixel position (in the upward direction of the current column), the pixels covered by the annular region formed by the second circle and the first circle in the current column of the current frame image are marked as the second pixel to be tested, which is different from the first pixel to be tested.

[0074] Then, the average value of the brightness of the second pixel to be tested is calculated, that is, the ratio of the sum of the brightness values ​​of all the second pixels to be tested in the current column of the current frame image to the total number of the second pixels to be tested in the current column of the current frame image is taken as the average value of the brightness of the second pixels to be tested; wherein, the annular area formed between the first circle and the second circle serves as a transition area for judging changes in light intensity, depending on the setting of the first positioning radius and the second positioning radius. The first positioning radius is preferably 3, and the second positioning radius is preferably 12, which can be expressed by pixel distance, the unit of which is the number of pixels, forming a sufficiently large transition area for judging changes in light intensity.

[0075] If the average brightness value of the second pixel to be tested is greater than the brightness value of the pixel located at the same line laser position found in the previous round, then the second pixel to be tested is determined to be a pixel that conforms to the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels without line laser positions, and the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. At the same time, it is determined that the light intensity of the environment where the robot is located is greater than the first preset light intensity threshold. The line laser position found in the same column in the previous round is the position of the convex hull center pixel finally determined in the same column of the reference frame image. Preferably, it is the initial pixel position in the same column of the reference frame image. The same column of the reference frame image is a column with the same column order as the current column of the current frame image.

[0076] In some embodiments, in step 1, if the initial pixel position cannot be obtained in the current column of the current frame image, the line laser position found in the previous round in the same column is updated to the initial pixel position, and the second preset pixel distance is updated to the search radius. Then, step 2 is repeated. Specifically, the line laser position found in the previous round in the same column is updated to the search center in the current column of the current frame image, and the search radius is set to the second preset pixel distance, which is not equal to the first preset pixel distance. Then, starting from the newly set search center, a pixel within a search radius is searched upwards along the current column, and a pixel within a search radius is searched downwards along the current column starting from the search center. Each pixel within the search radius is traversed until the convex hull center pixel in the corresponding column is found. Specifically, this includes matching the brightness values ​​of the upwardly searched pixels and the downwardly searched pixels in adjacent columns. The differences in the search states corresponding to the determined search centers, and the inter-frame matching relationships formed by the same type of values ​​of the current frame image relative to the reference frame image in the same column of pixels, are used to filter out the convex hull center pixels in the current column. After that, the adjacent pixel searched upwards or downwards from the search center along the current column is updated as the search center, and step 2 is executed again to obtain a new convex hull center pixel and update the new convex hull center pixel. Each search center is within the coverage area of ​​a search radius relative to the initial pixel position. The search radius is set to a second preset pixel distance. So, whenever the search center in the current column is updated once, the convex hull center pixel set in the current column is also updated once. The laser position of the line in the same column found in the previous round is the position of the convex hull center pixel finally determined in the same column of pixels of the reference frame image. Additionally, if the robot fails to find the convex hull center pixel in the same column (the current column of the current frame image) during the repeated execution of step 2, it is determined that the robot cannot find the line laser position in the same column. The pixels in the current column of the current frame image are then excluded from the pixel search range in step 2. At the same time, it is determined that the ambient light intensity of the current column of the current frame image is too high to identify the reflection position of the line laser hitting the object under test.

[0077] In summary, during the laser positioning method for tracking the reflected light of a line laser, when the robot detects that the current frame image captured by the camera is a bright frame image, it selects to input the current frame image into the processing rule model corresponding to the inter-frame tracking algorithm to output an effective laser position. This effectively filters out various ambient light interferences in scenarios where the camera is not too close to obstacles, reducing reliance on infrared filters. Specifically, among pixels conforming to convex hull features, the inter-frame matching is based on the numerical relationship between the brightness gradient generated in upward-searched pixels and the brightness gradient generated in downward-searched pixels, the difference in the search state corresponding to two adjacent determined search centers, the relationship between the brightness value of the currently searched pixel and the brightness value of the convex hull center pixel determined in the same column of the same frame image, and the inter-frame matching formed by the same type of values ​​in the same column of pixels in the current frame image relative to the reference frame image. The algorithm matches the relationships between pixels to select the center pixel of the convex hull in the current column. It then iterates through each pixel within the search radius relative to the initial pixel position in the current column and updates the center pixel of the convex hull. After eliminating interference from pixels that do not have a line laser position, it determines the final convex hull pixel of the current column. This method can more accurately and efficiently determine the information-rich area of ​​the line laser image and reduce the misjudgment of interference points in strong ambient light. It searches for a more accurate center pixel of the convex hull by tracking the number and brightness values ​​of pixels that conform to the convex hull characteristics of the reference frame image. After eliminating all interference points, the coordinates of the remaining center pixels of the convex hull are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. This method can accurately track the reflected light of the laser on the surface of the obstacle within a relatively comprehensive image area, making it suitable for robot navigation and walking scenarios to achieve the effect of robot obstacle localization.

[0078] As one embodiment, the laser positioning method further includes adjusting the exposure information of the camera, specifically including:

[0079] When the robot detects that the ambient light intensity is greater than the first preset light intensity threshold, it indicates that the robot has detected a high intensity of visible light in the current environment, resulting in a relatively high exposure of the camera. The robot then reduces the camera gain (image signal amplification parameter) to obtain the first gain, so that the image of the light reflected back from the surface of the object being tested by the line laser captured by the camera is not overexposed, especially the image information of the visible light portion, thereby improving the accuracy of extracting the aforementioned line laser position in scenes with strong ambient light. The first preset light intensity threshold is mainly a strong light threshold set based on the degree of overexposure of the image captured by the camera due to the strong visible light in the environment.

[0080] When the robot detects that the ambient light intensity is greater than the first preset light intensity threshold, it indicates that the robot has detected a high intensity of visible light in the current environment, resulting in a relatively high exposure of the camera. The robot then reduces the camera's exposure time to obtain the first exposure time, ensuring that the image of the line laser reflected from the surface of the object being tested is not overexposed, especially the visible light portion of the image. This improves the accuracy of extracting the aforementioned line laser position in scenes with strong ambient light. The first preset light intensity threshold is a strong light threshold set based on the degree of overexposure of the image captured by the camera due to the strong visible light in the environment.

[0081] When the robot detects that the light intensity of its environment is less than the second preset light intensity threshold, it means that the intensity of visible light in the current environment is low, and the exposure of the camera becomes relatively small. The robot then increases the gain of the camera (image signal amplification parameter) to obtain the second gain, so that the image of the light reflected back from the surface of the object being tested by the line laser captured by the camera does not appear underexposed (too dark). Preferably, the first gain is less than the second gain; however, if the gain before adjusting to the first gain in the aforementioned embodiment is already large enough to cope with the light intensity of the environment, then the first gain is not necessarily less than the second gain; thereby improving the accuracy of extracting the position of the aforementioned line laser in scenes with low ambient light. The second preset light intensity threshold is mainly a strong light threshold set based on the exposure degree of the image captured by the camera due to the relatively dark visible light in the environment. The second preset light intensity threshold is much smaller than the first preset light intensity threshold.

[0082] When the robot detects that the light intensity of its environment is less than the second preset light intensity threshold, it means that the robot has detected that the intensity of visible light in the current environment is low, and the camera's exposure becomes relatively small. In this case, the robot increases the camera's exposure time to obtain a second exposure time, so that the image of the light reflected back from the surface of the object being tested by the line laser captured by the camera does not appear underexposed. Preferably, the first exposure time is less than the second exposure time; however, if the exposure time before the first exposure time in the aforementioned embodiment is already large enough to cope with the light intensity of the environment, then the first exposure time may not necessarily be less than the second exposure time.

[0083] Therefore, this embodiment adjusts the camera gain and exposure time according to the current environmental conditions so that the image seen by the camera is not overexposed or underexposed, thereby achieving dynamic exposure adjustment of the camera.

[0084] As one example, when the camera's exposure information is used to adjust the power level of a line laser emitter, the following situations exist:

[0085] When the robot detects that the current exposure value of the camera is greater than the first preset exposure threshold, it increases the power level of the line laser emitter to emit the line laser, so that the intensity of the line laser emitted by the line laser emitter is configured to be equal to the product of the smoothing coefficient and the current exposure value.

[0086] In a preferred embodiment, the current exposure value of the camera includes the third gain and / or the third exposure time. When the light intensity of the environment in which the robot is located is greater, the third gain and / or the third exposure time are adjusted to be greater to adapt to the exposure of the current environment light intensity. At this time, the smoothing coefficient is set to a reasonable value to smooth the step size of the exposure value adjustment, thereby suppressing overexposure.

[0087] In the preferred embodiment, the current exposure value of the camera includes the first gain and / or the first exposure time. When the light intensity of the environment in which the robot is located is greater, the first gain and / or the first exposure time adjusted according to the aforementioned embodiment becomes smaller. At this time, the smoothing coefficient is set to a reasonable value to smooth the step size of the exposure value adjustment. This allows the intensity of the line laser emitted by the line laser emitter to be suppressed to become smaller as the first gain and / or the first exposure time becomes smaller, so as to adapt to the exposure amount of the current ambient light intensity.

[0088] Based on the aforementioned preferred embodiment one or preferred embodiment two, the power level of the line laser emitter for emitting the line laser is automatically adjusted until the intensity of the line laser emitted by the line laser emitter (the emission power of the line laser emitter) is equal to the product of the smoothing coefficient and the current exposure value. This enables the use of a stronger line laser level in high-brightness environments and avoids drastic changes in the current exposure value and prevents overexposure of the image of the light reflected back from the surface of the object being tested by the line laser emitted by the line laser emitter, which is captured by the camera. This allows the robot to accurately search for the position of the line laser from the current frame image according to the inter-frame tracking algorithm disclosed in the aforementioned embodiments, ensuring that at least the brightness value of the pixel and the gradient value between two pixels are within a reasonable range. Even when the ambient light is bright, the camera can still capture the image of the reflected light of the line laser on the surface of the obstacle.

[0089] When the robot detects that the current exposure value of the camera is less than the second preset exposure threshold, it lowers the power level of the line laser emitter to emit the line laser, so that the intensity of the line laser emitted by the line laser emitter is equal to the product of the smoothing coefficient and the current exposure value. The first preset exposure threshold is greater than the second preset exposure threshold to reflect the current dim ambient light.

[0090] In the preferred embodiment, the current exposure value of the camera includes the fourth gain and / or the fourth exposure time. Therefore, the lower the light intensity of the environment in which the robot is located, the lower the pre-adjusted third gain and / or the third exposure time will be to adapt to the exposure required by the light intensity of the current environment. At this time, the smoothing coefficient is set to a reasonable value to smooth the step size of the exposure value adjustment, thereby suppressing underexposure.

[0091] In the preferred embodiment, the current exposure value of the camera includes the second gain and / or the second exposure time. Therefore, the lower the light intensity of the environment in which the robot is located, the larger the second gain and / or the second exposure time adjusted according to the aforementioned embodiment becomes. At this time, the smoothing coefficient is set to a reasonable value to smooth the step size of the exposure value adjustment. This can suppress the intensity of the line laser emitted by the line laser emitter from becoming larger when the second gain and / or the second exposure time increases, so as to adapt to the exposure amount of the current ambient light intensity.

[0092] Based on the aforementioned preferred example three or four, the power level of the line laser emitter for emitting the line laser is automatically adjusted until the intensity of the line laser emitted by the line laser emitter (the emission power of the line laser emitter) is equal to the product of the smoothing coefficient and the current exposure value. This avoids drastic changes in the current exposure value and prevents underexposure in the image of the light reflected back from the surface of the object being tested by the line laser emitted by the line laser emitter, which is captured by the camera. This allows the robot to accurately search for the line laser position from the current frame image in a darker environment, and makes the image of the obstacle captured by the camera less overexposed in low ambient light, thus avoiding more reflection interference and facilitating the use of the laser positioning method to find a more accurate line laser position.

[0093] Preferably, the current exposure value of the camera is used to reflect the exposure of the camera in the current lighting environment; the current exposure value of the camera can reflect the ambient light intensity directly or indirectly. For example, the smaller the current exposure value of the camera is adjusted, the stronger the ambient light is, which will guide the power level of the line laser emitter to be increased, so that the reflected light of the line laser on the surface of the obstacle can be collected even when the ambient light is bright.

[0094] In summary, the intensity of the line laser emitted by the line laser emitter describes the mapping relationship between the power level of the line laser emitter and the current exposure value. Combined with the adjustment of the smoothing coefficient, this adapts to the acquisition of image data of the reflected light from the line laser by the structured light module under a series of different exposures. Based on the adjusted camera gain and exposure time, the power level of the line laser emitter can be adjusted to achieve the following: In high-brightness environments, a stronger line laser emission power level is used, ensuring that the line laser is visible even in bright ambient light without overexposure (e.g., in strong outdoor ambient light, the line laser is reflected back to the camera from a white obstacle, reducing the camera gain or exposure time). This avoids overexposure of the image captured by the camera due to excessively bright ambient light, thus finding a more accurate line laser position. In low-brightness environments, a weaker line laser emission power level is used, reducing overexposure of the obstacle image in low ambient light, thus minimizing reflection interference and facilitating a more accurate line laser position (corresponding to the convex hull center).

[0095] Based on the aforementioned embodiments of the laser positioning method, the robot's body is equipped with a structured light module. The structured light module includes a line laser emitter and a camera without an infrared filter. The camera receives light of various wavelengths from the line laser emitted by the line laser emitter without a filter (such as an infrared filter) on its lens, so that the image captured by the camera retains both infrared and visible light imaging information. An internal controller is installed in the robot and electrically connected to the structured light module. The controller is configured to execute the laser positioning method to obtain the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image, i.e., to obtain the line laser position in the bright frame image and the line laser position in the dark frame image; wherein the line laser emitted by the line laser emitter is within the field of view of the camera.

[0096] In this embodiment, the controller can control the operation of the line laser emitter and the camera. Optionally, the controller can control the camera's exposure and also control the line laser emitter to emit line laser light during the camera's exposure period, so that the camera can acquire environmental images detected by the line laser. The controller can control the camera and line laser emitter to operate simultaneously or alternately; this is not limited. It should be noted that the laser light reflected from the object being photographed (the surface of the object under test) is projected onto the photosensitive film through the camera lens, causing a chemical change and producing an image; this process is called exposure.

[0097] Preferably, the controller can employ image processing hardware with FPGA and DSP. To achieve faster processing speeds, considering the significant advantages of FPGA in handling streaming parallel computing, some operations involving morphological image processing are performed in the FPGA, while the remaining operations are performed in the DSP. Even with increased image resolution, the processing time will not increase. At an image size of 2048x2048 pixels, the image processing system can achieve a processing speed of 6 frames / s, simultaneously meeting the accuracy and real-time requirements of robot obstacle avoidance.

[0098] In some embodiments, the horizontal viewing angle of the camera is configured to receive the light reflected back from the line laser within the width of the robot body from in front of the robot, thereby obtaining an environmental image detected by the line laser. To obtain the corresponding horizontal viewing angle of the camera, a wide-angle lens or a non-wide-angle lens can be used, depending on the width of the robot body; it is sufficient to capture the line laser across the entire robot body.

[0099] The mounting height of the structured light module on the robot's body is configured to be positively correlated with the height of the obstacle to be measured, so that the obstacle occupies the effective field of view of the camera. The mounting height of both the line laser emitter and the camera needs to be determined based on the size of the obstacle. A larger mounting height of the structured light module on the robot's body allows for a larger longitudinal space coverage, but it also results in a greater distance from smaller obstacles, leading to fewer captured local details and reduced detection accuracy for smaller obstacles. Conversely, a smaller mounting height of the structured light module on the robot's body allows for a smaller longitudinal space coverage, but it is closer to smaller obstacles, resulting in more captured local details and improved detection accuracy for smaller obstacles.

[0100] Regarding the installation height, the line laser emitter and camera can be located at different heights within the structured light module. For example, on the top of the robot, the line laser emitter may be higher than the camera; or the camera may be higher than the line laser emitter; of course, the line laser emitter and camera can also be at the same height. In practical use, the structured light module is installed on a self-moving robot (such as a robotic vacuum cleaner, patrol robot, or other autonomous mobile device). In this case, the distances between the line laser emitter and the camera and the robot's working surface (such as the ground) are different. For example, the distance from the camera to the working surface is 32mm, and the distance from the line laser emitter to the working surface is 47mm.

[0101] As one embodiment, the upper viewing angle of the camera is configured to cover the bottom of the plane formed by the line laser emitted by the line laser emitter, specifically a laser surface composed of multiple line laser beams, extending and spreading along the emission direction of the line laser emitter and projecting onto the ground on which the robot travels. Preferably, the angle between the laser surface and the working surface of the robot is 15 degrees. The lower viewing angle of the camera is configured to cover the light reflected back from the surface of an obstacle in front of the robot by the line laser emitted by the line laser emitter. Therefore, the pitch angle of the camera can be appropriately adjusted according to the requirements of the map image needed for navigation. The lower viewing angle of the camera is the angle formed by detecting from top to bottom, forming the camera's top-down view. The upper viewing angle of the camera is the angle formed by detecting from bottom to top, forming the camera's bottom-up view. The pitch angle of the camera is divided into the lower viewing angle and the upper viewing angle. Preferably, the upper viewing angle of the camera is set to 24 degrees and the lower viewing angle of the camera is set to 18 degrees.

[0102] And / or the heading angle formed by the deflection of the camera (optical axis of the lens) relative to the central axis of the robot is kept within a preset error angle range, so that the optical axis of the camera is parallel to the direction of travel of the robot, and the camera receives the light reflected back by the line laser within the width of the robot body in front of the robot, so as to detect obstacles directly in front of the robot in real time during the robot's walking process.

[0103] And / or the roll angle generated by the camera rotating along its optical axis is kept within a preset error angle range, so that the camera receives the light reflected back by the line laser within the width of the robot body in front of the robot, wherein the camera is rotatably mounted on the robot body, and the preset error angle range is set to -0.01 to 0.01 degrees, so that the heading angle formed by the deflection of the camera (optical axis of the lens) relative to the central axis of the robot is kept at about 0 degrees, and the roll angle generated by the camera rotating along its optical axis is also kept at about 0 degrees.

[0104] After the structured light module is installed, the angle between the center line of the line laser emitted by the line laser emitter and the installation baseline of the line laser emitter is equivalent to the angle between the laser surface and the working surface of the robot, preferably 15 degrees. The installation baseline refers to the straight line where the line laser emitter is located when the line laser emitter is located at a certain installation height above the robot, or the straight line where the line laser emitter and the camera are located when the line laser emitter and the camera are located at the same installation height.

[0105] In this embodiment, the emission angle of the line laser emitter is not limited. This emission angle is related to the required detection distance of the robot housing the structured light module, the robot's body width, and the mechanical distance between the line laser emitter and the camera. Given that the required detection distance, robot body width, and mechanical distance between the line laser emitter and the camera are determined, the emission angle of the line laser emitter can be directly obtained through trigonometric relationships; that is, the emission angle is a fixed value.

[0106] Of course, if a specific emission angle is required, it can be achieved by adjusting the required detection distance of the robot containing the structured light module and the mechanical distance between the line laser emitter and the camera. In some applications, given a fixed detection distance and robot width, the emission angle of the line laser emitter can be varied within a certain range by adjusting the mechanical distance between the line laser emitter and the camera.

[0107] As one embodiment, a greater installation distance between the camera and the line laser module results in a larger coordinate offset relative to the camera center for the pixels representing the reflection position of the line laser on the obstacle's surface in the image captured by the camera. These pixels, including but not limited to the convex hull center pixels obtained in the aforementioned embodiments, pixels conforming to convex hull features, and the line laser position, contribute to the overall image quality. When the robot approaches an obstacle, a greater installation distance between the camera and the line laser module leads to a greater vertical jump in the pixels within the captured image, thereby acquiring more local detail information and improving obstacle detection accuracy.

[0108] It should be noted that the installation distance refers to the mechanical distance (or baseline distance) between the line laser emitter and the camera. This mechanical distance can be flexibly set according to the application requirements of the structured light module. The size of the measurement blind zone is determined to a certain extent by factors such as the mechanical distance between the line laser emitter and the camera, the required detection distance of the robot housing the structured light module, and the robot's body width. For the robot housing the structured light module, its body width is fixed, while the measurement range and the mechanical distance between the line laser emitter and the camera can be flexibly set according to requirements. This means that the mechanical distance and blind zone range are not fixed values. While ensuring the robot's measurement range (or performance), the blind zone range should be minimized. However, a larger mechanical distance between the line laser emitter and the camera allows for a larger controllable distance range, which is beneficial for better control of the blind zone size and improving obstacle detection accuracy.

[0109] In some applications, structured light modules are used in robotic vacuum cleaners, such as mounting them on the robot's bumper or the robot itself. For robotic vacuum cleaners, the following provides an example of a reasonable range of mechanical distances between the line laser emitter and the camera. For instance, the mechanical distance between the line laser emitter and the camera can be greater than 20mm; further optionally, the mechanical distance between the line laser emitter and the camera can be greater than 30mm. Even further, the mechanical distance between the line laser emitter and the camera can be greater than 41mm. It should be noted that the range of mechanical distances given here is not only applicable to the application of structured light modules in robotic vacuum cleaners, but also to applications of structured light modules in other devices with similar or comparable dimensions to robotic vacuum cleaners.

[0110] As one embodiment, the emission angle of the line laser emitter and the receiving angle of the camera are set as follows: the line laser emitter emits a line laser to a preset detection position in front of the robot body, and the line laser is reflected back to the camera at the preset detection position to form the pixel point conforming to the convex hull feature or the convex hull center pixel point in the image captured by the camera. The length of the laser line segment formed by the line laser at the preset detection position is greater than the width of the robot body. The reflection position of the line laser after hitting the ground depends on the lateral emission angle of the line laser (i.e., the emission angle of the line laser reflector) and the lateral pixel viewing angle of the camera (i.e., the receiving angle of the camera, corresponding to the horizontal viewing angle). The line laser hits in front so that the horizontal length of the line laser extracted by the camera is slightly wider than the width of the robot body.

[0111] Each time the robot travels a preset distance along the direction from its current position to the preset detection position, the horizontal distance between the preset detection position and the robot decreases. In some embodiments, the robot approaches the obstacle at the preset detection position. The coordinate offset of the pixel representing the same reflection position of the line laser at the preset detection position in the image captured by the camera increases relative to the center of the camera. That is, the closer the line laser hits the ground in front of the robot, the larger the vertical jump of the pixel representing the reflection position of the line laser in the image captured by the camera becomes when the robot travels the same distance. More local information is captured, and the detection accuracy for obstacles is higher. Specifically, during robot movement, as the distance between the obstacle and the camera (or the entire structured light module combined with the laser emitter) decreases, the vertical coordinate position of the pixels in the image captured by the camera that reflect the same reflection position of the line laser increases significantly. This increases the number of pixels representing the same local area of ​​the obstacle, potentially reducing the brightness gradient between pixels while maintaining the same number of pixels in the same frame. This improves obstacle detection accuracy. The pixels reflecting the reflection position of the line laser include those conforming to convex hull features. As the robot approaches the obstacle, the position of these pixels changes vertically (vertical coordinate changes), transforming the overall outline of the obstacle in the robot's camera image into a local outline. At least vertically, the height of the obstacle's outline changes, increasing the number of pixels required to represent the local outline before approaching the obstacle, thus improving obstacle detection accuracy. Furthermore, the greater the installation distance between the camera and the line laser emitter in the robot, for example, the greater the installation height of the line laser emitter relative to the camera, the greater the change in the vertical coordinate of the pixels representing the reflection position of the line laser on the surface of the obstacle. The obstacle in the same frame image captured by the camera changes from the original overall outline to a local outline. Therefore, the number of pixels representing the same local area of ​​the obstacle increases compared to before the installation height increases. Alternatively, after the robot travels a certain test distance, the number of pixels representing the same local area of ​​the obstacle in the current frame image captured in real time increases compared to before the installation height increases, improving the accuracy of obstacle detection. Compared to existing technologies, the size of obstacles that the robot can detect can become smaller.

[0112] It should be noted that the pixels conforming to the convex hull feature are configured as part or all of the point information of the laser line segment formed by the simulated line laser projected onto the surface of the object under test; the robot sets the set of pixels conforming to the convex hull feature in the image as pixels whose brightness values ​​decrease from the center of the convex hull along the current column to the upper and lower sides respectively, and the set of pixels formed by the center of the convex hull. This set of pixels forms a convex hull, wherein the center of the convex hull is the pixel with the largest brightness value in the set of pixels; within the set of pixels conforming to the convex hull feature, starting from the center of the convex hull, the brightness value decreases upward along the current column to generate the first gradient value, and the brightness value decreases downward along the current column to generate the second gradient value.

[0113] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0114] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A laser positioning method based on bright frame images, characterized in that, The main body executing the laser positioning method is a robot equipped with a structured light module, which includes a line laser emitter and a camera; The laser positioning method includes: The robot searches for the position of the line laser in the bright frame image by executing an inter-frame tracking algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. Inter-frame tracking algorithms include: The robot traverses the current frame image column by column to obtain the initial pixel position; Pixels in the current frame image that do not have a line laser position are excluded based on the pixels in the corresponding column that conform to the preset brightness distribution characteristics. Based on the difference between the brightness values ​​of the upward-searched pixels and the downward-searched pixels in the search states corresponding to the two adjacent search centers, and the inter-frame matching relationship formed by the same type of values ​​of the current frame image and the reference frame image in the same column of pixels, the convex hull center pixels in the current column are selected. Based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel in the current frame image, interference points are removed from the already selected convex hull center pixels. After the robot has traversed all the convex hull center pixels in the current frame image to remove all interference points, the coordinates of the remaining convex hull center pixels are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image.

2. The laser positioning method according to claim 1, characterized in that, The method by which the robot searches for the location of the outgoing laser line in a bright frame image by executing an inter-frame tracking algorithm includes: Step 1: The robot traverses the current frame image column by column and obtains the initial pixel position in the corresponding column of the current frame image. At the same time, it excludes pixels in the current frame image that do not have a line laser position based on the pixels in the corresponding column that conform to the preset brightness distribution characteristics. The line laser position is used to represent the reflection position of the line laser on the surface of the object to be tested. Step 2: Except for the column containing pixels where the line laser position does not exist, in the current column of the current frame image, the robot sets the initial pixel position of the current column as the search center, and then searches upwards along the current column for pixels within a search radius from the search center, and downwards along the current column for pixels within a search radius from the search center. Then, based on the difference between the brightness values ​​of the upward-searched pixels and the downward-searched pixels in the search states corresponding to the two adjacent determined search centers, and the inter-frame matching relationship formed by the same type of values ​​of pixels in the same column of the current frame image relative to the reference frame image, the convex hull center pixels in the current column are selected to update the convex hull center pixels determined in the current column of the current frame image in the previous step. The reference frame image is configured as the bright frame image where the line laser position was last found by the robot before the current frame image was acquired. Whenever the search center in the current column is updated, the convex hull center pixels set in the current column are also updated. Step 3: Based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel in the current frame image, interference points are removed from the already selected convex hull center pixels. After the robot has traversed all columns of pixels in the current frame image to remove all interference points, the coordinates of the remaining convex hull center pixels are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. Then, the robot searches for the determined line laser position in each column in the current frame image to connect the laser beam formed by the line laser emitted by the line laser emitter on the surface of the object under test. The algorithm identifies the line laser position in the current frame image by executing an inter-frame tracking algorithm. The line laser position determined within the same column is the location of the last updated convex hull center pixel in that column after the robot has traversed all pixels within that column. The coordinates of a line laser position are represented using the corresponding positioning coordinates. The selected convex hull center pixel is the last updated convex hull center pixel in each column of convex hull center pixels in the current frame image. The selected convex hull center pixel is the convex hull center pixel in each column of convex hull center pixels in the current frame image that is closest to the origin of the coordinate system of the current frame image. The current frame image is a bright frame image of the light reflected back from the surface of the object being tested by the line laser emitted by the line laser emitter, which is captured by the robot-controlled camera.

3. The laser positioning method according to claim 2, characterized in that, In step 2, whenever a convex hull center pixel is selected for a search center, the adjacent pixel searched upwards or downwards along the current column from the search center is updated as the search center, and step 2 is executed again to obtain a new convex hull center pixel and update the convex hull center pixel; each search center is within the coverage area of ​​a search radius relative to the initial pixel position, wherein the search radius is set to a first preset pixel distance; Specifically, the robot sets the set of pixels in the current column of the current frame image that conform to the convex hull feature as a set of pixels whose brightness values ​​decrease upwards and downwards along the current column starting from the center of the convex hull, and the set of pixels formed by the center of the convex hull, to form a convex hull. The center of the convex hull is the pixel with the largest brightness value in this set of pixels, and the pixel at the center of the convex hull is set as belonging to the pixel at the center of the convex hull. Within the set of pixels that conform to the convex hull feature, starting from the center of the convex hull and moving upwards along the same column, the brightness value of the pixel decreases upwards along the current column, generating a first gradient value between the brightness values ​​of two adjacent pixels. Furthermore, starting from the center of the convex hull and moving downwards along the same column, the brightness value of the pixel decreases downwards along the current column, generating a second gradient value between the brightness values ​​of two adjacent pixels, so that the center of the convex hull belongs to the search center.

4. The laser positioning method according to claim 3, characterized in that, In step 2, the method for selecting convex hull center pixels based on the difference between the brightness values ​​of upward-searched and downward-searched pixels in the search states corresponding to two adjacent determined search centers, and the inter-frame matching relationship formed by the same type of values ​​of pixels in the same column of the current frame image relative to the reference frame image, includes: In the current column of the current frame image, the brightness value of the search center is compared with the brightness value of the convex hull center pixel in the same column that was searched previously. The convex hull center pixel in the same column that was searched previously is the convex hull center pixel selected in the same column of the current frame image based on the previously determined search center. The previously determined search center is a pixel that is adjacent to the currently determined search center in the current column of the current frame image, either downwards or upwards. The column order of the pixels in the same column of the current frame image is equal to the column order of the current column of the current frame image. If the brightness value of the currently determined search center is greater than the brightness value of the convex hull center pixel found in the same column in the previous search, then in the current column of the current frame image, pixels are searched upwards from the search center, and the number of pixels whose brightness values ​​decrease according to the first gradient value is counted until the upward counting stop condition is met. Then, the number of pixels whose brightness values ​​decrease according to the first gradient value is marked as the upward gradient descent quantity, and the upward search of pixels stops until the search center is updated next time. Furthermore, pixels are searched downwards from the search center, and the number of pixels whose brightness values ​​decrease according to the second gradient value is counted until the downward counting stop condition is met. Then, the number of pixels whose brightness values ​​decrease according to the second gradient value is marked as the downward gradient descent quantity, and the downward search of pixels stops until the search center is updated next time. When the robot determines that the number of upward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents required to find the center pixel of the convex hull in the same column in the previous search, and / or determines that the number of downward gradient descents counted in the current column of the current frame image is greater than or equal to the number of downward gradient descents required to find the center pixel of the convex hull in the same column in the previous search, if the robot detects that among the pixels traversed in the current column of the current frame image, neither the first gradient value nor the second gradient value is equal to the first preset gradient parameter, and the absolute value of the difference between the first gradient value and the second gradient value is less than the second preset gradient parameter, then... Furthermore, if the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched upwards along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between the same type of brightness values ​​formed by searching upwards along the same column of pixels in the reference frame image, and the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downwards along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between the same type of brightness values ​​formed by searching downwards along the same column of pixels in the reference frame image, then the robot marks the currently determined search center as the convex hull center pixel; wherein, the first preset gradient parameter is less than the second preset gradient parameter.

5. The laser positioning method according to claim 4, characterized in that, The absolute value of the difference between the same type of brightness values ​​formed by searching upwards among the pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value found in the reference frame image, starting from the search center finally determined in the column with the same column order as the current column, and the brightness value of the pixel at the search center finally determined in the same column. The distance between the pixel with the smallest brightness value found in the upward search and the search center finally determined in the same column is less than or equal to the search radius. The absolute value of the difference between the same type of brightness values ​​formed by searching downwards among pixels in the same column of the reference frame image is the absolute value of the brightness value of the pixel with the smallest brightness value found in the reference frame image, starting from the search center finally determined in a column with the same column order as the current column, and the brightness value of the pixel at the search center finally determined in the same column. The distance between the pixel with the smallest brightness value found in the downward search and the search center finally determined in the same column is less than or equal to the search radius.

6. The laser positioning method according to claim 4, characterized in that, For a currently identified search center, step 2 further includes: If the brightness value of the pixel at the search center is greater than the brightness value of the convex hull center pixel found in the same column in the previous search, then in the current column of the current frame image, pixels are searched upwards from the search center and downwards from the search center. If, during the upward search from the search center, the robot detects that the brightness value of a pixel does not decrease according to the first gradient value, it counts the preset upward gradient anomaly count once. Then, the robot determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image. If yes, the robot stops searching for pixels upward along the current column of the current frame image and determines that the upward counting stop condition has been met. Otherwise, when the upward gradient anomaly frequency is greater than the first preset error count, the robot stops searching for pixels upward along the current column of the current frame image and determines that the upward counting stop condition has been met. Similarly, if, during the downward search from the search center, the robot detects that the brightness value of a pixel does not decrease according to the second gradient value, it counts the preset downward gradient anomaly count once. Then, the robot determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image. If yes, the robot stops searching for pixels downward along the current column of the current frame image and determines that the downward counting stop condition has been met. Otherwise, when the upward gradient anomaly frequency is greater than the second preset error count, the robot stops searching for pixels downward along the current column of the current frame image and determines that the downward counting stop condition has been met. Alternatively, during the upward search from the search center, the robot counts adjacent pixels with a brightness value of 255 along the current column of the current frame image, and marks the number of adjacent pixels with a brightness value of 255 as the upward overexposure count. When the robot detects that the upward overexposure count is greater than a third preset error count, and / or when it has finished counting all the pixels covered within the search radius along the current column of the current frame image, the robot stops searching for pixels upward along the current column of the current frame image and determines that the upward counting stop condition is met. Furthermore, during the downward search from the search center, the robot counts adjacent pixels with a brightness value of 255 along the current column of the current frame image, and marks the number of adjacent pixels with a brightness value of 255 as the downward overexposure count. When the robot detects that the upward overexposure count is greater than a fourth preset error count, and / or when it has finished counting all the pixels covered within the search radius along the current column of the current frame image, the robot stops searching for pixels downward along the current column of the current frame image and determines that the downward counting stop condition is met.

7. The laser positioning method according to claim 3, characterized in that, In step 3, the method for removing interference points from the already selected convex hull center pixels based on the relationship between the brightness value of the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel in the current frame image includes: After the robot has traversed all columns of pixels in the current frame image and obtained the latest convex hull center pixel in each column, and has saved the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image, for each convex hull center pixel in the current frame image, within a circular region centered at the position of the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image, and with a radius equal to the detection pixel distance, if the robot determines that the brightness value of at least one pixel within this circular region is greater than the brightness value of the convex hull center pixel in the current frame image with the same coordinates as the center of the circle by a preset ambient light brightness threshold, then the robot determines that the convex hull center pixel in the current frame image with the same coordinates as the center of the circle is an interference point. The robot cannot find the line laser position at the interference point and removes the interference point from the current frame image.

8. The laser positioning method according to claim 2, characterized in that, In step 1, the method of excluding pixels in the current frame image that do not have a line laser position based on pixels in the corresponding column that conform to the preset brightness distribution characteristics includes: If the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel at the same laser line position found in the previous round by a first preset brightness threshold, or if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel at the same laser line position found in the previous round by a second preset brightness threshold, then starting from a position one reference pixel distance upwards along the current column of the current frame image, a downward search for pixels is performed along the current column of the current frame image; if it is detected that the brightness value of the currently searched pixel is greater than that of the pixel at the same laser line position found in the previous round... If the brightness value of a pixel located at a line laser position in the same column is greater than a first preset brightness threshold, or if the brightness value of a currently searched pixel is equal to 255, then the error position counter is counted once, and the currently searched pixel is determined to be a pixel that conforms to the preset brightness distribution characteristics. When the robot detects that the error position counter count is greater than the reference pixel count threshold, it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels without line laser positions, and the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. The reference pixel distance is represented by the number of pixels, so that the reference pixel count threshold is equal to the reference pixel distance. Among them, the laser position of the line in the same column found in the previous round is the position of the convex hull center pixel finally determined among the pixels in the same column of the reference frame image.

9. The laser positioning method according to claim 2, characterized in that, In step 1, the method of excluding pixels in the current frame image that do not have a line laser position based on pixels in the corresponding column that conform to the preset brightness distribution characteristics includes: Using the initial pixel position in the current column of the current frame image as the center of a ring, the pixels covered by the annular region located below the center of the ring, with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius, are marked as the first test pixels. Then, the average brightness value of the first test pixels is calculated. If the average brightness value of the first test pixels is greater than the brightness value of the pixel at the line laser position found in the same column in the previous round, it is determined that the first test pixel is a pixel that conforms to the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels without line laser positions, and the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. Here, the first positioning radius is smaller than the second positioning radius, and the line laser position found in the same column in the previous round is the position of the convex hull center pixel finally determined in the same column of the reference frame image. Alternatively, taking the initial pixel position in the current column of the current frame image as the center of a ring, in the current column of the current frame image, the pixels covered by the annular region above the center of the ring, with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius, are marked as the second test pixels. Then, the average brightness value of the second test pixels is calculated. If the average brightness value of the second test pixels is greater than the brightness value of the pixel at the line laser position found in the previous round in the same column, then the second test pixel is determined to be a pixel that conforms to the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels without line laser positions, and the pixels in the current column of the current frame image are excluded from the pixel search range in step 2. Here, the first positioning radius is smaller than the second positioning radius, and the line laser position found in the previous round in the same column is the position of the convex hull center pixel finally determined in the same column of the reference frame image.

10. The laser positioning method according to claim 3, characterized in that, The initial pixel position is the position of the original pixel in the image captured by the camera after the line laser emitted by the line laser emitter is reflected back into the field of view of the camera from the robot's traveling plane, assuming there are no obstacles in front of the robot. Each original pixel is a reflection position on the robot's travel plane, used to represent the search starting point for searching the line laser position in each column of the same frame image; The reference frame image is a bright frame image configured to contain the latest line laser position found by the robot before the current frame image is acquired. The latest line laser position found by the robot is derived from the convex hull center pixel point set in the corresponding column of the reference frame image.

11. The laser positioning method according to claim 10, characterized in that, In step 1, if the initial pixel position cannot be obtained in the current column of the current frame image, the line laser position found in the previous round in the same column is updated to the initial pixel position, and the second preset pixel distance is updated to the search radius. Then, step 2 is repeated to search for the convex hull center pixel in the corresponding column. The line laser position found in the previous round in the same column is the position of the convex hull center pixel finally determined in the same column of the reference frame image or the initial pixel position in the same column of the first bright frame image. If the robot cannot find the convex hull center pixel in the same column during the repeated execution of step 2, it is determined that the robot cannot find the line laser position in the same column.

12. The laser positioning method according to any one of claims 1 to 11, characterized in that, The image sequence formed by the line laser emitted by the line laser emitter and reflected back from the surface of the object under test is configured to alternate between bright and dark frames, such that: when the current frame image captured by the camera is a bright frame image, the next frame image captured by the camera is a dark frame image; during the time interval between the camera capturing the current bright frame image and the camera capturing the next bright frame image, the camera captures the current dark frame image; after the camera captures the next bright frame image, the camera captures the next dark frame image. In the process of executing the laser positioning method, the first frame of the image sequence is a bright frame image.

13. The laser positioning method according to claim 12, characterized in that, The greater the installation distance between the camera and the line laser module, the greater the coordinate offset of the pixel representing the reflection position of the line laser on the surface of the obstacle relative to the center of the camera in the image captured by the camera.

14. The laser positioning method according to claim 12, characterized in that, The emission angle of the line laser emitter and the receiving angle of the camera are set as follows: the line laser emitter emits a line laser to a preset detection position in front of the robot body, and the line laser is reflected back to the camera at the preset detection position. The length of the laser line segment formed by the line laser at the preset detection position is greater than the width of the robot body. Whenever the robot travels a preset distance along the direction from its current position to the preset detection position, the horizontal distance between the preset detection position and the robot decreases, and the coordinate offset of the pixel in the image captured by the camera that represents the same reflection position of the line laser at the preset detection position increases relative to the center of the camera.

Images