Laser ray single-row center point searching method, positioning method and robot

Abstract

The invention discloses a laser line single-row center point searching method, a positioning method and a robot, and the laser line single-row center point searching method comprises the steps: controlling a line laser module to emit a laser line, and collecting a laser image through the line laser module; in a laser image collected by an on-line laser module, a plurality of pixel intervals to be judged are extracted along the same column by taking a ground line reference position point as a search center, and whether an environment reflected by the laser image is in a barrier-free state or not is judged based on gray difference among the plurality of pixel intervals to be judged; if yes, taking the reference position point of the ground line as a search center, extracting an effective search pixel interval along the same column, and processing each pixel point distributed in the effective search pixel interval by using a gray centroid method to obtain a single-column center point of the laser line so as to perform primary positioning on the ground line; and otherwise, processing all pixel points in the same column by using a gray centroid method to obtain a single-column center point of the laser rays so as to carry out primary positioning on the obstacle.

Description

Technical Field

[0001] The present application relates to the technical field of laser data processing, and in particular to a laser line single column center point search method, a positioning method and a robot. Background Art

[0002] Line laser module refers to any laser module that includes a line laser transmitter and a camera module. In a line laser module, the line laser transmitter is used to emit laser lines outward; the camera module can collect environmental images and also receive laser lines reflected by objects. The laser lines can help detect the outline size of raised obstacles on the robot's travel plane and the ground and other information, collectively referred to as the position information of the laser lines.

[0003] The line laser transmitter emits a laser line according to a preset modulation cycle and transmission power level, and the camera module periodically collects images to obtain data of multiple frames of images; each frame of the image contains a laser line segment formed by the laser line hitting the surface of an obstacle or the ground; in each frame of the image, a laser line segment is distributed with multiple coordinate data in columns; if the position of the laser line is determined, the Chinese patent CN115619860A needs to judge the grayscale difference between the pixels on the upper and lower sides of the same column and the inter-frame matching relationship between the pixels in the same column of two frames of images respectively, and on this basis, combined with the grayscale change law predicted in advance, search all the pixels in the same column and continuously update the search center, so the amount of calculation generated is relatively large, and the judgment method is not used to distinguish the obstacle-free position, and the laser line position cannot be determined from the column pixels in a single frame of the image, which is not suitable for robots with low main control computing power and few operating memory resources to perform rapid navigation operations in complex environments (including ground environments and non-ground environments). Summary of the invention

[0004] The present application discloses a laser line single column center point search method, a positioning method and a robot, and the specific technical solutions are as follows: A method for searching the center point of a single column of laser lines, the method comprising: step A, a robot controls a line laser module to emit a laser line, and then collects a laser image through the line laser module; then executes step B; wherein the line laser module stores a ground line reference position point in advance; each column of the laser image is parallel to the width direction of the laser line in the laser image; step B, in the laser image collected by the line laser module, taking the ground line reference position point as the search center, extracting a plurality of pixel intervals to be judged along the same column, and then based on the plurality of pixel intervals to be judged, extracting a plurality of pixel intervals to be judged. Whether the environment reflected by the laser image is in a state without obstacles is judged by the grayscale difference between them. If yes, execute step C, otherwise execute step D; step C, take the ground line reference position point as the search center, extract the effective search pixel interval along the same column, and then use the grayscale centroid method to process each pixel point distributed in the effective search pixel interval to obtain the center point of the laser line single column; step D, use the grayscale centroid method to process all the pixel points in the same column to obtain the center point of the laser line single column; wherein, the center point of the laser line single column represents the center point formed by the laser line in the single column of the laser image.

[0005] In summary, when the present application determines that the environment reflected by the laser image is in a state where there are no obstacles, the application selects the incomplete search of a single column of pixels to calculate the single column center point of a single laser line through step C; when the robot determines that the environment reflected by the laser image is not in a state where there are no obstacles, the application selects the complete search of a single column of pixels to calculate the single column center point of a single laser line through step D; therefore, by selectively executing step C or step D in various environments, the amount of calculation generated by the search is, as a whole, lower than the computing power consumed by uniformly selecting the complete search of a single column of pixels to calculate the single column center point of a single laser line.

[0006] In step C, the present application does not search for the center point of a single column of laser lines by completely processing the entire frame of laser images, but relies on pre-stored ground line reference position points, and preferentially calculates the center point of a single column of laser lines by searching for pixels starting from the vicinity of the ground line reference position points. Compared with the method of searching for a whole column of pixels of a full frame of laser images, the speed of searching and calculating the center point of a single column of laser lines is improved while ensuring the accuracy of identifying and locating the center point of a single column of laser lines.

[0007] Further, the step C includes: in the current search column, taking the ground line reference position point as the search center, starting from the search center, extracting two pixel intervals occupying a preset pixel distance along two opposite directions of the current search column, and then forming the effective search pixel interval with the two pixel intervals occupying the preset pixel distance, so that each pixel point distributed in the effective search pixel interval includes the pixel point distributed at the search center and all pixel points distributed in the two pixel intervals occupying the preset pixel distance except the pixel point distributed at the search center; then using the grayscale centroid method to process each pixel point distributed in the effective search pixel interval to obtain the center point of the laser line single column. Therefore, when the robot determines that the environment reflected in the laser image is in the absence of obstacles, the effective search pixel interval is selected from the single column of pixel points to search and locate a single column of laser line center point, which improves the search efficiency of the single column of laser line center point relative to the grayscale centroid method for weighted average processing of the grayscale values ​​of the entire column of pixel points.

[0008] Furthermore, in step B, in the laser image collected by the online laser module, the method of taking the ground line reference position point as the search center and extracting multiple pixel intervals to be judged along the same column includes: in the current search column, taking the ground line reference position point as the search center, starting from the search center, respectively extracting two pixel intervals occupying a preset pixel distance along two opposite directions of the current search column, and then forming the two pixel intervals occupying the preset pixel distance into an effective search pixel interval, so that each pixel point distributed in the effective search pixel interval includes the pixel point distributed at the search center and all pixel points distributed in the two pixel intervals occupying the preset pixel distance except the pixel point distributed at the search center; in the effective search pixel interval, along the first column direction or the second column direction direction, extract three non-overlapping pixel intervals with equal number of pixels in sequence, and then mark the three non-overlapping pixel intervals with equal number of pixels as the first pixel interval to be determined, the second pixel interval to be determined and the third pixel interval to be determined in the order of extraction, so that the coordinates of each pixel point distributed in the first pixel interval to be determined, the coordinates of each pixel point distributed in the second pixel interval to be determined and the coordinates of each pixel point distributed in the third pixel interval to be determined are arranged along the first column direction or the second column direction according to a fixed coordinate interval; wherein the multiple pixel intervals to be determined are the first pixel interval to be determined, the second pixel interval to be determined and the third pixel interval to be determined; wherein the two opposite directions of the current search column are the first column direction and the second column direction. Thus, in one column of the laser image, the effective search pixel interval is divided into three sections of the pixel intervals to be determined, so as to facilitate the subsequent judgment of the grayscale conditions of the three sections of the pixel intervals to be determined, without the need to conduct a complete search and judgment of the grayscale values ​​of the pixels in a single column, thereby improving the judgment speed.

[0009] Further, in step B, the method for judging whether the environment reflected by the laser image is in a state without obstacles based on the grayscale difference between multiple pixel intervals to be judged includes: calculating the average grayscale value of each pixel point distributed in the first pixel interval to be judged to obtain a first average grayscale value; calculating the average grayscale value of each pixel point distributed in the second pixel interval to be judged to obtain a second average grayscale value; calculating the average grayscale value of each pixel point distributed in the third pixel interval to be judged to obtain a third average grayscale value; judging whether the first average grayscale value is less than the second average grayscale value, and judging whether the second average grayscale value is less than the third average grayscale value. Whether the three average grayscale values ​​are less than the second average grayscale value; when it is judged that the first average grayscale value and the third average grayscale value are both less than the second average grayscale value, it is determined that the environment reflected by the laser image is in a state where there is no obstacle, and it is determined that there is a ground line in the laser image; then the step C is executed; when it is judged that the first average grayscale value is not less than the second average grayscale value, and / or the third average grayscale value is not less than the second average grayscale value, it is determined that the environment reflected by the laser image is not in a state where there is no obstacle, and it is determined that there is an obstacle in the environment reflected by the laser image; then the step D is executed.

[0010] In summary, the present application only needs to extract the average grayscale value of three pixel intervals, perform up and down diffusion judgment with the ground line reference position point as the center, and judge that it is in an obstacle-free state when the average grayscale value of the pixel interval in the middle is respectively greater than the average grayscale value of the pixel interval on both sides thereof. Subsequently, by executing step C, the laser center point feature of the ground line can be identified from the laser image, and the laser line near the ground line is preferentially searched based on the size comparison information between the average grayscale values ​​of the three pixel intervals, without the need to search the grayscale value of the complete column of pixel points of a single frame image.

[0011] In addition, when the average grayscale value of the pixel interval in the middle is not greater than the average grayscale value of the pixel interval on either side thereof, it is determined that an obstacle exists, and the obstacle is detected. Subsequently, step D can be performed to locate the obstacle in front.

[0012] Furthermore, the current search column is parallel to the ordinate axis of the pixel coordinate system where the laser image is located; the two opposite directions of the current search column are respectively the upward direction of the current search column and the downward direction of the current search column; when searching for a pixel point within the preset pixel distance along the current search column upward from the search center, a first detection pixel interval is extracted, so that the pixel points distributed in the first detection pixel interval include the pixel points distributed at the search center and each first pixel point to be processed distributed in sequence along the upward direction of the current search column, and the pixel distance of each first pixel point to be processed relative to the search center does not exceed the preset pixel distance; the first detection pixel interval is a pixel interval that occupies the preset pixel distance along the current search column upward from the search center; when searching for a pixel point within the preset pixel distance along the current search column upward from the search center When the current search column searches downward for a pixel point within the preset pixel distance, a second detection pixel interval is extracted, so that the pixel points distributed in the second detection pixel interval include the pixel points distributed at the search center and the second pixel points to be processed that are sequentially distributed along the current search column downward, and the pixel distance of each second pixel point to be processed relative to the search center does not exceed the preset pixel distance; the second detection pixel interval is a pixel interval that occupies the preset pixel distance along the current search column downward from the search center; wherein the width of the laser line in the laser image is less than the preset pixel distance; the width of the laser line in the laser image is the pixel distance formed in the column direction of the laser image after the laser line is reflected back to the line laser module; the width direction of the laser line in the laser image is parallel to each column of the laser image. Thus, the coverage area in two opposite column directions of the same column is delineated to define the effective search pixel interval.

[0013] Furthermore, the overlapping pixels between the first detection pixel interval and the second detection pixel interval are the pixels distributed at the search center, the number of first pixels to be processed distributed in the first detection pixel interval is equal to the number of second pixels to be processed distributed in the second detection pixel interval, so that the number of pixels distributed in the effective search pixel interval is equal to the sum of twice the number of the first pixels to be processed and the value 1; wherein the pixel distance between two adjacent pixels in the first detection pixel interval is equal to the pixel distance between two adjacent pixels in the second detection pixel interval. Therefore, the first pixel to be processed and the second pixel to be processed are symmetrically distributed in the effective search pixel interval with the search center as the symmetry center.

[0014] Furthermore, the method of using the grayscale centroid method to process the grayscale value of the pixel point to obtain the center point of a single column of the laser line includes: sequentially obtaining the coordinates of each pixel point to be processed in the current search column, and sequentially obtaining the grayscale value of each pixel point to be processed in the current search column; and counting the number of pixel points to be processed in the current search column; wherein the coordinates of the pixel point to be processed include the horizontal coordinate of the pixel point to be processed and the vertical coordinate of the pixel point to be processed, and the horizontal coordinate of the pixel point to be processed is the horizontal coordinate of the current search column; and then, according to the number of pixel points to be processed, the coordinates of the pixel point to be processed and the vertical coordinate of the pixel point to be processed are counted. The products between the gray values ​​are accumulated, and the sum obtained is the first distance value; and the gray values ​​of the pixels to be processed are accumulated according to the number of the pixels to be processed to obtain the second distance value; then the ratio between the first distance value and the second distance value is set as the centroid coordinates in the current search column to obtain a single column center point of the laser line in the current search column; wherein the pixels to be processed in the current search column in step C are the individual pixels distributed in the effective search pixel interval of the current search column; or the pixels to be processed in the current search column in step D are the individual pixels distributed in the current search column.

[0015] A positioning method based on laser lines, the positioning method comprising: executing the laser line single column center point search method to obtain the laser line single column center point in the current search column; then setting the laser line single column center point obtained in step C as the position point of the ground line corresponding to the current search column; or, setting the laser line single column center point obtained in step D as the obstacle corresponding to the current search column.

[0016] The present application sets the center point of a single column of laser lines obtained in step C as the position point of a column of the ground line corresponding to the laser image, thereby realizing a one-time positioning of the ground line. Therefore, under the condition of ensuring the search accuracy, compared with the search and calculation of the entire column of pixel points, the center point of a single column of laser lines is calculated by searching pixel points starting from the vicinity of the ground line reference position point, thereby improving the positioning speed. The present application may also set the center point of a single column of laser lines obtained in step D as the position point of an obstacle corresponding to a column of the laser image, thereby realizing a one-time positioning of the obstacle. Therefore, under the condition of ensuring the search and calculation accuracy, the challenges of identifying and locating the center point of a single column of laser lines in complex scenes (including non-ground scenes) can be addressed, and specifically, obstacles at close range can be positioned.

[0017] Furthermore, whenever the robot obtains the center point of the laser line single column in the current search column, the current search column is marked as the searched column and the next column is searched, and then the next column is updated as the current search column, and then step B is executed; wherein, the next column is a column that has not been searched in the laser image; after the robot searches all the valid search pixel intervals in the laser image, it obtains the center points of all the laser line single columns in the laser image to form the laser line position formed by the projection of the laser line. Whenever the current search column is determined, the search center, the first pixel point to be processed, the second pixel point to be processed and the center point of the laser line single column in the same column are distinguished by selecting the ordinate value or the row subscript value in the single column to identify the reflection position of the laser line on the ground. Since the number of pixels participating in the search in each column of the laser image varies depending on the judgment result of step B, each column does not need to perform a complete search operation, which speeds up the search for the center point of the laser line single column near the ground line to speed up the distinction of the state of whether there is an obstacle.

[0018] A robot, a body assembly line laser module of the robot, the line laser module comprising a line laser emitter and a camera, the line laser emitter is used to emit laser lines toward the ground or the surface of an obstacle; the camera is used to collect images formed by the laser lines reflected by the ground or the surface of an obstacle to obtain a laser image; a controller is arranged inside the robot, and the controller is electrically connected to the line laser module; the controller is configured to execute the laser line single-column center point search method, and / or execute the positioning method, so as to set the laser line single-column center point obtained in step C as the position point of the ground line corresponding to the current search column, and the laser line single-column center point obtained in step D can also be set as the position point of the obstacle corresponding to the current search column. Compared with Chinese patent CN115619860A, the robot disclosed in this application does not need to consider the inter-frame matching relationship between the same column of pixels in two frames of laser images, does not need to predict the grayscale change law of all pixels in the same column in advance, and does not need to continuously update the search center in a single-frame laser image. Instead, it can determine the position of the center point of a single column of laser lines from the local pixel interval in a single-frame image, greatly reducing the amount of calculation consumed by the robot to process laser images column by column. Furthermore, when the search and calculation speed of the center point of the single column of laser lines increases, the speed of the robot to collect and process single-frame laser images is greater, that is, the higher the working frame rate; it is suitable for robots with low main control computing power and few running memory resources to perform rapid navigation operations in complex environments (including ground environments and non-ground environments), and ensure that the line laser module forms a stable working frame rate during the search for the center point of a single column of laser lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention discloses a flow chart of a method for searching the center point of a single column of laser lines according to an embodiment of the present invention.

[0020] Figure 2 Another embodiment of the present application discloses a schematic diagram of sequentially dividing a column of a laser image into a first pixel interval to be determined AB, a second pixel interval to be determined CD, and a first pixel interval to be determined EF. Implementation

[0021] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0022] See also Figure 1 The present application discloses a method for searching the center point of a single row of laser lines, which is applied to a robot of a laser module of an assembly line, and the method for searching the center point of a single row of laser lines comprises: Step A, the robot controls the line laser module to emit laser line, and then collects laser image through the line laser module; then executes step B. Wherein, the line laser module pre-stores the ground line reference position point; the ground line reference position point is the coordinate position point of the ground line obtained by the laser line reflected from the ground after the line laser module is calibrated; each column of the laser image is parallel to the width direction of the laser line in the laser image; specifically, the laser image is the image of the laser stripe formed by the laser line reflected back to the line laser module; each column of the laser image is parallel to the width direction of the laser stripe; preferably, the robot's travel plane can be represented by the ground.

[0023] The ground line is a combination of multiple pixel points formed by the laser line reflected from the ground to the line laser module when there are no obstacles on the robot's traveling plane, that is, when there are no obstacles on the ground in front of the robot, and the calibrated line laser module emits a laser line to the ground in front of it; wherein the multiple pixel points can be regarded as light strips in the laser image, or defined as laser stripes, and one of the multiple pixel points is a ground line reference position point; if the pixel point is represented by the pixel point coordinate system of the laser image, then the pixel coordinate point corresponding to a ground line reference position point in the pixel point coordinate system is a coordinate position point of the ground line; multiple ground line coordinate position points formed by the laser line reflected from the ground are connected in sequence to form a ground line; the ground line can also be understood as a laser stripe formed by the laser line reflected from the ground. In the laser line single column center point search method, a ground line reference position point represents the search starting point / search center for searching the laser line single column center point in a column of the same frame laser image. The ground line corresponds to multiple ground line reference position points, which are distributed in multiple columns in the laser image.

[0024] Before step A, the internal and external parameters of the line laser module are calibrated and calculated. In some calibration scenarios, a calibration plate is set outside the line laser module, and a target is set on the surface of the calibration plate. The target image can be captured by the line laser module, and the image is generally a black and white rectangular pattern; then, by translating the calibration plate and capturing the target image in real time, the rotation matrix is ​​calculated or a general camera calibration model (for example, a distortion correction model) is used or a corresponding laser line equation is fitted (a straight line fitting equation or a surface fitting equation set based on the least squares method) to obtain the internal and external parameters of the line laser module, and the calibration of the line laser module is completed. At this time, the line laser transmitter or camera included in the line laser module has been calibrated. On this basis, when there is no obstacle in front of the robot (or an obstacle within the field of view of the camera), the laser line emitted by the line laser transmitter is reflected back to the field of view of the camera on the ground, and the ground line reference position point in the image captured by the camera is formed, that is, the coordinate position point of the ground line obtained by the line laser module after calibration, which can be saved in the storage medium inside the line laser module. This can predict the approximate range of the laser line's reflection position on the ground and enhance the ability to resist interference from ambient light.

[0025] In step A, for the working scene of the sweeping robot, when there are no obstacles on the ground or obstacles on the ground, the line laser emitter in the line laser module can be controlled to emit laser lines. After the laser lines are emitted to the obstacles on the path of the sweeping robot, they will be reflected back and collected as laser images by the camera in the line laser module. The environmental information reflected by the laser image covers at least the ground medium and the obstacles on the ground, wherein the laser lines will pass through the surface of the obstacle or the ground to form laser line segments in the laser image, that is, the image collected by the camera will contain laser stripes formed by the laser lines emitted by the line laser emitter after encountering objects or the ground, and each laser stripe has multiple pixel points distributed in each column to form a corresponding light strip, which can be searched by subsequent steps. Thus, a search environment is constructed for the center point of a single column of laser lines in the laser image.

[0026] It should be added that the set of ground line reference position points can be a set of points in the pixel point coordinate system (or camera coordinate system) converted from the world coordinate system to the laser point set; the laser point set is a description form of the laser point cloud; each pixel point in the laser image corresponds to a point on the obstacle surface or the ground, and the points on the obstacle surface or the ground corresponding to the pixel points in the laser image constitute the laser point set.

[0027] In the present application, each column of the laser image is parallel to the width direction of the laser line in the laser image, wherein the width direction of the laser line in the laser image is the width direction of the laser stripes formed by the laser line being reflected back to the line laser module; the width direction of the laser stripes is pre-calibrated to be parallel to each column of the laser image through the internal parameter matrix of the line laser module, and the length direction of the laser stripes is perpendicular to the width direction of the laser stripes. The laser stripes are evenly distributed with longitudinal light strips in multiple columns of the laser image, which can be understood as being composed of multiple longitudinal light strips arranged along the length direction of the laser stripes; the center point of the longitudinal light stripes can be calculated by the grayscale centroid method.

[0028] Step B, in the laser image collected by the online laser module, take the ground line reference position point as the search center, extract multiple pixel intervals to be judged along the same column, which can also be understood as setting three or more pixel intervals to be judged in the same column; then judge whether the environment reflected in the laser image is in a state without obstacles based on the grayscale difference between the multiple pixel intervals to be judged. If yes, execute step C, otherwise execute step D. Therefore, when the environment reflected in the laser image is in a state where there are no obstacles, the amount of data consumed in searching for pixels in a single column of pixels and the amount of calculation consumed in calculating the center point of a single column of laser lines are reduced by executing step C, and the calculation here uses the grayscale centroid method by default; or, when the environment reflected in the laser image is not in a state where there are no obstacles, a complete search of the single column of pixels is performed by executing step D, and the center point of the single column of laser lines is calculated using the entire column of pixels. The calculation here uses the grayscale centroid method by default, and the amount of data consumed and the amount of calculation are increased relative to step C. However, by selectively executing step C or step D in various environments, the amount of calculation generated by the search is lower than the amount of calculation generated by searching the center point of the single column of laser lines in each column of the laser image row by row and column by column in the prior art.

[0029] The area covered by the multiple pixel intervals to be judged is smaller than the area completely covered by a single column of pixels (a complete column of pixels). Each time step B is executed, a single column of pixels of a frame of laser image is searched, and the search range is limited to each pixel interval to be judged extracted from the same column, and each pixel interval to be judged is extracted from the same column in sequence along a single direction. Then, based on the grayscale difference between different pixel intervals to be judged, it is judged whether there are points representing obstacles in the laser image, so as to judge whether the environment reflected by the laser image is in a state where there are no obstacles in a local range.

[0030] If it is determined that there is no obstacle, it is determined that there is an obstacle point in the laser image, and then the entire column of pixels needs to be searched to locate the center point of the single column of the laser line.

[0031] If it is determined that there is no obstacle, then it is determined that there is no obstacle on the ground or the low obstacle on the ground is not enough to constitute an obstacle for the robot to cross in the environment reflected by the laser image. The low obstacle here is an obstacle that can be crossed by the robot. Then, the center point of the single column of laser lines can be located without searching the entire column of pixels in the subsequent process, so as to increase the search speed. Therefore, step B is equivalent to a judgment of the obstacle existence state that must be performed before locating the center point of the single column of laser lines, so as to determine the search range of the subsequent single column of pixels.

[0032] Preferably, among the extracted multiple pixel intervals to be determined, the grayscale value of the pixel points in the pixel interval to be determined located in the middle position is the largest, and the pixel points in each pixel interval to be determined distributed from the middle position to both sides thereof gradually become darker.

[0033] Step C, taking the ground line reference position point as the search center, extracting the effective search pixel interval along the same column, which is equivalent to setting the effective search pixel interval in the same column; then using the grayscale centroid method to process each pixel point distributed in the effective search pixel interval, obtaining the center point of the laser line single column, so as to determine the position of the ground line, and can determine the reflection position of the laser line on the ground or the position that can cross the obstacle surface. In step C, the robot has judged that the environment reflected by the laser image is in a state where there is no obstacle, then in a column of pixel points, only the effective search pixel interval is selected for search and the pixel points distributed in various directions are preferentially searched from the ground line reference position point, and then the grayscale values ​​of each pixel point searched in the effective search pixel interval are substituted into the weighted average calculation formula corresponding to the conventional grayscale centroid method, and the grayscale centroid position coordinates are calculated in a single column, and then the center point of the laser line single column is obtained, so that under the condition of ensuring the search accuracy, compared with the search and calculation of the entire column of pixel points, the center point of the laser line single column is calculated by searching the pixel points preferentially from the area near the ground line reference position point, thereby improving the search speed.

[0034] Step D: Use the grayscale centroid method to process all the pixels in the same column to obtain the center point of a single column of laser lines to determine the position of the obstacle; thus, after judging that the environment reflected by the laser image is not in a state where there are no obstacles, that is, when it has been judged that the environment reflected by the laser image is in a state where there are obstacles, under the condition of ensuring the search and calculation accuracy, cope with the challenges brought by complex scenes (including non-ground scenes) in identifying and locating the center point of a single column of laser lines. Specifically, obstacles at close range can be located, but the aforementioned ground lines cannot be located.

[0035] It should be noted that the grayscale centroid method is to search for a single column of pixels in the image and obtain the coordinates of the grayscale center of gravity by weighted averaging the grayscale values ​​of the pixels, that is, the coordinates of the center of gravity position point in the pixel coordinate system. In the partial area (effective search pixel interval) or the entire area of ​​a single column of the laser image, the center of gravity position point calculated by the grayscale centroid method is the center point of the single column of the laser line. The center point of the single column of the laser line represents the center point formed by the laser line in the single column of the laser image. Since the laser line is reflected back to the line laser module to form a laser stripe so that the image of the laser stripe becomes the laser image, the center point of the single column of the laser line represents the center point formed by the laser stripe in the single column of the laser image, that is, in this application, the center point of the single column of the laser line is represented as the center point formed by the laser line in the single column of the laser image, which also belongs to the center point of the aforementioned longitudinal light stripe. The laser lines are distributed in multiple columns of the laser image, and each column is provided with a center point of the single column of the laser line, so a laser line can be searched for multiple center points of the single column of the laser line in the laser image.

[0036] In summary, when the present application determines that the environment reflected by the laser image is in a state where there are no obstacles, the application selects the incomplete search of a single column of pixels to calculate the single column center point of a single laser line through step C; when the robot determines that the environment reflected by the laser image is not in a state where there are no obstacles, the application selects the complete search of a single column of pixels to calculate the single column center point of a single laser line through step D; therefore, by selectively executing step C or step D in various environments, the amount of calculation generated by the search is, as a whole, lower than the computing power consumed by uniformly selecting the complete search of a single column of pixels to calculate the single column center point of a single laser line.

[0037] In step C, the present application does not search for the center point of a single column of laser lines by completely processing the entire frame of laser images, but relies on pre-stored ground line reference position points, and preferentially calculates the center point of a single column of laser lines by searching for pixels starting from the vicinity of the ground line reference position points. Compared with the method of searching for a whole column of pixels of a full frame of laser images, the search speed and calculation speed of the center point of a single column of laser lines are improved while ensuring the recognition and positioning accuracy of the center point of a single column of laser lines.

[0038] Moreover, compared with Chinese patent CN115619860A, the present application does not need to consider the inter-frame matching relationship between the same column of pixels in two frames of laser images, does not need to predict the grayscale change law of all pixels in the same column in advance, and does not need to continuously update the search center in a single frame of laser image. Instead, the present application can determine the position of the laser line from the local pixel interval in the single frame image, greatly reducing the amount of computing power consumed by the robot to process laser images column by column.

[0039] The greater the increase in the search and calculation speed of the center point of a single column of laser lines, the faster the robot can collect and process single-frame laser images, that is, the higher the working frame rate. This is suitable for robots with low main control computing power and few running memory resources to perform fast navigation operations in complex environments (including ground environments and non-ground environments), and to ensure that the line laser module forms a stable working frame rate during the search for the center point of a single column of laser lines.

[0040] As an embodiment, the step C includes: the robot searches the maximum search range of the laser line of the laser image column by column; in the current search column, the ground line reference position point is used as the search center, and starting from the search center, two pixel intervals occupying a preset pixel distance are extracted along two opposite directions of the current search column, generally starting from the search center and searching for pixels row by row along two opposite directions of the current search column and obtaining the grayscale value of the pixels; and then the two pixel intervals occupying the preset pixel distance form the effective search pixel interval, so that each pixel point distributed in the effective search pixel interval includes the pixel point distributed at the search center and all the pixel points distributed in the two pixel intervals occupying the preset pixel distance except the pixel point distributed at the search center. Among them, the current search column is a column that the robot is currently searching in the process of searching the laser image by column; the robot's current search status of the pixel points in each column of the laser image is determined by defining the current search column. Whenever searching in a column, the pixel points are searched by row. The method of searching the laser image by column is preferably: the robot can search the laser image column by column, or search at a certain column interval to save time, and the specific search method can be related to the resolution of the laser image. For example, when the resolution of the laser image is 680X380, the number of pixels occupied by the laser image on the ordinate axis of the pixel coordinate system is 380, that is, the number of complete pixels in a single column is 380; the number of pixels occupied by the laser image on the abscissa axis of the pixel coordinate system is 680, that is, there are 680 columns in the laser image. If the laser image is searched column by column, 680 columns need to be searched, causing the robot to search by column 680 times; if the laser image is searched every other column, 340 columns need to be searched, causing the robot to search by column 340 times.

[0041] Specifically, the current search column is parallel to the width direction of the laser line in the laser image, wherein the width direction of the laser line in the laser image is the width direction of the laser stripes formed by the laser line after reflection, and the width direction of the laser line in the laser image is pre-calibrated by the intrinsic parameter matrix of the line laser module to be parallel to the current search column; preferably, the current search column is parallel to the ordinate axis of the pixel coordinate system where the laser image is located or parallel to the abscissa axis of the pixel coordinate system where the laser image is located; wherein the coordinates of the pixel point include the abscissa of the pixel point and the ordinate of the pixel point; when the row coordinate of the pixel point in the pixel coordinate system is the ordinate of the pixel point, the two opposite directions of the current search column can be set to the upward direction of the current search column and the downward direction of the current search column, respectively belonging to two column directions in opposite directions.

[0042] When searching for pixels within a preset pixel distance along the current search column upward from the search center, a first detection pixel interval is extracted, that is, in the upward direction of the current search column, in the area with a preset pixel distance relative to the search center, each first pixel to be processed is searched in turn, and then combined with the search center to form a first detection pixel interval, so that the pixels distributed in the first detection pixel interval include the pixels distributed at the search center and the first pixel to be processed distributed in turn along the upward direction of the current search column; the pixel distance of each first pixel to be processed relative to the search center does not exceed the preset pixel distance; therefore, the first detection pixel interval is a pixel interval that occupies the preset pixel distance along the current search column upward from the search center, and each pixel in the first detection pixel interval is discretely distributed along the upward direction of the current search column. Preferably, when the laser line is not reflected by an obstacle protruding from the robot's travel plane (regarded as a scenario where the ground reflects the laser line back to the laser module), the grayscale value of each pixel in the first detection pixel interval tends to decrease along the upward direction of the current search column.

[0043] When searching for pixels within a preset pixel distance along the current search column downward from the search center, a second detection pixel interval is extracted, that is, in the downward direction of the current search column, in the area relative to the search center at a preset pixel distance, each second pixel to be processed is searched in turn, and then combined with the search center to form a second detection pixel interval, so that the pixels distributed in the second detection pixel interval include the pixels distributed at the search center and the second pixel to be processed distributed in turn along the downward direction of the current search column, and the pixel distance of each second pixel to be processed relative to the search center does not exceed the preset pixel distance; therefore, the second detection pixel interval is a pixel interval occupying the preset pixel distance along the current search column downward from the search center, and each pixel point in the second detection pixel interval is discretely distributed along the downward direction of the current search column. Preferably, when the laser line is not reflected by an obstacle protruding from the robot's travel plane (regarded as a scenario where the ground reflects the laser line back to the laser module), the grayscale value of each pixel point in the second detection pixel interval tends to decrease along the downward direction of the current search column.

[0044] The two pixel intervals occupying the preset pixel distance are respectively a first detection pixel interval and a second detection pixel interval, which respectively cover the top and the bottom of the effective search pixel interval set in the same column.

[0045] Wherein, the width of the laser line in the laser image (corresponding to Figure 2 The longitudinal distance of the white area in the laser image) is less than the preset pixel distance and also less than the length of the laser line in the laser image (corresponding to Figure 2 The width of the laser line in the laser image is preferably represented by 15 pixels distributed along the column direction.

[0046] It should be noted that the width of the laser line in the laser image is the pixel distance formed in the column direction of the laser image after the laser line is reflected back to the line laser module; the width direction of the laser line in the laser image is parallel to each column of the laser image.

[0047] Schematically, combined Figure 2 It can be seen that when the robot searches for the column where the ground line reference position point O is located, the column where the ground line reference position point O is located is marked as the current search column, and then the ground line reference position point O is set as the search center, corresponding to Figure 2 The black cells are represented by; starting from the ground line reference position point O, the first detection pixel interval is extracted in the upward direction of the current search column. Figure 2 The distance occupied by the inner column along the column direction is Figure 2The straight-line distance OA is calculated, and the second detection pixel interval is extracted from the ground line reference position point O along the downward direction of the current search column. The second detection pixel interval is Figure 2 The distance occupied by the inner column along the column direction is Figure 2 Thus, the coverage areas in two opposite column directions of the same column are delimited to form the pixel interval defined by the box where the dotted line segment AF is located, which is recorded as the effective search pixel interval.

[0048] Then, the grayscale centroid method is used to process each pixel point distributed in the effective search pixel interval to obtain the coordinates of the center point of the single column of the laser line, so as to obtain the center point of the single column of the laser line and be recognized by the robot as the center point of the laser line distribution in the current search column. Preferably, the grayscale centroid method is used to perform weighted average processing on the grayscale values ​​of each pixel point distributed in the effective search pixel interval, and the grayscale center of gravity position coordinates in the effective search pixel interval are calculated, which is recognized by the robot as a center point of a single column of laser lines in the current search column. Therefore, when the robot determines that there are no obstacles in the environment reflected in the laser image, the effective search pixel interval is selected from the single column of pixel points to search and locate a center point of a single column of laser lines, which improves the search efficiency of the center point of a single column of laser lines compared to the grayscale centroid method used to perform weighted average processing on the grayscale values ​​of the entire column of pixels.

[0049] As an embodiment, in step B, in the laser image collected by the online laser module, the method of taking the ground line reference position point as the search center and extracting multiple pixel intervals to be determined along the same column includes: In the current search column, the ground line reference position point is taken as the search center, and starting from the search center, two pixel intervals occupying a preset pixel distance are extracted along two opposite directions of the current search column, and then the two pixel intervals occupying the preset pixel distance are combined into an effective search pixel interval, so that each pixel point distributed in the effective search pixel interval includes the pixel point distributed at the search center and all pixel points distributed in the two pixel intervals occupying the preset pixel distance except the pixel point distributed at the search center. Among them, the current search column is a column that the robot is currently searching in the process of searching the laser image by column; the robot's current search status of the pixel points in each column of the laser image is determined by defining the current search column. Whenever searching in a column, the pixel points can be searched row by row. The method of searching the laser image by column is preferably: the robot can search the laser image column by column, or search at a certain column interval to save time, and the specific search method can be related to the resolution of the laser image. For example, when the resolution of the laser image is 680X380, the number of pixels occupied by the laser image along the ordinate axis of the pixel coordinate system is 380, that is, the number of complete pixels in a single column is 380; the number of pixels occupied by the laser image along the abscissa axis of the pixel coordinate system is 680, that is, there are 680 columns in the laser image. If the laser image is searched column by column, 680 columns need to be searched, causing the robot to search by column 680 times; if the laser image is searched every other column, 340 columns need to be searched, causing the robot to search by column 340 times.

[0050] Specifically, when searching for pixels within the preset pixel distance from the search center upward along the current search column, a first detection pixel interval is extracted, that is, in the upward direction of the current search column, within the area at the preset pixel distance relative to the search center, each pixel point is searched in sequence to form a first detection pixel interval, so that the pixel points distributed in the first detection pixel interval include the pixel points distributed at the search center and the first pixel points to be processed that are distributed in sequence along the upward direction of the current search column, that is, the pixel points distributed at the search center and the first pixel points to be processed that are distributed in sequence along the upward direction of the current search column constitute the first detection pixel interval; the pixel distance of each first pixel point to be processed relative to the search center does not exceed the preset pixel distance; therefore, the first detection pixel interval is a pixel interval that occupies the preset pixel distance from the search center along the current search column upward, and each pixel point in the first detection pixel interval is discretely distributed along the upward direction of the current search column. Combined with Figure 2 It can be seen that when the robot searches for the column where the ground line reference position point O is located, the column where the ground line reference position point O is located is marked as the current search column, and then the ground line reference position point O is set as the search center, corresponding to Figure 2The black cells are represented by; starting from the ground line reference position point O, the first detection pixel interval is extracted in the upward direction of the current search column. Figure 2 The distance occupied by the inner column along the column direction is Figure 2 The straight-line distance OA.

[0051] When searching for pixels within the preset pixel distance starting from the search center and downward along the current search column, a second detection pixel interval is extracted. Specifically, in the downward direction of the current search column, within the area at the preset pixel distance relative to the search center, each pixel point is sequentially searched to form a second detection pixel interval, so that the pixel points distributed in the second detection pixel interval include the pixel points distributed at the search center and the second pixel points to be processed that are sequentially distributed along the downward direction of the current search column, that is, the pixel points distributed at the search center and the second pixel points to be processed that are sequentially distributed along the downward direction of the current search column form the second detection pixel interval; the pixel distance of each second pixel point to be processed relative to the search center does not exceed the preset pixel distance; therefore, the second detection pixel interval is a pixel interval that occupies the preset pixel distance starting from the search center and downward along the current search column, and each pixel point in the second detection pixel interval is discretely distributed along the downward direction of the current search column. Combined with Figure 2 It can be seen that the second detection pixel interval is extracted from the ground line reference position point O along the downward direction of the current search column. Figure 2 The distance occupied by the inner column along the column direction is Figure 2 Thus, the coverage areas in two opposite column directions of the same column are delimited to form the pixel interval defined by the box where the dotted line segment AF is located, which is recorded as the effective search pixel interval.

[0052] Wherein, the width of the laser line in the laser image (corresponding to Figure 2 The longitudinal distance of the white area within the laser image) is less than the preset pixel distance, and the width of the laser line in the laser image is preferably represented by 15 pixels continuously distributed along the column direction.

[0053] It should be noted that the width of the laser line in the laser image is the pixel distance formed in the column direction of the laser image after the laser line is reflected back to the line laser module; the current search column is parallel to the width direction of the laser line in the laser image, which can be understood as the width direction of the laser line in the laser image is parallel to each column of the laser image; wherein, the width direction of the laser line in the laser image is the width direction of the laser stripes formed by the reflection of the laser line, and the width direction of the laser line in the laser image is pre-calibrated by the intrinsic parameter matrix of the line laser module to be parallel to the current search column; preferably, the current search column is parallel to the ordinate axis of the pixel coordinate system where the laser image is located or the abscissa axis of the pixel coordinate system where the laser image is located.

[0054] In this embodiment, the two pixel intervals occupying the preset pixel distance are the first detection pixel interval and the second detection pixel interval. Therefore, in this embodiment, the two opposite directions of the current search column are the first column direction and the second column direction. Preferably, the upward direction of the current search column is the first column direction, and the downward direction of the current search column is the second column direction.

[0055] In the effective search pixel interval, three non-overlapping pixel intervals with equal number of pixels are extracted in sequence along the first column direction or the second column direction, and then the three non-overlapping pixel intervals with equal number of pixels are marked as the first pixel interval to be determined, the second pixel interval to be determined and the third pixel interval to be determined in the order of extraction, so that the coordinates of each pixel point distributed in the first pixel interval to be determined, the coordinates of each pixel point distributed in the second pixel interval to be determined and the coordinates of each pixel point distributed in the third pixel interval to be determined are arranged along the first column direction or the second column direction according to a fixed coordinate interval; wherein the multiple pixel intervals to be determined are the first pixel interval to be determined, the second pixel interval to be determined and the third pixel interval to be determined, so that the effective search pixel interval is divided into three sections of the pixel intervals to be determined; there is a certain pixel interval between two adjacent sections of the pixel intervals to be determined. In the same column, the second pixel interval to be determined is located between the first pixel interval to be determined and the third pixel interval to be determined, so that the second pixel interval to be determined is located in the middle area of ​​the effective search pixel interval.

[0056] exist Figure 2In the illustrated embodiment, in the process of extracting multiple pixel intervals to be determined, starting from the pixel point at the top position point A of the effective search pixel interval, the pixel points are searched along the second column direction until the pixel point at position point B is searched, and 6 pixel points are selected in sequence (which can be obtained by counting the searched pixel points) to form the first pixel interval to be determined; then starting from the next pixel point, that is, the pixel point at position point C, the pixel points are continued to be searched along the second column direction until the pixel point at position point D is searched, and 6 pixel points are selected in sequence (which can be obtained by counting the searched pixel points) to form the second pixel interval to be determined; then starting from the next pixel point, that is, the pixel point at position point E, the pixel points are continued to be searched along the second column direction until the pixel point at position point F is searched, and 6 pixel points are selected in sequence (which can be obtained by counting the searched pixel points) to form the third pixel interval to be determined. At this point, the pixel point at the bottom position point F of the effective search pixel interval has been searched in the current search column. Therefore, in a column of the laser image, the effective search pixel interval is divided into three segments of the pixel intervals to be judged, so as to facilitate the subsequent judgment of the grayscale conditions of the three segments of the pixel intervals to be judged, without the need to perform a complete search and judgment on the grayscale values ​​of a single column of pixels, thereby improving the judgment speed.

[0057] In the step B, the method for judging whether the environment reflected by the laser image is in a state where there are no obstacles based on the grayscale difference between the plurality of pixel intervals to be judged includes: Calculate the average grayscale value of each pixel point distributed in the first pixel interval to be judged to obtain a first average grayscale value. The specific implementation is to calculate the ratio between the grayscale value sum of all pixel points in the first pixel interval to be judged and the number of all pixel points in the first pixel interval to be judged in the current search column, as the first average grayscale value, to characterize the grayscale distribution characteristics in the first pixel interval to be judged.

[0058] Calculate the average grayscale value of each pixel point distributed in the second pixel interval to be determined to obtain a second average grayscale value; specifically, calculate the ratio between the grayscale value and the number of all pixel points in the second pixel interval to be determined in the current search column as the second average grayscale value to characterize the grayscale distribution characteristics in the second pixel interval to be determined.

[0059] Calculate the average grayscale value of each pixel point distributed in the third pixel interval to be determined to obtain a third average grayscale value; specifically, calculate the ratio between the grayscale value and the number of all pixels in the third pixel interval to be determined in the current search column as the third average grayscale value to characterize the grayscale distribution characteristics in the third pixel interval to be determined.

[0060] Then, it is determined whether the first average grayscale value is less than the second average grayscale value, and it is determined whether the third average grayscale value is less than the second average grayscale value. When it is determined that both the first average grayscale value and the third average grayscale value are less than the second average grayscale value, that is, when it is determined that the average grayscale value of the second pixel interval to be determined is respectively greater than the average grayscale values ​​of the pixel intervals on both sides thereof, it is determined that the environment reflected by the laser image is in a state where there are no obstacles, and it is determined that there is a ground line in the laser image, and then the step C is performed to identify the laser center point feature of the ground line from the laser image to realize the positioning of the ground line; wherein, the environment reflected by the laser image includes the ground environment.

[0061] When it is determined that the first average grayscale value is not less than the second average grayscale value, and / or the third average grayscale value is not less than the second average grayscale value, it is determined that the environment reflected by the laser image is not in a state where there are no obstacles, then it is determined that there are obstacles in the environment reflected by the laser image, and the ground line cannot be effectively extracted from the laser image; then step D is executed.

[0062] In summary, the present embodiment only needs to extract the average grayscale value of the three pixel intervals, perform the up and down diffusion judgment with the ground line reference position point as the center, and judge that it is in an obstacle-free state when the average grayscale value of the pixel interval in the middle is respectively greater than the average grayscale value of the pixel interval on both sides thereof. Subsequently, by executing step C, the laser center point feature of the ground line can be identified from the laser image, and then the laser line near the ground line is preferentially searched based on the size comparison information between the average grayscale values ​​of the three pixel intervals, without the need to search the grayscale value of the complete column of pixel points of a single frame image.

[0063] In addition, when the average grayscale value of the pixel interval in the middle is not greater than the average grayscale value of the pixel interval on either side thereof, it is determined that an obstacle exists, and the obstacle is detected. Subsequently, step D can be performed to locate the obstacle in front.

[0064] On the basis of extracting the first detection pixel interval and the second detection pixel interval in the aforementioned embodiment, the overlapping pixels between the first detection pixel interval and the second detection pixel interval are the pixels distributed at the search center, the number of the first to-be-processed pixels distributed in the first detection pixel interval is equal to the number of the second to-be-processed pixels distributed in the second detection pixel interval, so that the number of pixels distributed in the effective search pixel interval is equal to the sum of twice the number of the first to-be-processed pixels and the value 1 or the sum of twice the number of the second to-be-processed pixels and the value 1; wherein, the pixel distance between two adjacent pixels in the first detection pixel interval is equal to the pixel distance between two adjacent pixels in the second detection pixel interval, and preferably, each pixel is searched in accordance with a preset pixel step in both the first detection pixel interval and the second detection pixel interval. The first to-be-processed pixels and the second to-be-processed pixels obtained thereby are symmetrically distributed in the effective search pixel interval with the search center as the symmetric center.

[0065] Preferably, the number of first pixels to be processed distributed in the first detection pixel interval is equal to 10, which can be regarded as starting from the search center along the current search column in the upward direction, continuously adding 10 ordinates or 10 subscripts to obtain 10 first pixels to be processed; the number of second pixels to be processed distributed in the second detection pixel interval is equal to 10, which can be regarded as starting from the search center along the current search column in the downward direction, continuously subtracting 10 ordinates or 10 subscripts to obtain 10 second pixels to be processed; and then taking into account a pixel point distributed at the search center, a total of 21 pixels are obtained in the effective search pixel interval. Therefore, by taking 10 pixels up and down from the ground line reference position point, the row search range of the center point of the single column of the laser line is limited to 21 rows but not the row range covered by the entire column, corresponding to step C is to expand the pixel search range of the ground line in a single column from the traditional single row to the coverage range of the first detection pixel interval and the coverage range of the second detection pixel interval, so as to overcome the interference of ambient light intensity to obtain the effective search pixel interval. Furthermore, the grayscale values ​​of 21 pixels within the pixel interval whose row search range is limited to 21 rows are substituted into the weighted average calculation formula involved in the grayscale centroid method. Compared with directly calculating 380 pixels in a single column, calculating 21 pixels can obtain the same result, while the amount of calculated data is nearly 20 times less, thereby improving the search speed of the center point of a single column of laser lines.

[0066] In combination with the embodiment corresponding to the aforementioned step B, it can be known that when it is judged that the first average grayscale value and the third average grayscale value are both smaller than the second average grayscale value, that is, when it is judged that the average grayscale value of the second pixel interval to be judged is respectively greater than the average grayscale values ​​of the pixel intervals on both sides thereof, it is determined that there is a ground line in the laser image; since the first detection pixel interval is searched upward along the current search column starting from the search center, and the second detection pixel interval is searched downward along the current search column, therefore, if the grayscale value of each second pixel point to be processed shows a decreasing trend in the downward direction of the current search column within the second detection pixel interval, and the grayscale value of each first pixel point to be processed shows a decreasing trend in the upward direction of the current search column within the first detection pixel interval, then the grayscale value change within the first detection pixel interval and the grayscale value change within the second detection pixel interval can comprehensively reflect the grayscale value distribution characteristics of the pixel points near the ground line in the local area.

[0067] It should be noted that each pixel point distributed in the effective search pixel interval is represented by a square in each column of the laser image, so that the pixel distance is represented by the number of pixel points; when the size of the square is larger, the resolution of the laser image is lower, the distance between two adjacent pixel points is larger, and the number of pixel points used to represent the same pixel distance becomes smaller; when the size of the square is smaller, the resolution of the laser image is higher, the distance between two adjacent pixel points is smaller, and the number of pixel points used to represent the same pixel distance becomes larger.

[0068] If the grayscale value range of the pixel is also expressed as a value of 0 to 255, each frame of laser image captured by the line laser module can be regarded as converted into a grayscale image, where grayscale is grayscale, and the larger the grayscale value, the larger the grayscale value, where the value 0 can represent the blackest pixel and the value 255 can represent the whitest pixel.

[0069] The pixel mentioned in this application is an indivisible unit in a frame of laser image. Each frame of laser image is composed of a large number of pixel points. It exists as a square of a single color. The grayscale image uses a capacity of one byte to store one pixel point.

[0070] As an embodiment, the method of using the grayscale centroid method to process the grayscale value of the pixel point to obtain the center point of a single column of the laser line includes: sequentially obtaining the coordinates of each pixel point to be processed in the current search column, and sequentially obtaining the grayscale value of each pixel point to be processed in the current search column; and counting the number of pixel points to be processed in the current search column; wherein, the pixel points to be processed in the current search column in step C are the pixel points distributed in the effective search pixel interval of the current search column; or the pixel points to be processed in the current search column in step D are the pixel points distributed in the current search column. The coordinates of the pixel point to be processed include the horizontal coordinate of the pixel point to be processed (equivalent to the column coordinate) and the vertical coordinate of the pixel point to be processed (equivalent to the row coordinate), and the horizontal coordinate of the pixel point to be processed is the horizontal coordinate of the current search column.

[0071] Then, the product between the coordinates of the pixel points to be processed and the grayscale value of the same pixel points to be processed is accumulated according to the number of the pixel points to be processed to obtain a first distance value, wherein the coordinates of the pixel points to be processed can be recorded as a weight, so that the first distance value is a weighted sum value representing the grayscale value of the pixel points to be processed; and the grayscale values ​​of the pixel points to be processed are accumulated according to the number of the pixel points to be processed to obtain a second distance value; then the ratio between the first distance value and the second distance value is set as the centroid coordinates in the current search column as a result of the weighted average of the grayscale values ​​of the pixel points to be processed, so as to obtain a single column center point of the laser line in the current search column; Then, the coordinates of the center point of a single column of laser lines are composed of the horizontal coordinate of the current search column and the center coordinates of the current search column, thereby obtaining the center point of the single column of laser lines in the current search column and forming a center point of the distribution of the laser lines in the current search column, that is, the center point of the light stripes distributed in the current search column.

[0072] Specifically, when the current search column is parallel to the ordinate axis of the pixel coordinate system where the laser image is located, the product between the ordinate of the pixel to be processed and the grayscale value of the same pixel to be processed is accumulated according to the number of pixel points to be processed to obtain a first distance value; and the grayscale values ​​of the pixel points to be processed are accumulated according to the number of pixel points to be processed to obtain a second distance value; then the ratio between the first distance value and the second distance value is set as the ordinate of the center of mass in the current search column; and then the abscissa of the current search column (a predetermined fixed coordinate value) and the ordinate of the center of mass of the current search column (corresponding to the center of mass coordinate) are combined to form the coordinates of the center point of a single column of laser lines, thereby obtaining the center point of a single column of laser lines in the current search column and forming a center point of the distribution of the laser lines in the current search column.

[0073] Alternatively, when the current search column can be selected to be parallel to the horizontal axis of the pixel coordinate system where the laser image is located, the product between the horizontal coordinate of the pixel to be processed and the grayscale value of the same pixel to be processed is accumulated according to the number of pixel points to be processed to obtain a first distance value; and the grayscale values ​​of the pixel points to be processed are accumulated according to the number of pixel points to be processed to obtain a second distance value; then the ratio between the first distance value and the second distance value is set as the horizontal coordinate of the center of mass in the current search column; and then the horizontal coordinate of the center of mass of the current search column (corresponding to the center of mass coordinate) and the vertical coordinate of the current search column (predetermined fixed coordinate value) are combined to form the coordinates of the center point of a single column of laser lines, thereby obtaining the center point of the single column of laser lines in the current search column and forming a center point of the distribution of the laser lines in the current search column.

[0074] Based on the aforementioned embodiments, the present application also discloses a laser line-based positioning method, which includes: executing the laser line single-column center point search method disclosed in the aforementioned embodiments to obtain the laser line single-column center point in the current search column, including obtaining the laser line single-column center point from step C to determine a position point of the ground line when it is determined in step B that the environment reflected by the laser image is in a state where there are no obstacles, or obtaining the laser line single-column center point from step D to determine a position point of the obstacle when it is determined in step C that the environment reflected by the laser image is not in a state where there are no obstacles.

[0075] Then, the center point of the single column of laser lines obtained in step C is set as the position point of the ground line corresponding to the current search column; or, the center point of the single column of laser lines obtained in step D is set as the position point of the obstacle corresponding to the current search column. Among them, the center point of the single column of laser lines represents the center point formed by the laser lines in the single column of the laser image. The current search column is the column that the robot is currently searching in the process of searching the laser image by column; the current search column is defined to determine the robot's current search status for the pixel points in each column of the laser image. Whenever searching in a column, search the pixel points by row.

[0076] Specifically, in step C, the robot has determined that the environment reflected by the laser image is in a state where there are no obstacles. Then, in a column of pixels, only the effective search pixel interval is selected for search and the pixels distributed in various directions are preferentially searched starting from the reference position point of the ground line. Then, the grayscale values ​​of each pixel searched in the effective search pixel interval are substituted into the weighted average calculation formula corresponding to the conventional grayscale centroid method, and the grayscale centroid position coordinates are calculated in a single column to obtain a laser line single column center point. Then, the laser line single column center point obtained in step C is set as the position point of the ground line corresponding to a column of the laser image, so as to realize a one-time positioning of the ground line. Therefore, under the condition of ensuring the search accuracy, compared with the search and calculation of the entire column of pixels, the laser line single column center point is preferentially calculated by searching pixels starting from the vicinity of the reference position point of the ground line, thereby improving the positioning speed.

[0077] When step D has determined that the environment reflected by the laser image is in a state where there are obstacles, the grayscale centroid method is directly used to process all the pixels in the same column to obtain a single-column center point of the laser line, and then the single-column center point of the laser line obtained in step D is set as the position point of the obstacle corresponding to a column of the laser image, so as to realize one-time positioning of the obstacle. In this way, under the condition of ensuring the search and calculation accuracy, the challenges brought by complex scenes (including non-ground scenes) in identifying and locating the single-column center point of the laser line can be dealt with. Specifically, obstacles at close range can be positioned, but the aforementioned ground lines cannot be positioned.

[0078] It should be noted that the grayscale centroid method searches for pixels in a single column of the image and obtains the coordinates of the grayscale center of gravity by weighted averaging the grayscale values ​​of the pixels, that is, the coordinates of the center of gravity position point in the pixel coordinate system. In a partial area (effective search pixel interval) or the entire area of ​​a single column of the laser image, the center of gravity position point calculated by the grayscale centroid method is the center point of the single column of the laser line. Therefore, in this application, the center point of the single column of the laser line is represented as the center point formed by the laser line in the single column of the laser image, which is also the result of the column search. The laser lines are distributed in multiple columns of the laser image, and each column is set with the center point of the single column of the laser line. Therefore, a plurality of center points of the single column of the laser line can be searched for one laser line in the laser image.

[0079] Therefore, compared with Chinese patent CN115619860A, this application does not need to consider the inter-frame matching relationship between the same column of pixels in two frames of laser images, does not need to predict the grayscale change law of all pixels in the same column in advance, and does not need to continuously update the search center in a single-frame laser image. Instead, it can determine the position of the laser line from the local pixel interval in the single-frame image, greatly reducing the amount of calculation consumed by the robot to process the laser image column by column. The greater the increase in the search and calculation speed of the center point of a single column of laser lines, the faster the robot collects and processes single-frame laser images, that is, the higher the working frame rate; it is suitable for robots with low main control computing power and few running memory resources to perform rapid navigation operations in complex environments (including ground environments and non-ground environments), and ensure that the line laser module forms a stable working frame rate during the search for the center point of a single column of laser lines.

[0080] In some embodiments, whenever the robot obtains the center point of the single column of laser lines in the current search column, the current search column is marked as a searched column and the next column is searched to implement a column-switch search, and then the next column is updated as the current search column, and step B is executed again. Then, a new center point of the single column of laser lines is obtained in the next column by executing steps B to D; wherein the next column is a column that has not been searched in the laser image, and preferably the next column is an adjacent column to the current search column.

[0081] For the same frame of the laser image, the robot obtains the center point of the laser line in each column by repeatedly executing steps B to D; specifically, in the process of repeatedly executing steps B to D, the robot calculates the center point of the laser line in each column of the laser image search, which can form the laser stripes formed by the laser line emitted by the line laser module reflected on the obstacle surface or the ground, and can be further reflected back to the line laser module to form a laser line segment in the laser image, including the ground line or the obstacle contour line. Therefore, every time the current search column is determined, the search center, the first pixel point to be processed, the second pixel point to be processed and the center point of the laser line in the same column are distinguished by selecting the ordinate value or subscript value to identify the reflection position of the laser line on the ground. Since the number of pixels participating in the search in each column of the laser image varies depending on the judgment result of step B, each column does not need to perform a complete search operation, which speeds up the search for the center point of the laser line in the vicinity of the ground line to speed up the distinction between the presence or absence of obstacles.

[0082] After the robot searches all valid search pixel intervals in the laser image, it obtains the center points of all laser lines in the laser image to form the laser line position formed by the projection of the laser line, so as to form the ground line in step C or calculate the position of the obstacle in step D. Specifically, the process in which the robot searches all valid search pixel intervals in the laser image includes searching the laser image column by column, or searching at a certain column interval (it can be a search of every other column); wherein, in each column search, steps B to D are executed to obtain the center points of the laser lines in the corresponding column, so as to obtain the center points of the laser lines in each column within the maximum search range, which form the effective laser line position formed by the projection of the laser line in the pixel coordinate system; and it is determined that the robot has searched the ground line position or obstacle position from the laser image by executing the grayscale centroid method in the appropriate pixel interval.

[0083] In some embodiments, the center point of each column of laser lines relatively accurately represents a point of a laser line segment where the laser line hits the surface of an obstacle or the ground. After obtaining the center points of each column of laser lines in the same frame of laser image, the center points of each column of laser lines can be connected or fitted into a laser line segment representing the laser line to achieve the positioning of the obstacle or the ground where the laser line segment is located, thereby facilitating the robot's timely obstacle avoidance navigation.

[0084] The present application also discloses a chip storing a computer program. When the computer program is executed by the chip, it implements the method for searching the center points of a single column of laser lines as disclosed in the aforementioned embodiment, that is, executing steps A, B, C and D to obtain the center points of a single column of laser lines in different columns; and / or implements the positioning method disclosed in the aforementioned embodiment, which can use the coordinates of the center points of a single column of laser lines in different columns to locate ground lines or obstacles. Therefore, when the environment reflected in the laser image is in a state where there are no obstacles, the amount of data consumed in searching for pixels in a single column of pixels and the amount of calculation consumed in calculating the center point of a single column of laser lines are reduced by executing step C, and the calculation here uses the grayscale centroid method by default; or, when the environment reflected in the laser image is not in a state where there are no obstacles, a complete search of the single column of pixels is performed by executing step D, and the center point of the single column of laser lines is calculated using the entire column of pixels. The calculation here uses the grayscale centroid method by default, and the amount of data consumed and the amount of calculation are increased relative to step C. However, by selectively executing step C or step D in various environments, the amount of calculation generated by the search is lower than the amount of calculation generated by searching the center point of the single column of laser lines in each column of the laser image row by row and column by column in the prior art.

[0085] Based on the aforementioned embodiments, a robot is also disclosed, the body of the robot is assembled with a line laser module; the line laser module includes a line laser emitter and a camera, both of which are fixedly mounted on the robot; the line laser emitter is used to emit a laser line toward the ground or the surface of an obstacle, and the laser line is reflected on the surface of the obstacle or the ground to form a laser point set, which may be a laser point set obtained after the line laser emitter is calibrated; the laser point set may be regarded as being located in the intersection of the emitted laser plane and the obstacle surface or the ground or in the vicinity of the intersection, and if the laser point set originates from the intersection between the laser plane and the ground, it may be regarded as the ground line disclosed in the aforementioned embodiments; the camera is used to capture an image formed by the reflection of the laser line from the ground or the obstacle surface to obtain a laser image, and an image containing pixel information of the laser stripes may be obtained, and the image data is obtained after the camera is calibrated.

[0086] A controller is arranged inside the robot, and the controller is electrically connected to the line laser module; the controller is configured to execute the laser line single-column center point search method disclosed in the aforementioned embodiment, and can calculate the laser line single-column center point column by column in the laser image, and then obtain the center point coordinates of the laser line emitted by the line laser transmitter in the laser image; and / or execute the positioning method to set the laser line single-column center point obtained in step C as the position point of the ground line corresponding to the current search column, and can also set the laser line single-column center point obtained in step D as the position point of the obstacle corresponding to the current search column.

[0087] Compared with Chinese patent CN115619860A, the robot disclosed in this application does not need to consider the inter-frame matching relationship between the same column of pixels in two frames of laser images, does not need to predict the grayscale change law of all pixels in the same column in advance, and does not need to continuously update the search center in a single-frame laser image. Instead, it can determine the position of the center point of a single column of laser lines from the local pixel interval in a single-frame image, greatly reducing the amount of calculation consumed by the robot to process laser images column by column. Furthermore, when the search and calculation speed of the center point of the single column of laser lines increases, the speed of the robot to collect and process single-frame laser images is greater, that is, the higher the working frame rate; it is suitable for robots with low main control computing power and few running memory resources to perform rapid navigation operations in complex environments (including ground environments and non-ground environments), and ensure that the line laser module forms a stable working frame rate during the search for the center point of a single column of laser lines.

[0088] In addition, since the robot is moving in the working posture, the grayscale value distribution characteristics of the same column of pixels in each frame of laser image collected by the camera are not necessarily the same. Therefore, the position of the center point of a single column of laser lines in the same column of each frame of laser image also changes dynamically.

[0089] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable technicians familiar with the technical field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for searching the center point of a single laser line, characterized in that: The laser line single column center point search method comprises: Step A, the robot controls the line laser module to emit a laser line, and then collects a laser image through the line laser module; then executes step B; wherein the line laser module pre-stores a ground line reference position point; each column of the laser image is parallel to the width direction of the laser line in the laser image; Step B, in the laser image collected by the online laser module, taking the ground line reference position point as the search center, extracting multiple pixel intervals to be judged along the same column, and then judging whether the environment reflected by the laser image is in a state where there is no obstacle based on the grayscale difference between the multiple pixel intervals to be judged, if yes, executing step C, otherwise executing step D; Step C: Taking the ground line reference position point as the search center, extracting the effective search pixel interval along the same column, and then using the grayscale centroid method to process each pixel point distributed in the effective search pixel interval to obtain the center point of a single column of the laser line; Step D: Use the grayscale centroid method to process all pixels in the same column to obtain the center point of a single column of laser lines; The center point of a single column of laser lines refers to the center point formed by the laser lines in a single column of the laser image.

2. The method for searching the center point of a single column of laser lines according to claim 1, characterized in that: The step C comprises: In the current search column, the ground line reference position point is taken as the search center, and starting from the search center, two pixel intervals occupying a preset pixel distance are extracted along two opposite directions of the current search column, and then the two pixel intervals occupying the preset pixel distance form the effective search pixel interval, so that each pixel point distributed in the effective search pixel interval includes the pixel point distributed at the search center and all pixel points distributed in the two pixel intervals occupying the preset pixel distance except the pixel point distributed at the search center; Then, the grayscale centroid method is used to process each pixel point distributed in the effective search pixel interval to obtain the center point of a single column of laser lines.

3. The method for searching the center point of a single column of laser lines according to claim 1, characterized in that: In the step B, in the laser image collected by the online laser module, the method of taking the ground line reference position point as the search center and extracting multiple pixel intervals to be determined along the same column includes: In the current search column, the ground line reference position point is taken as the search center, and starting from the search center, two pixel intervals occupying a preset pixel distance are extracted along two opposite directions of the current search column, and then the two pixel intervals occupying the preset pixel distance form an effective search pixel interval, so that each pixel point distributed in the effective search pixel interval includes the pixel point distributed at the search center and all pixel points distributed in the two pixel intervals occupying the preset pixel distance except the pixel point distributed at the search center; In the effective search pixel interval, three non-overlapping pixel intervals with an equal number of pixel points are sequentially extracted along the first column direction or the second column direction, and the three non-overlapping pixel intervals with an equal number of pixel points are respectively marked as a first pixel interval to be determined, a second pixel interval to be determined, and a third pixel interval to be determined in the order in which they are extracted, so that the coordinates of each pixel point distributed in the first pixel interval to be determined, the coordinates of each pixel point distributed in the second pixel interval to be determined, and the coordinates of each pixel point distributed in the third pixel interval to be determined are arranged along the first column direction or the second column direction at fixed coordinate intervals; The plurality of pixel intervals to be determined are respectively a first pixel interval to be determined, a second pixel interval to be determined and a third pixel interval to be determined; The two opposite directions of the current search column are respectively the first column direction and the second column direction.

4. The method for searching the center point of a single column of laser lines according to claim 3, characterized in that: In step B, the method for judging whether the environment reflected by the laser image is in a state where there are no obstacles based on the grayscale difference between the plurality of pixel intervals to be judged includes: Calculate the average grayscale value of each pixel point distributed in the first pixel interval to be determined to obtain a first average grayscale value; Calculate the average grayscale value of each pixel point distributed in the second pixel interval to be determined to obtain a second average grayscale value; Calculate the average grayscale value of each pixel point distributed in the third pixel interval to be determined to obtain a third average grayscale value; Determine whether the first average grayscale value is less than the second average grayscale value, and determine whether the third average grayscale value is less than the second average grayscale value; When it is determined that the first average grayscale value and the third average grayscale value are both smaller than the second average grayscale value, it is determined that the environment reflected by the laser image is in a state where there is no obstacle, and it is determined that there is a ground line in the laser image; then the step C is executed; When it is determined that the first average grayscale value is not less than the second average grayscale value, and / or the third average grayscale value is not less than the second average grayscale value, it is determined that the environment reflected by the laser image is not in a state where there are no obstacles, then it is determined that there are obstacles in the environment reflected by the laser image; then step D is executed.

5. The method for searching the center point of a single column of laser lines according to claim 2 or 3, characterized in that: The two opposite directions of the current search column are respectively an upward direction of the current search column and a downward direction of the current search column; When searching for a pixel point within the preset pixel distance from the search center along the current search column upward, extracting a first detection pixel interval, so that the pixel points distributed in the first detection pixel interval include the pixel points distributed at the search center and the first pixel points to be processed that are distributed in sequence along the current search column upward, and the pixel distance of each first pixel point to be processed relative to the search center does not exceed the preset pixel distance; the first detection pixel interval is a pixel interval occupying the preset pixel distance from the search center along the current search column upward; When searching for a pixel point within the preset pixel distance from the search center along the current search column downward, extracting a second detection pixel interval, so that the pixel points distributed in the second detection pixel interval include the pixel points distributed at the search center and each second pixel point to be processed distributed in sequence in the downward direction along the current search column, and the pixel distance of each second pixel point to be processed relative to the search center does not exceed the preset pixel distance; The second detection pixel interval is a pixel interval starting from the search center and occupying the preset pixel distance downward along the current search column.

6. The method for searching the center point of a single column of laser lines according to claim 5, characterized in that: The pixels overlapping between the first detection pixel interval and the second detection pixel interval are the pixels distributed at the search center, the number of first pixels to be processed distributed in the first detection pixel interval is equal to the number of second pixels to be processed distributed in the second detection pixel interval, and the number of pixels distributed in the effective search pixel interval is equal to the sum of twice the number of the first pixels to be processed and 1; The pixel distance between two adjacent pixels in the first detection pixel interval is equal to the pixel distance between two adjacent pixels in the second detection pixel interval.

7. The method for searching the center point of a single column of laser lines according to claim 5, characterized in that: Methods for processing the grayscale values ​​of pixels using the grayscale centroid method to obtain the center points of a single column of laser lines include: The coordinates of each pixel to be processed in the current search column are sequentially obtained, and the grayscale value of each pixel to be processed in the current search column is sequentially obtained; and the number of the pixel to be processed in the current search column is counted; Then, according to the number of pixels to be processed, the products of the coordinates of the pixels to be processed and the grayscale values ​​of the same pixels to be processed are accumulated, and the sum obtained is the first distance value; and according to the number of pixels to be processed, the grayscale values ​​of the pixels to be processed are accumulated to obtain the second distance value; Then, the ratio between the first distance value and the second distance value is set as the centroid coordinate in the current search column to obtain a single column center point of the laser line in the current search column; Among them, the pixel points to be processed in the current search column in step C are the pixel points distributed in the effective search pixel interval of the current search column; or the pixel points to be processed in the current search column in step D are the pixel points distributed in the current search column.

8. A positioning method based on laser line, characterized in that: The positioning method comprises: Execute the laser line single column center point search method according to any one of claims 1 to 4 to obtain the laser line single column center point in the current search column; Then, the center point of the single column of laser lines obtained in step C is set as the position point of the ground line corresponding to the current search column; or, the center point of the single column of laser lines obtained in step D is set as the position point of the obstacle corresponding to the current search column.

9. The positioning method according to claim 8, characterized in that: Whenever the robot obtains the center point of a single column of laser lines in the current search column, the current search column is marked as a searched column and the next column is searched, and then the next column is updated as the current search column, and step B is performed again; wherein the next column is a column that has not been searched in the laser image; After the robot has searched all valid search pixel intervals in the laser image, the center points of all single columns of laser lines in the laser image are obtained to form the laser line positions formed by the projection of the laser lines.

10. A robot, the robot body is equipped with a line laser module, the line laser module includes a line laser transmitter and a camera, the line laser transmitter is used to transmit a laser line toward the ground or the surface of an obstacle; the camera is used to collect an image formed by the laser line reflected by the ground or the surface of an obstacle to obtain a laser image; a controller is arranged inside the robot, and the controller is electrically connected to the line laser module; characterized in that, The controller is configured to execute the laser line single column center point search method described in any one of claims 1 to 7, and / or execute the positioning method described in any one of claims 8 to 9.

Images