Method and robot for extracting center point of laser stripe in search area

Abstract

The present application discloses a method and robot for extracting the center point of a laser stripe within a search partition. The extraction method includes: step A, selecting an initial search partition, and then extracting the starting center point based on the grayscale centroid method; step B, setting a search partition adjacent to the initial search partition as the current search partition along a first preset search direction; then setting the current search center point within the current search partition based on the starting center point; then setting a current search sub-partition along a second preset search direction and its opposite direction with the current search center point as the center; then extracting the center point of the laser stripe within the current search sub-partition based on the grayscale centroid method; then executing step C; step C, updating the current search partition to the initial search partition, and updating the center point of the laser stripe within the current search sub-partition to the starting center point, and then executing step B. This method eliminates the need to search the pixels of the laser image pixel by pixel when executing the grayscale centroid method, thereby improving the efficiency of extracting the center point of the laser stripe.

Description

Technical Field

[0001] The present application relates to the technical field of laser data processing, and in particular to a method and robot for extracting the center point of a laser stripe within a search partition. Background Art

[0002] Extracting the center of mass (also known as the center of gravity) of an object is a fundamental task with wide applications in computer vision, image processing, and target tracking. It is crucial for tasks such as object detection, tracking, recognition, and measurement. The center of mass of an object is often used to describe its position and posture for subsequent analysis and processing.

[0003] Chinese patent CN115619860A does not directly solve the laser line positioning problem within a single frame image, but tracks the reflected position of the laser line through the matching relationship between the previous frame image and the current frame image; in the process of tracking the reflected position of the laser line, when updating a search center in a single column, it is easy to make the coordinate span between the two search centers (search starting points) before and after the update unclear, resulting in the ranges of two adjacent searches mostly overlapping, and time is spent on processing repeated areas. In fact, the pixel width of the light strip in the image is only a very small part relative to the width or length of the image. If the pixel points of the entire image are traversed row by row and column by column, a large amount of resources will be consumed to process invalid pixels, which is not convenient for real-time extraction of the center points of the light strips in each column.

[0004] Since the pixel width of the laser light stripe in the image accounts for only a very small part (5% to 10%) of the width or length of the image, the grayscale value of the laser light stripe is much larger than the grayscale value of the background area in which it is located. Therefore, when the camera is not equipped with an infrared filter, the operation of screening the center points of each column of light stripes by using the rule of decreasing grayscale values ​​on both sides of the search center in Chinese patent CN115619860A is subject to background interference.

[0005] Chinese patent CN115619860A can only execute the grayscale centroid algorithm in some image frames with low brightness. The main principle is to determine the center of gravity of the object based on the grayscale value of the pixel points in the image. Although this algorithm can achieve sub-pixel accuracy, it requires a global traversal of the pixels of the entire image, for example, starting from the pixels in the top row of the current column, and traversing to the pixels in the bottom row of the current column, and starting from the pixels in the leftmost column of the current row, and traversing to the pixels in the rightmost column of the current row. The pixel width of the laser light strip in the actual image only accounts for a very small part of the width or length of the image (accounting for 5% to 10%), so a lot of resources are wasted on processing invalid pixels, which is not convenient for extracting the center point of each column of light strips in real time. Summary of the Invention

[0006] This application aims to disclose a method and robot for extracting the center point of a laser stripe within a search zone. The specific technical solutions include:

[0007] A method for extracting the center point of a laser stripe in a search partition, wherein a robot controls a line laser transmitter to emit a laser line, and a robot controls a camera to collect an image of a laser stripe formed by reflection of the laser line, and then marks the image of the laser stripe as a laser image; the laser image is evenly divided into multiple search partitions along the length direction of the laser stripe, so that the laser stripe is divided into a single light stripe partition within a single search partition; the length direction of the laser stripe is a first preset search direction or its opposite direction; the extraction method comprises: step A, selecting an initial search partition from the multiple search partitions divided in the laser image, and then extracting the center of gravity position point of the initial search partition based on the grayscale center of gravity method, and then marking the center of gravity position point of the initial search partition as the starting center point, and determining that the initial search partition has been searched; and then executing step B; within the initial search partition, the distribution direction of the pixel points is pre-calibrated to be parallel to the width direction of the laser stripe; the length direction of the laser stripe is perpendicular to the width direction of the laser stripe; step B, setting the search partition adjacent to the initial search partition along the first preset search direction Set as the current search partition; then set the current search center point in the current search partition based on the starting center point; then set the current search sub-partition along the second preset search direction and the opposite direction with the current search center point as the center; then extract the center of gravity position point of the current search sub-partition based on the grayscale centroid method, and then mark the center of gravity position point of the current search sub-partition as the light stripe center point of the laser stripe in the current search sub-partition, and then determine that the current search partition has been searched and determine that the light stripe center point is extracted from the current search partition; then execute step C; wherein the second preset search direction is perpendicular to the first preset search direction; step C, update the current search partition to the initial search partition, and update the light stripe center point of the laser stripe in the current search sub-partition to the starting center point, and then repeat step B until all search partitions in the laser image have been searched, so as to achieve the extraction of the light stripe center points formed by the laser stripe in each light stripe partition, and the light stripe center point in the current search sub-partition corresponds to the light stripe center point formed by the laser stripe in the corresponding light stripe partition.

[0008] In summary, the present application determines the starting center point from the initial search partition by executing steps A to C, and then uses recursive extraction of other search partitions and a combination of search center points and grayscale centroid method to extract the light stripe center point formed by the laser stripe in the corresponding light stripe partition. Moreover, in each of the other search partitions, a small range of pixels is defined with the predetermined light stripe center point as the search center. There is no need to search the pixel points of the laser image pixel by pixel, thereby reducing the extraction time of the search center point without affecting the extraction accuracy of the light stripe center point, improving the extraction efficiency of the light stripe center point, and saving memory resources.

[0009] Furthermore, the step B specifically includes: step B1, judging whether there is a valid pixel interval in the current search sub-partition based on a preset grayscale value threshold, if yes, extracting the valid pixel interval from the current search sub-partition, and then executing step B2; otherwise, executing step B3; step B2, extracting the center of gravity position point of the valid pixel interval based on the grayscale centroid method, and determining to extract the center of gravity position point of the current search sub-partition, and then marking the center of gravity position point of the current search sub-partition as the center point of the laser stripe in the current search sub-partition, and then determining that the current search partition has been searched and the center point of the laser stripe has been extracted from the current search partition, and then executing step B5; step B3, judging whether the current search sub-partition currently covers the boundary of the current search partition in the second preset search direction, if yes, determining that the boundary of the current search partition in the second preset search direction has been searched, and then executing step B4; otherwise, executing step B7; wherein, the boundary of the current search partition in the second preset search direction is the boundary pointed to by the second preset search direction in the current search partition; the boundary of the current search partition in the opposite direction of the second preset search direction is the In the current search partition, the boundary pointed to by the opposite direction of the second preset search direction; step B4, determine whether the boundary of the current search partition in the opposite direction of the second preset search direction has been searched, if so, execute step B5; otherwise, update the opposite direction of the second preset search direction to the second preset search direction, and then execute step B6; step B5, determine whether the current search sub-partition currently covers the boundary of the laser image in the first preset search direction, if so, determine that all search partitions in the laser image have been searched, otherwise execute step C; step B6, along the second preset search direction updated in step B4, update the search sub-partition that is adjacent to the current search sub-partition set for the first time in the current search partition and covers the preset search step to the current search sub-partition, and then execute step B1; step B7, along the second preset search direction, update the search sub-partition that is adjacent to the current search sub-partition and covers the preset search step to the current search sub-partition, and then execute step B1; wherein the preset search step is the length occupied by the search sub-partition in the second preset search direction, and the preset search step is measured in terms of the number of pixels.

[0010] In summary, based on the aforementioned steps B1 to B7, in the process of extracting the center point of the light strip within the current search partition, the update of a search center within a single search partition is to update a pixel point separated by a preset search step (the pixel distance occupied by a search sub-partition) in the second preset search direction or in its opposite direction, so that the coordinate span between the two search centers (search starting points) before and after the update is obvious, so that the pixel ranges of two adjacent searches are basically not repeated. In the present application, the center point of the light bar extracted from the previous search partition is used as the current search center, and the current search sub-partition is set within the current search partition using a preset search step size, and then the grayscale threshold and grayscale centroid method are combined within the current search sub-partition to extract the center point of the light bar from the current search partition; if the center point of the light bar cannot be found within the current search sub-partition, an adjacent current search sub-partition is set along the second preset search direction or its opposite direction until the search for the current search partition is stopped, or if the center point of the light bar cannot be found within the current search partition, the search partition is switched; therefore, whenever the center point of the light bar cannot be extracted within a search sub-partition within a search partition, the adjacent search sub-partitions are searched within the same search partition until the center point of the light bar is extracted within a search sub-partition within the same search partition, or if the boundary of a search partition in the second preset search direction and its opposite direction has been searched without extracting the center point of the light bar, the search within the search partition is stopped. Moreover, whenever the center point of the light stripe is extracted within a search partition, the search sub-partition at the new area is switched by executing step C until the boundary of the laser image in the first preset search direction is found to complete the search of all search partitions within the laser image; thereby avoiding wasting time processing pixel areas with repeated positions and avoiding searching all pixel points in the laser image row by row and column by column. Therefore, the present application limits the processing of pixel points (derived from the valid pixel interval within the search sub-partition) within partially adjacent search sub-partitions or a single search sub-partition within each search partition, reducing the amount of calculation required to extract the center position point of the valid pixel interval based on the grayscale centroid method, and speeding up the extraction of the light stripe center point in each search partition or each light stripe partition.

[0011] Furthermore, the step A specifically includes: step A1, judging whether there is a valid pixel interval in the first search partition based on a preset grayscale value threshold, if yes, setting the first search partition as the initial search partition, extracting the valid pixel interval from the initial search partition, and then executing step A2; otherwise, executing step A3; step A2, extracting the centroid position point of the valid pixel interval based on the grayscale centroid method, and determining the centroid position point of the initial search partition; step A3, judging whether the first search partition covers the boundary of the laser image in the first preset search direction, if yes, determining that all search partitions in the laser image have been searched and determining that the light stripe cannot be found in the laser image Center point; otherwise, execute step A4; Step A4: Update the search partition adjacent to the first search partition along the first preset search direction to the first search partition, and then execute step A1; wherein, when step A1 is executed for the first time, the first search partition is configured as the search partition closest to the boundary of the laser image in the direction opposite to the first preset search direction among all the search partitions in the laser image; wherein, the boundary of the laser image in the direction opposite to the first preset search direction is the boundary in the laser image pointed to by the direction opposite to the first preset search direction; and the boundary of the laser image in the first preset search direction is the boundary in the laser image pointed to by the first preset search direction. Therefore, if the currently determined first search partition does not have a valid pixel interval and the currently determined first search partition does not cover the boundary of the laser image in the first preset search direction, the search partition adjacent to the first search partition is updated to the currently determined first search partition, thereby searching for a first search partition with a valid pixel interval one by one, thereby determining an initial search partition from all the search partitions in the laser image.

[0012] Furthermore, a method for determining whether there is a valid pixel interval in a pixel interval to be searched based on a preset grayscale value threshold includes: step P1, determining whether the grayscale values ​​of the pixel points covered by the boundaries of the pixel interval to be searched in the second preset search direction and the opposite direction are all greater than the preset grayscale threshold, if so, determining that the pixel interval to be searched is the valid pixel interval, otherwise executing step P2; wherein, the pixel interval to be searched is the first search partition in step A; the pixel interval to be searched is the current search sub-partition in step B; step P2, starting from one side boundary of the pixel interval to be searched, searching for pixel points in sequence along the second preset search direction or the opposite direction, and detecting the grayscale value of each pixel point; if the grayscale value of the searched pixel point is greater than the preset grayscale threshold, the pixel points whose grayscale value is greater than the preset grayscale threshold are formed into the valid pixel interval; if no pixel point whose grayscale value is greater than the preset grayscale threshold is found, determining that the valid pixel interval does not exist in the pixel interval to be searched; wherein, the pixel interval to be searched is the first search partition in step A; the pixel interval to be searched is the current search sub-partition in step B; step P2, searching for pixel points in sequence along the second preset search direction or the opposite direction from the side boundary of the pixel interval to be searched, and detecting the grayscale value of each pixel point; if the grayscale value of the searched pixel point is greater than the preset grayscale threshold, the pixel points whose grayscale value is greater than the preset grayscale threshold are formed into the valid pixel interval; if the pixel point whose grayscale value is greater than the preset grayscale threshold is found, determining that there is no valid pixel interval in the pixel interval to be searched; The one side boundary of the pixel interval is the boundary of the pixel interval to be searched in the second preset search direction or the boundary of the pixel interval to be searched in the opposite direction of the second preset search direction; or, pixel points are searched sequentially from both sides of the pixel interval to be searched, and the grayscale value of each pixel point is detected; if a pixel point having a grayscale value greater than a preset grayscale threshold is first found from the boundary of the pixel interval to be searched in the second preset search direction, and a pixel point having a grayscale value greater than the preset grayscale threshold is first found from the boundary of the pixel interval to be searched in the opposite direction of the second preset search direction, the pixel point first found in the second preset search direction and the pixel point first found in the opposite direction of the second preset search direction are marked as two endpoints of a valid pixel interval, and the pixels distributed within the pixel interval to be searched at the two endpoints, together with the two endpoints, constitute the valid pixel interval, and determine that the grayscale value of each pixel point between the two endpoints in the pixel interval to be searched is greater than the preset grayscale threshold. A valid pixel interval is thus screened out based on the gradient change pattern of the grayscale value within a range (including the pattern of decreasing from the center to the sides) and the preset grayscale threshold. Based on steps P1 and P2, if the pixel interval to be searched is the first search partition, steps A1 to A4 are executed to determine an initial search partition from all search partitions in the laser image. If the pixel interval to be searched is the current search sub-partition, step B is executed to determine the current search sub-partition with the valid pixel interval within the corresponding search partition in the laser image. During step B, steps P1 and P2 are executed for each search sub-partition set within each search partition to detect whether there is a valid pixel interval, thereby screening out all search sub-partitions capable of calculating the center of gravity position point (extracting the center point of the light stripe) within each search partition of the laser image.

[0013] Furthermore, when the distance of pixels covered by the current search sub-partition within the laser image is less than the preset search step, the following conditions exist: if the boundary of the current search sub-partition in the second preset search direction is already set outside the laser image, then the pixels at the boundary of the laser image currently covered by the current search sub-partition are updated to the pixels covered by the boundary of the current search sub-partition in the second preset search direction; if the boundary of the current search sub-partition in the direction opposite to the second preset search direction is already set outside the laser image, then the pixels at the boundary of the laser image currently covered by the current search sub-partition are updated to the pixels covered by the boundary of the current search sub-partition in the direction opposite to the second preset search direction; wherein, there are no pixels outside the laser image. This causes the robot to not determine the valid pixel interval and calculate the grayscale values ​​of the pixels in the area outside the current search sub-partition.

[0014] Furthermore, in step B, the method for setting the current search center point within the current search partition based on the starting center point includes: in the laser image, setting the coordinate value of the starting center point in the second preset search direction as the coordinate value of the current search center point in the second preset search direction; wherein the coordinate value of the starting center point in the first preset search direction is not equal to the coordinate value of the current search center point in the first preset search direction; wherein the coordinate value of the pixel point in the second preset search direction is the coordinate value in the first coordinate axis of the pixel coordinate system defined in the laser image, and the coordinate value of the pixel point in the first preset search direction is the coordinate value in the second coordinate axis of the pixel coordinate system defined in the laser image. The current search center point is obtained by translating the starting center point along the first preset search direction.

[0015] Furthermore, the method for extracting the centroid position point of the valid pixel interval based on the grayscale centroid method includes: when a first preset search direction is pre-calibrated to be parallel to the horizontal axis direction of the pixel coordinate system, and a second preset search direction is pre-calibrated to be parallel to the vertical axis direction of the pixel coordinate system, the vertical coordinate value of the centroid position point of the valid pixel interval is:

[0016]

[0017] Wherein, i is the horizontal coordinate value of the pixel point in the effective pixel interval, which represents the order of the effective pixel interval in the laser image along the first preset search direction; j is the vertical coordinate value of the pixel point in the effective pixel interval, which represents the order of the pixel point in the effective pixel interval along the second preset search direction; m is the number of all pixels in the effective pixel interval; I(i,j) is the grayscale value of the pixel point (i,j); the first coordinate axis is the vertical coordinate axis, and the second coordinate axis is the horizontal coordinate axis; then the coordinates of the center of gravity point of the effective pixel interval are set to (i,v c ), where the vertical coordinate value v of the center of gravity point c Less than or equal to m.

[0018] Therefore, the coordinate positions and grayscale values ​​of the pixel points in each search sub-partition or the valid pixel interval within the search partition are weighted averaged to obtain the vertical coordinate value of the center point of the valid pixel interval, that is, the vertical coordinate value of the center point of the light bar in the corresponding search sub-partition or the search partition is extracted, and then combined with the horizontal coordinate value of the corresponding search sub-partition or the search partition, the position of the center point of the light bar in the pixel coordinate system can be determined.

[0019] Furthermore, the method for extracting the centroid position point of the valid pixel interval based on the grayscale centroid method includes: when a first preset search direction is pre-calibrated to be parallel to the vertical axis direction of the pixel coordinate system, and a second preset search direction is pre-calibrated to be parallel to the horizontal axis direction of the pixel coordinate system, the horizontal coordinate value of the centroid position point of the valid pixel interval is:

[0020]

[0021] Wherein, j is the horizontal coordinate value of the pixel point in the effective pixel interval, which represents the order of the pixel point in the effective pixel interval along the second preset search direction; m is the number of all pixel points in the effective pixel interval; i is the vertical coordinate value of the pixel point in the effective pixel interval, which represents the order of the effective pixel interval in the laser image along the first preset search direction; I(j,i) is the grayscale value of the pixel point (j,i); the first coordinate axis is the horizontal coordinate axis, and the second coordinate axis is the vertical coordinate axis; then the coordinates of the center of gravity point of the effective pixel interval are set to (v c ,i), where the horizontal coordinate value v of the center of gravity point c Less than or equal to m. If, during steps A to C, the robot searches the laser image line by line to obtain the center point of the light bar belonging to the corresponding search partition, and only the horizontal coordinate value is selected to represent the center point (or center of gravity point) of the light bar in each search partition, the width information of the position where the line laser is reflected on the obstacle surface can be identified.

[0022] Furthermore, in step B, the method of setting the current search sub-partition along the second preset search direction and the opposite direction with the current search center point as the center includes: taking the current search center point as the center, dividing a pixel interval of a preset search distance along the second preset search direction and the opposite direction, and then forming the divided pixel intervals into the current search sub-partition; making two pixel intervals of a preset search distance along the second preset search direction form the current search sub-partition, and the two pixel intervals of the preset search distance are distributed on both sides of the current search center point along the second preset search direction.

[0023] The preset search distance is half of the preset search step size. The greater the number of pixels represented by the preset search distance, the smaller the laser stripe jump within a single search partition. The smaller the number of pixels represented by the preset search distance, the larger the laser stripe jump within a single search partition. Therefore, the preset search step size is measured using the number of pixels covered by the search sub-partition within the current search partition to represent the laser stripe jump within a single search partition.

[0024] A robot, a body assembly line laser module of the robot, the line laser module comprising a line laser emitter and a camera; the laser line emitted by the line laser emitter is reflected back to the camera by the surface to be measured, forming laser stripes; the laser stripes are displayed in the image captured by the camera, so that the robot captures the laser image; the laser stripes are divided into multiple light stripe partitions within the laser image; the laser image is evenly divided into multiple search partitions along the length direction of the laser stripes; the robot is configured to execute the extraction method; wherein the light stripe center point calculated within a search partition corresponds to the light stripe center point formed by the laser stripe in the corresponding light stripe partition; before the robot executes the extraction method, the camera is pre-calibrated so that the distribution direction of pixel points within a search partition within the laser image is parallel to the width direction of the laser stripe; wherein the length direction of the laser stripe is perpendicular to the width direction of the laser stripe, the width direction of the laser stripe is parallel to the second preset search direction, and the length direction of the laser stripe is parallel to the first preset search direction.

[0025] The robot disclosed in the present application first determines the starting center point from the initial search partition by executing steps A to C, and then uses recursive extraction of other search partitions and a combination of search center points and grayscale centroid method to extract the light stripe center point formed by the laser stripe in the corresponding light stripe partition. In addition, in each of the other search partitions, a small range of pixels is defined with the predetermined light stripe center point as the search center, and the update is started from the starting center point. There is no need to search the pixel points of the laser image row by row and column by column, thereby reducing the extraction time of the search center point without affecting the extraction accuracy of the light stripe center point, improving the extraction efficiency of the light stripe center point, and saving memory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flowchart of a method for extracting the center point of a laser stripe within a search zone, disclosed in one embodiment of the present application.

[0027] Figure 2 An embodiment of the present application discloses a schematic diagram of searching for the center point of a light bar in each search partition (each column cell area) from left to right in the pixel coordinate system UOV; wherein, except for the first column on the left, each cell area passed by the arrow in each column is a pixel interval for calculating the center of gravity position point (the center point of the light bar of the laser stripe in the current search sub-partition, referred to as the center point of the light bar in the current search sub-partition, which can also be recorded as the center point of the light bar extracted in the current search partition or the current search sub-partition), the white dot in the figure represents the center of gravity position point or the center point of the light bar, and the black square where the white dot in the figure is located is the cell area occupied by the center point of the light bar. DETAILED DESCRIPTION

[0028] The following describes the technical solutions in the embodiments of the present invention in detail, in conjunction with the accompanying drawings. The present invention provides drawings to further illustrate the various embodiments. These drawings form part of the disclosure and are primarily used to illustrate the embodiments and, in conjunction with the relevant descriptions in the specification, to explain the operating principles of the embodiments. By referring to these drawings, those skilled in the art will be able to understand other possible implementations and the advantages of the present invention.

[0029] This application discloses a method for extracting the center point of a laser stripe within a search zone. Specifically, the method extracts information about the light stripe formed by the reflection of a laser line at the reflection position of a surface to be measured, thereby locating the surface to be measured. The extraction method disclosed in this application is performed by a robot that relies on line laser navigation and positioning. The robot is equipped with a line laser module, which includes a line laser emitter and a camera. The robot controls the line laser emitter to emit a laser line, and the robot controls the camera to capture an image of the laser stripe formed by the reflection of the laser line. The image of the laser stripe is then marked as a laser image.

[0030] In this application, the laser image is evenly divided into multiple search partitions along the length direction of the laser stripe, so that the laser stripe is divided into a single light stripe partition within a single search partition. When each search partition is presented in the form of a column Figure 2 When the pixel coordinate system UOV is shown, the laser stripes are Figure 2 The black cells in the image occupy the corresponding pixel coordinate positions, which can be recorded as a single light stripe partition divided by the laser stripe in a single search partition. Figure 2 The white dot in the image represents the center of the light stripe, but it needs to be searched and located by executing the extraction method. In order to reduce the amount of calculation generated by the search, the application sets the length direction of the laser stripe to the first preset search direction or the opposite direction of the first preset search direction; wherein, the length of the laser stripe is greater than the width of the laser stripe; and the first preset search direction can be pre-configured as a coordinate axis parallel to the pixel coordinate system UOV during the camera calibration process, that is, the length direction of the laser stripe is calibrated to be parallel to the first preset search direction, which can also be understood as the first preset search direction being calibrated to the length direction of the laser stripe; wherein, the length direction of the laser stripe is determined by the assembly angle and height between the camera and the line laser emitter, specifically, the assembly angle and height between the calibrated camera and the calibrated line laser emitter. Before executing the extraction method, the corresponding internal reference calibration can be completed through a calibration plate or other calibration fixture to achieve calibration of the length direction of the laser stripe.

[0031] See Figure 1 , the extraction method comprises:

[0032] Step A: Select an initial search partition from the multiple search partitions divided in the laser image as the initial search partition extracted / searched from the laser image. Generally, it is a search partition in the laser image that is closest to the boundary and can extract the center point of the light strip; then extract the center of gravity position point of the initial search partition based on the grayscale centroid method. Before extracting the center of gravity position point, the grayscale value of each pixel point in the initial search partition may be detected to determine whether to use the grayscale centroid method to calculate the center of gravity position point for the initial search partition to avoid wasting computing power on invalid pixel points; in some embodiments, all pixel points in the initial search partition can also be used for calculation to roughly but more comprehensively determine the center of gravity position point; then mark the center of gravity position point of the initial search partition as the starting center point, and determine that the initial search partition has been searched; then execute step B.

[0033] In the initial search partition, the distribution direction of the pixel points is pre-calibrated to be parallel to the width direction of the laser stripe; the length direction of the laser stripe is perpendicular to the width direction of the laser stripe; Figure 2As shown, the initial search partition can be represented as the first column of cells, counting from left to right. The first column of cells is traversed by the two arrows above and below the black cell. The white dot within the black cell is the starting center point. Therefore, the initial search partition can be considered perpendicular to the width of the laser stripe. The light stripes divided by the laser stripe within the initial search partition are calculated using the grayscale centroid method to determine their center points, forming the light stripe center points of the initial search partition. Preferably, within the initial search partition, the pixel distribution direction is parallel to the width of the laser stripe.

[0034] Step B: Set the search partition adjacent to the initial search partition as the current search partition along the first preset search direction. If the initial search partition is Figure 2 The first column cell area counted from left to right in the , then along the first preset search direction, the search partition adjacent to the initial search partition is Figure 2 The second column of cells counted from left to right in the search partition; if the initial search partition is updated to the Rth (R is greater than 1)th column of cells, the current search partition is updated to the (R+1)th column of cells. Then, based on the starting center point, the current search center point is set within the current search partition. To track the individual light stripes divided by the laser stripe along the first preset search direction between different search partitions, the starting center point is generally translated from the first preset search direction to the current search partition along the first preset search direction to form the current search center point. Then, with the current search center point as the center, the current search sub-partition is set along the second preset search direction and its reverse direction. Generally, the current search sub-partition set for the first time is located within the current search partition. If the current search sub-partition is not suitable for calculating the center of gravity point due to grayscale factors, the next search sub-partition is searched within the same search partition, or along the second preset search direction or its reverse direction to update it to the current search sub-partition, and the current search center point is updated. Of course, after the current search partition is updated, the current search sub-partition and the search center point are also updated.

[0035] In step B, the center of gravity position of the current search sub-region is extracted based on the grayscale center of gravity method. Compared with searching and calculating all pixels in the entire search sub-region, this saves pixel storage space and speeds up the calculation of the center of gravity position. The center of gravity position of the current search sub-region is then marked as the center point of the laser stripe in the current search sub-region. It is then determined that the current search sub-region has been searched and the center point of the laser stripe has been extracted from the current search sub-region. Then, step C is executed.

[0036] Step C, update the current search partition to the initial search partition, and update the light stripe center point of the laser stripe in the current search sub-partition to the starting center point, and then repeat step B until all search partitions in the laser image have been searched; if the search partitions are arranged column by column in the laser image (correspondingly, the width direction of the laser stripe is perpendicular to the robot's travel plane), then the process of repeating step B is to update the initial search partition and the starting center point column by column until all search partitions in the laser image are searched column by column to obtain a group of light stripe centers distributed in columns. If the search partitions are arranged row by row in the laser image (correspondingly, the width direction of the laser stripe is parallel to the robot's travel plane), then the process of repeating step B is to update the initial search partition and the starting center point column by column until all search partitions in the laser image are searched row by row to obtain a group of light stripe centers distributed in rows. Thereby, the extraction of the laser stripes in each light stripe partition ( Figure 2 The center point of the light bar is formed in the black cells in each column shown and includes a white center point inside; wherein, the center point of the light bar in the current search sub-division corresponds to the center point of the light bar formed by the laser stripe in the corresponding light bar division. When the laser stripe changes in the corresponding light bar division, the center point of the light bar in the current search sub-division and inside it also changes, achieving a tracking effect.

[0037] Schematically, as Figure 2As shown, the cell area in the second column counting from left to right (the search partition where the cell passed by arrow AB is located), the cell area in the third column counting from left to right (the search partition where the cell passed by arrow C1D1 is located), the cell area in the fourth column counting from left to right, ..., the cell area in the seventh column counting from left to right (the search partition where the cell passed by arrow C2D2 is located), and the cell area in the eighth column counting from left to right (the search partition where the cell passed by arrow C3D3 is located) all search the center of the light bar column by column, wherein the cell area in the second column counting from left to right (the search partition where the cell passed by arrow AB is located), the cell area in the third column counting from left to right (the search partition where the cell passed by arrow C1D1 is located) are all searched for the center of the light bar column by column. The vertical coordinates of the center points of the light bars extracted within the cell area in the 4th column from left to right, ..., and the cell area in the 7th column from left to right (the search partition where the cell passed by arrow C2D2 is located) are all equal, and the search partitions covered by the cell area in the 3rd column from left to right (the search partition where the cell passed by arrow C1D1 is located), the cell area in the 4th column from left to right, ..., and the cell area in the 7th column from left to right (the search partition where the cell passed by arrow C2D2 is located), and the cell area in the 8th column from left to right (the search partition where the cell passed by arrow C3D3 is located) are all within the same vertical coordinate range.

[0038] Schematically, as Figure 2 As shown, there is no light bar center point between the 12th column cell area counted from left to right and the 16th column cell area counted from left to right, and the 16th column cell area counted from left to right spans search partitions in which no light bar center point exists in multiple columns relative to the 12th column cell area counted from left to right; wherein, the vertical coordinate of the black cell passed by the arrow J2K2 in the 12th column cell area counted from left to right is not equal to the vertical coordinate of the black cell passed by the arrow LQ in the 16th column cell area counted from left to right, and the direction of the arrow LQ is opposite to that of the arrow J2K2, which means that in the 16th column cell area counted from left to right, all search sub-partitions located below the vertical coordinate of the black cell passed by the arrow J2K2 have not extracted the light bar center point, and only the search sub-partitions located above the vertical coordinate of the black cell passed by the arrow J2K2 can extract the light bar center point; wherein, the search sub-partition located above the vertical coordinate of the black cell passed by the arrow J2K2 in the 16th column cell area counted from left to right is the cell area passed by the arrow LQ.

[0039] In summary, the present application determines the starting center point from the initial search partition by executing steps A to C, and then uses recursive extraction of other search partitions and a combination of search center points and grayscale centroid method to extract the light stripe center point formed by the laser stripe in the corresponding light stripe partition. Moreover, in each of the other search partitions, a small range of pixels is defined with the predetermined light stripe center point as the search center. There is no need to search the pixel points of the laser image pixel by pixel, thereby reducing the extraction time of the search center point without affecting the extraction accuracy of the light stripe center point, improving the extraction efficiency of the light stripe center point, and saving memory resources.

[0040] As an embodiment, the step B specifically includes:

[0041] Step B1: Based on a preset grayscale threshold, determine whether there is a valid pixel interval within the current search sub-region. If so, extract the valid pixel interval from the current search sub-region and proceed to step B2; otherwise, proceed to step B3. Specifically, if the robot determines that there is a valid pixel interval within the current search sub-region, it extracts the valid pixel interval from the current search sub-region and proceeds to step B2. If the robot determines that there is no valid pixel interval within the current search sub-region, it proceeds to step B3. The current search sub-region is divided from the current search sub-region according to the aforementioned embodiment. This embodiment sets a preset grayscale threshold to filter out background pixels within the current search sub-region, thereby reducing interference with the extraction of the center point of the light strip.

[0042] The current search sub-partition is a search sub-partition that is searched within the current search partition when the robot currently executes step B1 or step B.

[0043] Step B2: extract the center of gravity of the effective pixel interval based on the grayscale centroid method, and determine to extract the center of gravity of the current search sub-area; then mark the center of gravity of the current search sub-area as the center of the laser stripe in the current search sub-area, and then determine that the current search area has been searched and the center of the laser stripe is extracted from the current search area; corresponding to Figure 2 In the example, the robot executes steps B1 and B2 to extract the center points of the light stripes within the cell regions of columns 2 through 8, counting from left to right. A light stripe center point is extracted from the current search sub-region within each column of cell regions (considered the current search region or the updated initial search region). Step B5 is then executed to determine whether the robot has exhausted all search regions within the laser image.

[0044] Step B3: Determine whether the current search sub-area covers the boundary of the current search sub-area in the second preset search direction. If so, determine that the boundary of the current search sub-area in the second preset search direction has been searched, and then execute step B4; otherwise, execute step B7 to continue searching within the current search sub-area. The boundary of the current search sub-area in the second preset search direction is the boundary of the current search sub-area pointed to by the second preset search direction; the boundary of the current search sub-area in the second preset search direction is located within the boundary of the laser image in the second preset search direction. The boundary of the current search sub-area in the opposite direction of the second preset search direction is the boundary of the current search sub-area pointed to by the opposite direction of the second preset search direction.

[0045] Determining whether the current search sub-partition covers the boundary of the current search sub-partition in the second preset search direction is specifically to determine whether the current search sub-partition reaches the boundary of the laser image in the second preset search direction during the process of the robot searching each search sub-partition along the second preset search direction.

[0046] In the case that the current search sub-partition covers the boundary of the current search partition in the second preset search direction, the boundary of the current search sub-partition in the second preset search direction overlaps with the boundary of the current search partition in the second preset search direction, or the boundary of the current search partition in the second preset search direction is located inside the current search sub-partition, step B4 is executed on this basis.

[0047] When the current search sub-partition does not cover the boundary of the current search partition in the second preset search direction, there is a certain distance between the boundary of the current search sub-partition in the second preset search direction and the boundary of the current search partition in the second preset search direction, and step B8 is executed on this basis.

[0048] Step B4: Determine whether the boundary of the current search partition in the opposite direction of the second preset search direction has been searched. If so, execute step B5 to determine whether the current search partition has been searched but the center point of the light bar has not been extracted from the current search partition; otherwise, update the opposite direction of the second preset search direction to the second preset search direction, and then execute step B6.

[0049] In step B4, the robot has determined that the current search sub-partition currently covers the boundary of the current search partition in the second preset search direction, but needs to determine whether the boundary of the current search partition in the opposite direction of the second preset search direction has been searched; if the boundary of the current search partition in the opposite direction of the second preset search direction has been searched, then based on the execution order of the aforementioned steps, it is determined that during the search in the opposite direction of the second preset search direction within the current search partition, no light bar center point is extracted within the current search partition; if the boundary of the current search partition in the opposite direction of the second preset search direction has not been searched, then it is necessary to subsequently search in the opposite direction of the second preset search direction within the current search partition, specifically to update the opposite direction of the second preset search direction to the second preset search direction, and then execute step B6, in order to extract the light bar center point within the current search partition, because along the second preset search direction of the current search partition, no light bar center point is extracted in the current search sub-partition set based on the starting center point.

[0050] Step B5, determine whether the current search sub-partition covers the boundary of the laser image in the first preset search direction. If so, determine that all search partitions in the laser image have been searched, that is, the robot completes the search of the laser image to extract the light stripe center point formed by the laser stripe in the corresponding light stripe partition in each valid pixel interval; otherwise, execute step C, so that: the robot updates the current search partition to the initial search partition, and updates the light stripe center point of the laser stripe in the current search sub-partition to the starting center point, and then, by executing step B disclosed in the aforementioned embodiment, along the first preset search direction, the search partition adjacent to the initial search partition is set as the current search partition, and then the current search center point is set in the current search partition based on the starting center point, and then the current search sub-partition is set in the current search partition with the current search center point as the center, and then the currently set current search sub-partition is specifically updated to the current search sub-partition described in step B1, and then step B1 is repeated.

[0051] If the process jumps from step B2 to step B5, then when the current search sub-area covers the boundary of the laser image in the first preset search direction, not only the center point of the light stripe extracted by the robot in the laser image for the last time is obtained, but also the search of all search sub-areas in the laser image is completed, and then step B and step C can be stopped. Schematically, as Figure 2As shown, when the current search sub-partition is set in the 20th column cell area counted from left to right, the current search sub-partition can cover the black cells (containing white circular dots) and the right boundary of the current search sub-partition reaches the right edge of the laser image. It is impossible to extract the center point of the light bar to the right column, so the search of the entire frame laser image is completed.

[0052] If step B4 is jumped to step B5, it is determined that the current search partition has been searched but the center point of the light bar has not been extracted from the current search partition. In this case, when the current search sub-partition covers the boundary of the laser image in the first preset search direction, although the search of all search partitions in the laser image is completed, the center point of the light bar has not been extracted from the current search partition. It is also possible that the center point of the light bar has not been extracted from any search partition other than the search partition described in step A.

[0053] Step B6, along the second preset search direction updated in step B4, update the search subpartition adjacent to the current search subpartition set for the first time in the current search partition and covering the preset search step to the current search subpartition, and then execute step B1. The current search subpartition set for the first time in the current search partition can be understood as the current search subpartition set along the second preset search direction and its opposite direction with the current search center point set for the first time in the current search partition as the center. Therefore, the current search subpartition set for the first time in the current search partition belongs to the current search subpartition obtained by executing step B for the first time in the current search partition. The search subpartition covering the preset search step along the second preset search direction updated in step B4 can be updated to the current search subpartition, that is, the length of the current search subpartition in the second preset search direction is equal to the preset search step.

[0054] In step B6, schematically, as Figure 2As shown, when the 13th column of cell area counted from left to right is set as the current search partition, the second preset search direction is the positive direction of the V coordinate axis of the pixel coordinate system, that is, the vertical downward direction shown in the figure; within the 13th column of cell area counted from left to right (corresponding to the current search partition), the cell area passed by the arrow 2 currently used for searching covers the boundary of the current search partition in the second preset search direction (the bottom boundary of the 13th column of cell area counted from left to right), and at this time, it is determined that the boundary of the current search partition in the opposite direction of the second preset search direction has not been searched, that is, the top boundary of the 13th column of cell area counted from left to right has not been searched by the subsequent arrow 5 for searching, and then the arrow 2 currently used for searching is adjusted to the boundary of the current search partition in the opposite direction of the second preset search direction. Search arrow 3; then, along the direction of arrow 3, the search sub-area adjacent to the cell area passed by arrow 1 and covering the preset search step (equal number of cells) along the second preset search direction is set as the cell area passed by arrow 3; wherein the cell area passed by arrow 1 is the current search sub-area set for the first time within the 13th column of cells counted from left to right, and the coordinate value of the current search center set in the cell area passed by arrow 1 in the V coordinate axis is equal to the coordinate value of the light bar center point extracted in the 12th column of cells counted from left to right in the V coordinate axis. The current search center set in the cell area passed by arrow 1 is equivalent to the black cell in the 12th column of cells counted from left to right shifted one cell to the right. The direction of arrow 3 is opposite to the direction of arrow 2 currently used for search to achieve updating the opposite direction of the second preset search direction to the second preset search direction, wherein, if the second preset search direction is the vertical downward direction shown in the figure, the opposite direction of the second preset search direction is the vertical upward direction shown in the figure. Then, in step B6, a new current search sub-area is set and the light bar center point is extracted along the vertical upward direction shown in the figure.

[0055] In the 13th column of cell areas counted from left to right, the cell areas passed by arrow 3, the cell areas passed by arrow 4, and the cell areas passed by arrow 5 are set in sequence along the vertical upward direction shown in the figure. When setting the cell area passed by arrow 5, the top boundary of the 13th column of cell areas counted from left to right has been covered (it can be understood that the current search sub-partition currently covers the boundary of the current search partition in the second preset search direction, and the second preset search direction here can be updated from the opposite direction of the second preset search direction), but no cell in the light bar is extracted. The center point of the light bar is obtained by searching the cell area in the 13th column from left to right, that is, no black cell containing a white dot is found in the cell area in the 13th column from left to right, which proves that there is no center point of the light bar in the current search partition. Therefore, the next search partition can be processed, that is, the pixel points in the next column (the cell area in the 14th column from left to right) can be processed. Moreover, the extraction of the center point of the light bar in the cell area in the 13th column from left to right and the cell area in the 14th column from left to right both start from the coordinate value position of the center point of the light bar extracted in the cell area in the 12th column from left to right on the V coordinate axis of the pixel coordinate system.

[0056] When the cell area of ​​the 16th column counted from left to right is set as the current search partition, the cell area passed by the arrow MN currently used for searching covers the boundary of the current search partition in the second preset search direction (the bottom boundary of the cell area of ​​the 16th column counted from left to right). At this time, it is also determined that the boundary of the current search partition in the opposite direction of the second preset search direction has not been searched, that is, the top boundary of the cell area of ​​the 16th column counted from left to right has not been searched in advance, and then the arrow MN currently used for searching is adjusted to the arrow LQ used for searching; then, along the direction of the arrow LQ, from the cell area adjacent to the cell area passed by the arrow LM and covering the cell area along the second preset search direction. The search sub-partition with a preset search step (equal number of cells) is set to the cell area passed by the arrow LQ; wherein, the cell area passed by the arrow LM is the current search sub-partition set for the first time in the cell area of ​​the 16th column counted from left to right, and the coordinate value of the current search center set in the cell area passed by the arrow LM in the V coordinate axis is equal to the coordinate value of the center point of the light bar extracted in the cell area of ​​the 12th column counted from left to right in the V coordinate axis, and the current search center set in the cell area passed by the arrow LM is equivalent to the black cell in the cell area of ​​the 12th column counted from left to right (the black cell passed by the arrow J2K2) shifted one cell to the right. In the cell area of ​​the 16th column counting from left to right, the cell area passed by the arrow LQ is set along the vertical upward direction of the diagram. The cell area passed by the arrow LQ has covered the black cells, which proves that the center point of the light bar is extracted in the cell area of ​​the 16th column counting from left to right. Then stop setting the search sub-partition and extracting the center point of the light bar along the vertical upward direction of the diagram. Therefore, the next search partition can be processed, that is, the next column (the cell area of ​​the 17th column counting from left to right) of pixels can be processed. Moreover, the extraction of the center point of the light bar in the cell area of ​​the 16th column counting from left to right starts from the coordinate value position of the center point of the light bar extracted in the cell area of ​​the 12th column counting from left to right on the V coordinate axis of the pixel coordinate system. Return to execute step B1 and step B2 in this manner, and extract the center points of the light stripes in the 17th column unit area to the 20th column unit area counting from left to right in sequence, wherein one light stripe center point is extracted from each column; until it is determined in step B5 that the current search sub-partition covers the boundary of the laser image in the first preset search direction, it is determined that all search partitions in the laser image have been searched, so as to extract the center points of the light stripes formed by the laser stripes in the corresponding light stripe partitions in each column unit area.

[0057] Step B7: Update the search sub-partition adjacent to the current search sub-partition and covering the preset search step to the current search sub-partition along the second preset search direction, and then execute step B1; wherein, the second preset search direction is perpendicular to the first preset search direction; the preset search step is the length occupied by the search sub-partition in the second preset search direction, and the preset search step is measured in terms of the number of pixels; it can be understood that if the current search sub-partition completely covers the current search partition of the laser image, then the number of pixels covered by the current search sub-partition along the second preset search direction can be regarded as equal to the preset search step.

[0058] Preferably, in the process of repeatedly executing step B7 along the second preset search direction that has not been updated, the currently set current search sub-area is adjacent to the previously set current search sub-area, and there may be no gap, but they will not overlap by more than 1 pixel.

[0059] In step B7, schematically, as Figure 2 As shown, the second preset search direction is the positive direction of the V coordinate axis of the pixel coordinate system, that is, the vertical downward direction shown in the figure; the first preset search direction is the positive direction of the U coordinate axis of the pixel coordinate system, that is, the horizontal direction shown in the figure. When the 9th column of cell area counted from left to right is set as the current search partition, the 9th column of cell area counted from left to right includes the cells passed by arrow EF and the cells passed by arrow FG. The cell area passed by arrow EF represents a search sub-partition in which the center point of the light bar cannot be extracted for the first time within the current search partition. At the same time, it is determined that the cell area passed by arrow EF does not cover the upper boundary of the 9th column of cell area counted from left to right, does not cover the lower boundary of the 9th column of cell area counted from left to right, and does not cover the right boundary of the laser image. Then, by executing step B7, the cell area passed by arrow FG is set below the cell area passed by arrow EF. The cell area passed by arrow FG is adjacent to the cell area passed by arrow EF, and the number of pixels covered by the cell area passed by arrow FG is equal to the number of pixels covered by the cell area passed by arrow EF. For example, the number of pixels covered by the cell area passed by arrow FG is expressed using the preset search step; the pixel distance between the search center of the cell area passed by arrow FG and the search center of the cell area passed by arrow EF is equal to the preset search step.

[0060] exist Figure 2It can be understood that each arrow located in the laser image except the pixel coordinate system UOV and the cell area of ​​the first column corresponds to a search sub-partition (supports updating to the current search sub-partition); the cell area passed by the two arrows with the same direction marked in the cell area of ​​the first column represents the initial search partition, which belongs to the search partition with the starting center point (the white dot contained in the black cell of the first column) as the center of gravity.

[0061] Preferably, the preset search step is equal to 10, and two adjacent search sub-areas may occupy only one common pixel point, so that the coordinate span between the search centers set in the two steps B (or the starting center points updated in step C) is obvious.

[0062] After executing steps B1 and B2, the center point of the light bar can be extracted from the cell area where the arrow FG passes, as shown in FIG. Figure 2 As shown by the white dots in the black cells in the 9th column cell area counting from left to right; it is determined that the cell area passed by the arrow FG represents a search sub-partition in which the center point of the light bar is extracted for the first time in the current search partition, and then stop in the 9th column cell area counting from left to right and continue to execute step B, and then search for the center point of the light bar in the adjacent column by executing step C.

[0063] In summary, based on the aforementioned steps B1 to B7, in the process of extracting the center point of the light strip within the current search partition, the update of a search center within a single search partition is to update a pixel point separated by a preset search step (the pixel distance occupied by a search sub-partition) in the second preset search direction or in its opposite direction, so that the coordinate span between the two search centers (search starting points) before and after the update is obvious, so that the pixel ranges of two adjacent searches are basically not repeated.

[0064] In the present application, the center point of the light bar extracted from the previous search partition is used as the current search center, and the current search sub-partition is set within the current search partition using a preset search step size, and then the grayscale threshold and grayscale centroid method are combined within the current search sub-partition to extract the center point of the light bar from the current search partition; if the center point of the light bar cannot be found within the current search sub-partition, an adjacent current search sub-partition is set along the second preset search direction or its opposite direction until the search for the current search partition is stopped, or if the center point of the light bar cannot be found within the current search partition, the search partition is switched; therefore, whenever the center point of the light bar cannot be extracted within a search sub-partition within a search partition, the adjacent search sub-partitions are searched within the same search partition until the center point of the light bar is extracted within a search sub-partition within the same search partition, or if the boundary of a search partition in the second preset search direction and its opposite direction has been searched without extracting the center point of the light bar, the search within the search partition is stopped. Moreover, whenever the center point of the light strip is extracted in a search partition, the search sub-partition at the new area is switched by executing step C until the boundary of the laser image in the first preset search direction is found to complete the search of all search partitions in the laser image; thereby avoiding wasting time processing pixel areas with repeated positions and avoiding searching all pixel points in the laser image row by row and column by column.

[0065] Therefore, the present application limits the processing of pixel points (originating from the valid pixel interval within the search sub-partition) to partially adjacent search sub-partitions within each search partition or a single search sub-partition, reducing the amount of calculation required to extract the center position point of the valid pixel interval based on the grayscale centroid method, and speeding up the extraction of the light bar center point in each search partition or each light bar partition.

[0066] As an embodiment, step A specifically includes: step A1, determining whether there is a valid pixel interval within the first search partition based on a preset grayscale value threshold; if so, setting the first search partition as the initial search partition, selecting an initial search partition from multiple search partitions divided within the laser image, extracting a valid pixel interval from the first search partition, and then executing step A2; otherwise, executing step A3, i.e., if it is determined that there is no valid pixel interval within the first search partition, executing step A3. Step A1 must be executed before executing step B or step B1. Thus, the first search partition is set as the search partition that defines the initial search range for the center point of the light stripe within the laser image.

[0067] It should be noted that, when step A1 is executed for the first time, the first search partition is configured as a search partition closest to the boundary of the laser image in the opposite direction of the first preset search direction among all the search partitions in the laser image, corresponding to Figure 2In the first column of cell areas counted from left to right, the first preset search direction is the positive direction of the U coordinate axis of the pixel coordinate system, that is, the horizontal direction shown in the figure, so that the first search partition is configured for the first time to cover the boundary of the laser image in the opposite direction of the first preset search direction, and the first search partition is also configured as the first search partition arranged along the first preset search direction among all the search partitions in the laser image; wherein, the boundary of the laser image in the first preset search direction is the boundary pointed to by the first preset search direction in the laser image; the boundary of the laser image in the opposite direction of the first preset search direction is the boundary pointed to by the opposite direction of the first preset search direction in the laser image, corresponding to Figure 2 The left boundary of the laser image.

[0068] Step A2: extracting the center of gravity position point of the effective pixel interval based on the grayscale center of gravity method, and determining to extract the center of gravity position point of the initial search partition; then marking the center of gravity position point of the initial search partition as the light stripe center point of the laser stripe in the initial search partition, and then determining that the first search partition has been searched and determining to extract the light stripe center point from the initial search partition.

[0069] Step A3: Determine whether the first search partition covers the boundary of the laser image in the first preset search direction. If so, determine that all search partitions in the laser image have been searched and that the center point of the light strip cannot be found in the laser image; otherwise, execute step A4; thereby determining the conditions for stopping the search.

[0070] Step A4: Update the search partitions adjacent to the first search partition along the first preset search direction to the first search partition, and then execute Step A1. Therefore, if the currently determined first search partition does not have a valid pixel interval and does not cover the boundary of the laser image in the first preset search direction, the search partitions adjacent to the first search partition are updated to the currently determined first search partition, thereby searching for a first search partition with a valid pixel interval one by one, thus determining the initial search partition from all search partitions within the laser image.

[0071] In the aforementioned embodiment, the method for determining whether there is a valid pixel interval within a pixel interval to be searched based on a preset grayscale value threshold includes:

[0072] Step P1, determine whether the grayscale values ​​of the pixel points covered by the boundaries of the pixel interval to be searched in the first preset search direction and the opposite direction are all greater than the preset grayscale threshold. If so, determine that the pixel interval to be searched is the said valid pixel interval, otherwise execute step P2; preferably, if the laser image is filtered in advance using a Gaussian filter, the preset grayscale threshold is higher than the grayscale threshold set in the Gaussian filter, so as to filter out the interference of pixel points in the pixel interval to be searched that are close to the grayscale value of the laser stripe, thereby improving the accuracy of the calculation of the center of gravity position point (center point of the light stripe) of the said valid pixel interval.

[0073] Based on the above embodiment, the pixel interval to be searched is the first search partition in step A to accept the detection of the valid pixel interval; the pixel interval to be searched is the current search sub-partition in step B to realize the judgment whether there is a valid pixel interval in the current search sub-partition based on the preset grayscale value threshold.

[0074] Step P2: Starting from one side boundary of the pixel interval to be searched, search for pixel points in sequence along a second preset search direction or its opposite direction, and detect the grayscale value of each pixel point; if the grayscale value of the searched pixel point is greater than the preset grayscale threshold, the pixel points whose grayscale value is greater than the preset grayscale threshold are formed into the valid pixel interval; if no pixel point whose grayscale value is greater than the preset grayscale threshold is found, it is determined that the valid pixel interval does not exist in the pixel interval to be searched.

[0075] Alternatively, in step P2, pixel points are searched in sequence starting from the two side boundaries of the pixel interval to be searched, and the grayscale value of each pixel point is detected; if a pixel point with a grayscale value greater than a preset grayscale threshold is first searched from the boundary of the pixel interval to be searched in the first preset search direction, and a pixel point with a grayscale value greater than the preset grayscale threshold is first searched from the boundary of the pixel interval to be searched in the opposite direction of the first preset search direction, then the pixel point first searched in the first preset search direction and the pixel point first searched in the opposite direction of the first preset search direction are respectively marked as the two endpoints of the valid pixel interval, and then the pixel points distributed in the pixel interval to be searched at the two endpoints together with the two endpoints form the valid pixel interval, and it is determined that the grayscale values ​​of the pixel points between the two endpoints of the pixel interval to be searched are all greater than the preset grayscale threshold, so as to filter out noise information in the pixel interval to be searched.

[0076] Thus, based on the gradient change law of the grayscale value in an interval (including the law of decreasing from the center to both sides) and the preset grayscale threshold, the valid pixel interval is screened out. Based on step P1 and step P2, when the pixel interval to be searched is the first search partition, the initial search partition is determined from all the search partitions in the laser image by executing steps A1 to A4. When the pixel interval to be searched is the current search sub-partition, the current search sub-partition with the valid pixel interval in the corresponding search partition is determined in the laser image by executing steps B1 to B7, wherein, in the process of executing steps B1 to B7, each time a search sub-partition is set in each search partition, steps P1 and P2 need to be executed to detect whether there is a valid pixel interval, thereby screening out all search sub-partitions that can calculate the center of gravity position point (extract the center point of the light stripe) in each search partition of the laser image.

[0077] Based on the above embodiment, for the current search sub-area set in step B, when the pixel distance covered by the current search sub-area in the laser image is less than the preset search step size, there exists:

[0078] If the boundary of the current search sub-area in the second preset search direction is already set outside the laser image, the pixel points at the boundary of the laser image currently covered by the current search sub-area are updated to the pixel points covered by the boundary of the current search sub-area in the second preset search direction. Figure 2 In the example, the second preset search direction is the positive direction of the V coordinate axis of the pixel coordinate system (i.e., the vertical downward direction shown in the figure). Figure 2 The boundaries of the search sub-area corresponding to the arrow MN and the search sub-area corresponding to the arrow 2 in the second preset search direction are both set outside the laser image ( Figure 2 Not shown), then Figure 2 Only the pixels covered by the search sub-partition corresponding to the arrow MN in the 16th column cell area counted from left to right participate in the judgment of the valid pixel interval and the grayscale centroid method; Figure 2Only the pixels covered by the search sub-partition corresponding to arrow 2 within the 13th column of cell areas, counted from left to right, participate in the determination of the valid pixel interval and the grayscale centroid method. Therefore, the pixel points at the boundary of the laser image currently covered by the search sub-partition corresponding to arrow MN are updated to the pixel points covered by the boundary of the current search sub-partition in the second preset search direction, and the pixel points at the boundary of the search sub-partition corresponding to arrow MN in the opposite direction of the second preset search direction are updated to the pixel points covered by the boundary of the current search sub-partition in the opposite direction of the second preset search direction. As a result, the pixel points covered by the boundaries of the cell area passed by arrow MN in the second preset search direction and its opposite direction are respectively set to the pixel points covered by the boundary of the current search sub-partition in the second preset search direction and its opposite direction.

[0079] If the boundary of the current search sub-area in the opposite direction of the second preset search direction has been set outside the laser image, the pixel points at the boundary of the laser image currently covered by the current search sub-area are updated to the pixel points covered by the boundary of the current search sub-area in the opposite direction of the second preset search direction. Figure 2 , the opposite direction of the second preset search direction is the negative direction of the V coordinate axis of the pixel coordinate system (i.e., the vertical upward direction shown in the figure). Figure 2 The boundaries of the search sub-areas corresponding to the arrow 5 in the opposite direction of the second preset search direction are all set outside the laser image ( Figure 2 Not shown), then Figure 2 Only the pixels covered by the search sub-region corresponding to arrow 5 within the 13th column of cells, counting from left to right, participate in the determination of the valid pixel interval and the grayscale centroid method. Therefore, the pixels at the boundary of the laser image currently covered by the search sub-region corresponding to arrow 5 are updated to the pixels covered by the boundary of the current search sub-region in the direction opposite to the second preset search direction. As a result, the pixels covered by the boundaries of the cell region passed by arrow 5 in the second preset search direction and its opposite direction are respectively set to the pixels covered by the boundary of the current search sub-region in the second preset search direction and its opposite direction.

[0080] There are no pixels outside the laser image, so that the robot does not judge the effective pixel interval and calculate the grayscale value of the pixel in the area outside the current search sub-partition, including using the grayscale centroid method for calculation.

[0081] It should be noted that the number of pixels represented by the preset search step or the number of pixels covered by the search sub-region corresponds to Figure 2The search sub-regions corresponding to arrows MN, arrow 2, and arrow 5 still have some areas outside the laser image and are not covered. Only the covered pixel areas inside the laser image are displayed to participate in the grayscale value calculation of the pixel points.

[0082] In addition, Figure 2 It can be understood that each arrow located in the laser image except the pixel coordinate system UOV and the cell area of ​​the first column corresponds to a search sub-partition (supports updating to the current search sub-partition); the cell area passed by the two arrows with the same direction marked in the cell area of ​​the first column represents the initial search partition, which belongs to the search partition with the starting center point (the white dot contained in the black cell of the first column) as the center of gravity.

[0083] As an embodiment, in step B, the method of setting the current search center point in the current search partition based on the starting center point includes:

[0084] In the laser image, the coordinate value of the starting center point in the second preset search direction is set to the coordinate value of the current search center point in the second preset search direction, so that the starting center point is translated along the first preset search direction to obtain the current search center point; wherein, the coordinate value of the starting center point in the first preset search direction is not equal to the coordinate value of the current search center point in the first preset search direction; the coordinate value of the pixel point in the second preset search direction is the coordinate value in the first coordinate axis of the pixel coordinate system set in the laser image, and the coordinate value of the pixel point in the first preset search direction is the coordinate value in the second coordinate axis of the pixel coordinate system set in the laser image; the first coordinate axis is perpendicular to the second coordinate axis.

[0085] Combine Figure 2It can be seen that the white dot inside the black cell in the first column of cells, counting from left to right, is the starting center point. Based on steps B and C, the coordinate value of the current search center point in the width direction of the laser stripe is equal to the coordinate value of the starting center point in the width direction of the laser stripe, wherein the width direction of the laser stripe is parallel to each column of the laser image. The coordinate value of the black cell in the first column of cells, counting from left to right, in the first coordinate axis V is equal to the coordinate value of the current search center point, first set in the second column of cells, counting from left to right, in the first coordinate axis V. It can be understood that the coordinate value of the black cell in the first column of cells, counting from left to right, in the first coordinate axis V is equal to the coordinate value of the center of the cell area passed by arrow AB, wherein the coordinate value of the center of gravity point extracted in the first column of cells, counting from left to right, in the first coordinate axis V is the coordinate value of the center of the cell area passed by arrow AB, and the center of the cell area passed by arrow AB is the current search center point, first set in the second column of cells, counting from left to right. Similarly, the coordinate value of the center of the cell area passed by arrow C1D1 (corresponding to the search center of the search sub-partition) in the first coordinate axis V is the coordinate value of the center point of the light bar extracted from the cell area in the second column counted from left to right (the white dot inside the black cell) in the first coordinate axis V, the coordinate value of the center of the cell area passed by arrow C2D2 (corresponding to the search center of the search sub-partition) in the first coordinate axis V is the coordinate value of the center point of the light bar extracted from the cell area in the sixth column counted from left to right (the white dot inside the black cell) in the first coordinate axis V, and the coordinate value of the center of the cell area passed by arrow C3D3 (corresponding to the search center of the search sub-partition) in the first coordinate axis V is the coordinate value of the center point of the light bar extracted from the cell area passed by arrow C3D3 (the white dot inside the black cell) in the first coordinate axis V. After crossing the cell area where the light bar center point does not exist starting from the search partition where the arrow J2K2 is located, the cell area passed by the arrow LM is the current search sub-partition set for the first time in the cell area of ​​the 16th column counted from left to right. The coordinate value of the current search center set in the cell area passed by the arrow LM in the V coordinate axis is equal to the coordinate value of the light bar center point extracted in the cell area of ​​the 12th column counted from left to right in the V coordinate axis. The current search center set in the cell area passed by the arrow LM is equivalent to the black cell in the cell area of ​​the 12th column counted from left to right (the black cell passed by the arrow J2K2) shifted one cell to the right.

[0086] Schematically, as Figure 2As shown, the coordinate value of the pixel point in the second preset search direction can be recorded as the vertical coordinate value, and the coordinate value of the pixel point in the first preset search direction can be recorded as the horizontal coordinate value; the 9th column cell area counting from left to right (the search partition where the cell passed by arrow FG is located), the 10th column cell area counting from left to right (the search partition where the cell passed by arrow J1K1 is located), the 11th column cell area counting from left to right, and the 12th column cell area counting from left to right (the search partition where the cell passed by arrow J2K2 is located) all search for the center of the light bar column by column; the vertical coordinates of the extracted center points of the light bar in the 9th column cell area counting from left to right (the search partition where the cell passed by arrow FG is located), the 10th column cell area counting from left to right (the search partition where the cell passed by arrow J1K1 is located), the 11th column cell area counting from left to right, and the 12th column cell area counting from left to right (the search partition where the cell passed by arrow J2K2 is located) are all equal.

[0087] Among them, the vertical coordinates of the current search centers set in the 10th column cell area counting from left to right (the search partition where the cell passed by arrow J1K1 is located), the 11th column cell area counting from left to right, and the 12th column cell area counting from left to right (the search partition where the cell passed by arrow J2K2 is located) are all equal, and each column cell area has only one current search sub-partition set and is within the same vertical coordinate range.

[0088] As an embodiment, a method for extracting the centroid position point of the valid pixel interval based on the grayscale centroid method includes: when a first preset search direction is pre-calibrated to be parallel to the horizontal axis direction of the pixel coordinate system, and a second preset search direction is pre-calibrated to be parallel to the vertical axis direction of the pixel coordinate system, the vertical coordinate value (which can also be recorded as a row coordinate value) of the centroid position point of the valid pixel interval is:

[0089]

[0090] Where i is the abscissa value of the pixel points within the valid pixel interval, representing the order of the valid pixel interval within the laser image along the first preset search direction; j is the ordinate value of the pixel points within the valid pixel interval, representing the order of the pixel points within the valid pixel interval along the second preset search direction; m is the number of all pixels within the valid pixel interval; I(i, j) is the grayscale value of pixel point (i, j). Σ represents a summation operation.

[0091] In this embodiment, the second coordinate axis is the horizontal coordinate axis, and the column coordinate values ​​(equivalent to the horizontal coordinate values) are set in sequence in the horizontal coordinate axis. The columns of the laser image are perpendicular to the horizontal coordinate axis, and the search partitions and search sub-partitions are perpendicular to the horizontal coordinate axis; the first coordinate axis is the vertical coordinate axis, and the row coordinate values ​​(equivalent to the vertical coordinate values) are set in sequence in the vertical coordinate axis. The rows of the laser image are perpendicular to the vertical coordinate axis, and the search partitions and search sub-partitions are parallel to the vertical coordinate axis.

[0092] In this embodiment, the laser stripes formed by the line laser emitted by the line laser emitter on the surface of the object to be measured or the ground are close to the horizontal line, and the horizontal line is close to parallel to the horizontal coordinate axis U of the pixel coordinate system. Therefore, the first coordinate axis is the longitudinal coordinate axis to represent the vertical coordinate axis of the pixel coordinate system, and the second coordinate axis is the transverse coordinate axis to represent the horizontal coordinate axis of the pixel coordinate system; therefore, the first preset search direction is pre-calibrated to be parallel to the transverse coordinate axis direction of the pixel coordinate system, and the second preset search direction is pre-calibrated to be parallel to the longitudinal coordinate axis direction of the pixel coordinate system.

[0093] Then set the coordinates of the center of gravity of the effective pixel interval to (i, v c ), and then determine to extract a light bar center point (i, v c ); where the vertical coordinate value v of the center of gravity point c Less than or equal to m, the vertical coordinate value v of the center of gravity c Fractions may be present to achieve sub-pixel accuracy.

[0094] Therefore, the coordinate positions and grayscale values ​​of the pixel points in each search sub-partition or the valid pixel interval within the search partition are weighted averaged to obtain the vertical coordinate value of the center point of the valid pixel interval, that is, the vertical coordinate value of the center point of the light bar in the corresponding search sub-partition or the search partition is extracted, and then combined with the horizontal coordinate value of the corresponding search sub-partition or the search partition, the position of the center point of the light bar in the pixel coordinate system can be determined.

[0095] If the robot, in the process of executing steps A to C, traverses the laser image column by column to obtain the center points of the light strips belonging to the corresponding search partition, and only selects the vertical coordinate value to represent the center points of the light strips (or center of gravity points) in each column, then the height information of the reflection position of the line laser on the obstacle surface can be identified.

[0096] As another embodiment, a method for extracting the centroid position point of the valid pixel interval based on the grayscale centroid method includes: when a first preset search direction is pre-calibrated to be parallel to the vertical axis direction of the pixel coordinate system, and a second preset search direction is pre-calibrated to be parallel to the horizontal axis direction of the pixel coordinate system, the horizontal coordinate value of the centroid position point of the valid pixel interval is:

[0097]

[0098] Wherein, j is the abscissa value of the pixel points within the valid pixel interval, indicating the order of the pixel points within the valid pixel interval along the second preset search direction; m is the number of all pixels within the valid pixel interval; i is the ordinate value of the pixel points within the valid pixel interval, indicating the order of the valid pixel interval within the laser image along the first preset search direction; I(j,i) is the grayscale value of pixel point (j,i). Σ represents a summation operation.

[0099] The first coordinate axis is the horizontal coordinate axis, and the column coordinate values ​​(equivalent to the horizontal coordinate values) are set in sequence in the horizontal coordinate axis. The columns of the laser image are perpendicular to the horizontal coordinate axis, and the search partitions and search sub-partitions are parallel to the horizontal coordinate axis; the second coordinate axis is the vertical coordinate axis, and the row coordinate values ​​(equivalent to the vertical coordinate values) are set in sequence in the vertical coordinate axis. The rows of the laser image are perpendicular to the vertical coordinate axis, and the search partitions and search sub-partitions are perpendicular to the vertical coordinate axis.

[0100] In this embodiment, the laser stripes formed on the surface of the object to be measured by the line laser emitted by the line laser transmitter are regarded as vertical lines perpendicular to the ground, and the vertical lines are approximately parallel to the vertical coordinate axis of the pixel coordinate system. The coordinate values ​​of the pixel points in the first preset search direction are the coordinate values ​​in the second coordinate axis of the pixel coordinate system set in the laser image. Therefore, the second coordinate axis is the vertical coordinate axis to represent the vertical coordinate axis of the pixel coordinate system, and the first coordinate axis is the horizontal coordinate axis to represent the horizontal coordinate axis of the pixel coordinate system; therefore, the first preset search direction is pre-calibrated to be parallel to the vertical coordinate axis direction of the pixel coordinate system, and the second preset search direction is pre-calibrated to be parallel to the horizontal coordinate axis direction of the pixel coordinate system.

[0101] Then set the coordinates of the center of gravity of the effective pixel interval to (v c ,i), and then determine to extract a light bar center point (v c ,i); where the horizontal coordinate value v of the center of gravity point c Less than or equal to m, the horizontal coordinate value v of the center of gravity point c Fractions may be present to achieve sub-pixel accuracy.

[0102] If the robot searches the laser image line by line to obtain the center point of the light bar belonging to the corresponding search partition during the execution of steps A to C, and only the horizontal coordinate value is selected to represent the center point of the light bar (or the center of gravity position point) in each search partition, the width information of the line laser reflected on the obstacle surface can be identified.

[0103] As an embodiment, in step B, the method of setting the current search sub-area along the second preset search direction and the reverse direction thereof with the current search center point as the center includes:

[0104] Taking the current search center point as the center, a pixel interval of a preset search distance is divided along the second preset search direction and the opposite direction, and then the divided pixel intervals form the current search sub-partition; wherein, the preset search distance is half of the preset search step length, so that along the second preset search direction, two pixel intervals of the preset search distance constitute the current search sub-partition, and the two pixel intervals of the preset search distance are distributed on both sides of the current search center point along the second preset search direction.

[0105] In this embodiment, the preset search step size is measured using the number of pixels covered by the search sub-region within the current search region to represent the laser stripe jump within a single search region. The preset search step size corresponds to the number of pixels covered by the search sub-region. The greater the number of pixels represented by the preset search distance, the smaller the laser stripe jump within a single search region; the smaller the number of pixels represented by the preset search distance, the larger the laser stripe jump within a single search region.

[0106] The assembly positions of the line laser emitter and the camera in the robot are fixed, the ground medium outside the robot is stable and the obstacles are static, and the width of the laser stripes formed by the laser line reflected by the surface of the object to be measured does not fluctuate much, so the collected light strip width range is relatively stable; therefore, a certain pixel search range can be set for each search sub-partition, and the preset search step size is set to a fixed number of pixels, preferably 10.

[0107] It should be noted that for a frame of laser image, the laser image can be converted into a grayscale image so as to regard it as a two-dimensional function, where the grayscale value of each pixel represents the intensity or brightness of the function at the corresponding position. Each pixel corresponds to a cell in the image. The grayscale value of the pixel is also expressed as a value from 0 to 255, so each frame of laser image captured by the camera can be regarded as converted into a grayscale image; in this embodiment, the closer to the center of the laser stripe or light bar, the larger the grayscale value and the brightness value; the value 0 can represent the blackest pixel, and the value 255 can represent the whitest pixel. The pixel mentioned in this embodiment is an indivisible unit in a frame of image. Each frame of laser image is composed of many pixels. It exists as a small grid of a single color, and the grayscale image uses a byte of capacity to store a pixel.

[0108] Based on the various embodiments of the aforementioned laser positioning method, the present application also discloses a robot, a body assembly line laser module of the robot, and the line laser module includes a line laser emitter and a camera; the laser line emitted by the line laser emitter is reflected back to the camera by the surface to be measured to form laser stripes, and the laser stripes are composed of laser line segments formed by the reflection of the laser line on the surface of the obstacle or the ground on which the robot is moving; the laser stripes are displayed in the image captured by the camera, so that the robot captures the laser image; the laser stripes are divided into multiple light strip partitions row by row or column by column in the laser image, and the division row by row or column by column is specifically determined according to the distribution mode of the laser stripes in the pixel coordinate system; the laser image is evenly divided into multiple search partitions along the length direction of the laser stripe, so that the laser stripe is divided into a single light strip partition in a single search partition, and the center of gravity position point, that is, the center point of the light stripe, can be calculated by the grayscale centroid method in the single light strip partition; when each search partition is presented in the form of columns Figure 2 When the pixel coordinate system UOV is shown, the laser stripes are Figure 2 The black cells in the occupies the corresponding pixel coordinate position, and it is also allowed to express the pixel coordinate in decimal form without filling the cell, so as to meet the sub-pixel accuracy. Figure 2 The white dot in the figure represents the center of the light bar, but it needs to be searched and located by executing the extraction method.

[0109] In order to reduce the amount of calculation generated by the search, the present application sets the length direction of the laser stripe to include a first preset search direction and the opposite direction of the first preset search direction; wherein, the length of the laser stripe is greater than the width of the laser stripe; and the first preset search direction can be pre-configured in the process of calibrating the camera to be parallel to one of the coordinate axes of the pixel coordinate system UOV, that is, the length direction of the laser stripe is calibrated to be parallel to the first preset search direction, which can also be understood as the first preset search direction being calibrated to the length direction of the laser stripe; wherein, the length direction of the laser stripe is determined by the assembly angle and height between the camera and the line laser emitter, specifically, the assembly angle and height between the calibrated camera and the calibrated line laser emitter. Before executing the extraction method, the corresponding internal reference calibration can be completed by a calibration plate or other calibration fixture for the line laser module to achieve the calibration of the length direction of the laser stripe. In the present application, the camera is calibrated or the coordinate axis direction of the laser image is pre-corrected to achieve that the length direction of the laser stripe is perpendicular to the width direction of the laser stripe, the width direction of the laser stripe is parallel to the second preset search direction, and the length direction of the laser stripe is parallel to the first preset search direction.

[0110] The robot is configured to execute the extraction method disclosed in the aforementioned embodiment; wherein, the center point of the light stripe calculated within a search partition corresponds to the center point of the light stripe formed by the laser stripe in the corresponding light stripe partition. Before the robot executes the extraction method, the camera is pre-calibrated, including calibration of the internal parameter matrix and distortion parameters. The relevant camera calibration function of the opencv image processing library can be called to calibrate the laser image so that the distribution direction of the pixel points within a search partition within the laser image is parallel to the width direction of the laser stripe. Before calibration, the laser image can be filtered using a Gaussian filter to reduce the noise in the image, which helps to ensure that the grayscale changes in the laser image are smoother and improve the accuracy of the center of gravity position calculation.

[0111] The robot disclosed in the present application first determines the starting center point from the initial search partition by executing steps A to C, and then uses recursive extraction of other search partitions and a combination of search center points and grayscale centroid method to extract the light stripe center point formed by the laser stripe in the corresponding light stripe partition. In addition, in each of the other search partitions, a small range of pixels is defined with the predetermined light stripe center point as the search center, and the update is started from the starting center point. There is no need to search the pixel points of the laser image row by row and column by column, thereby reducing the extraction time of the search center point without affecting the extraction accuracy of the light stripe center point, improving the extraction efficiency of the light stripe center point, and saving memory resources. Specifically, based on the aforementioned steps B1 to B7, the robot disclosed in the present application uses the center point of the light bar extracted from the previous search partition as the current search center, and uses a preset search step size to set the current search sub-partition within the current search partition, and then extracts the center point of the light bar from the current search partition by combining the grayscale threshold and grayscale centroid method within the current search sub-partition; if the center point of the light bar cannot be found in the current search sub-partition, an adjacent current search sub-partition is set along the second preset search direction or its opposite direction until the center point of the light bar is extracted, and the search for the current search partition is stopped; or if the center point of the light bar cannot be found in the current search partition, the search partition is switched along the first preset search direction; therefore, whenever the center point of the light bar cannot be extracted in a search sub-partition within a search partition, the adjacent search sub-partitions are searched within the same search partition until the center point of the light bar is extracted in a search sub-partition within the same search partition, or if the boundary of a search partition in the second preset search direction and its opposite direction has been searched without extracting the center point of the light bar, the search within the search partition is stopped. Moreover, whenever the center point of the light strip is extracted in a search partition, the search sub-partition at the new area is switched by executing step C until the boundary of the laser image in the first preset search direction is found to complete the search of all search partitions in the laser image; thereby avoiding wasting time on processing pixel areas with repeated positions, reducing the amount of calculation required for extracting the center position point of the effective pixel interval based on the grayscale centroid method, and speeding up the extraction of the light strip center point in each search partition or each light strip partition.

[0112] It should be noted that the operating principle of the line laser module is as follows: a line laser emitter emits a laser line. After reaching the surface of an obstacle, a portion of the emitted laser line is reflected back and formed into pixels on the image through the optical imaging system of the camera. Due to the different distances from the object surface to the return point, the flight time of the reflected light varies. By measuring the flight time of the reflected light, each pixel can obtain independent distance and direction information. Then, using trigonometric conversion relationships, the height and width information are obtained, and the coordinates of the pixel on the laser image are marked as the coordinate information, which is collectively referred to as position information.

[0113] As the robot moves, the line laser emitter in the line laser module is controlled to emit a laser line. When the laser line encounters obstacles in its path, it is reflected back, covering at least the ground surface and any low obstacles on the ground. Furthermore, the camera in the line laser module is controlled to capture images of the surrounding area in front of it. During this period, if the laser line is reflected by an obstacle in its path, it forms a laser line segment on the surface of the object. This laser line segment can be captured by the camera, forming laser stripes distributed across multiple rows or columns. The angle between the laser stripes formed on the surface of the object and the horizontal plane is not limited; for example, it can be parallel or perpendicular to the horizontal plane, or at any angle, depending on the application requirements.

[0114] Among them, each laser stripe contains multiple pixel points, and the center point of the light stripe of the same laser stripe in different light strip partitions will be extracted by executing steps A to C. Each light stripe center point can correspond to a point on the surface of the obstacle or the ground, and can constitute the obstacle point cloud data. After obtaining the obstacle point cloud data, the height information and length and width information of the obstacle are obtained, and the type of obstacle can be identified based on the obstacle point cloud data. In this application, it is not limited to identifying the type of obstacle and the area it occupies through obstacle point cloud data. For example, the obstacle point cloud data can be input into a deep learning model to identify the type of obstacle. Alternatively, the obstacle can be depicted based on the obstacle point cloud data to obtain the obstacle outline, and the type of obstacle can be determined based on the obstacle outline, or based on cluster analysis, threshold filtering, and confidence judgment of the obstacle points.

[0115] This application does not limit the implementation form of the line laser emitter, which can be any device / product form that can emit laser lines. For example, the line laser emitter can be but is not limited to a laser tube. Similarly, the implementation form of the camera is not limited, and all visual devices that can capture environmental images are applicable to this application. For example, the camera can include but is not limited to a monocular camera, a binocular camera, etc. In this application, the wavelength of the laser line emitted by the line laser emitter can be limited to the wavelength of infrared light. For example, it can be an infrared laser, and the emitted laser line is close to a horizontal line (closely parallel to the horizontal coordinate axis of the pixel coordinate system).

[0116] In the process of implementing the extraction method, the camera can receive laser lines of various wavelengths emitted by the line laser transmitter without installing a filter (such as an infrared filter) on the lens; of course, in some embodiments, there is no limitation on the installation position, installation angle, etc. of the line laser transmitter, as well as the installation position relationship between the line laser transmitter and the camera module. In this application, the number of line laser transmitters is also not limited, for example, it can be one, two, or more. Similarly, the number of cameras is not limited, for example, it can be one, two, or more.

[0117] In some embodiments, the camera's field of view includes a vertical field of view and a horizontal field of view. In this application, a camera with an appropriate field of view can be selected based on application requirements, as long as the laser line emitted by the line laser emitter is within the camera's field of view. The angle between the laser stripes formed by the laser line on the surface of the object and the horizontal plane is not limited. For example, it can be parallel or perpendicular to the horizontal plane, or at any angle to the horizontal plane, depending on the application requirements.

[0118] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0119] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A method for extracting the center point of a laser stripe within a search zone, wherein a robot controls a line laser emitter to emit a laser line, and the robot controls a camera to capture an image of a laser stripe formed by reflection of the laser line, and then marks the image of the laser stripe as a laser image; the laser image is evenly divided into multiple search zones along the length direction of the laser stripe, so that a single light stripe zone is divided within each search zone; the length direction of the laser stripe is a first preset search direction or its opposite direction; It is characterized in that The extraction method comprises: Step A: Select an initial search partition from the multiple search partitions divided within the laser image, extract the center of gravity of the initial search partition based on the grayscale centroid method, mark the center of gravity of the initial search partition as the starting center point, and determine that the initial search partition has been searched; then perform step B: Within the initial search partition, the distribution direction of the pixels is pre-calibrated to be parallel to the width direction of the laser stripe; the length direction of the laser stripe is perpendicular to the width direction of the laser stripe; Step B: setting a search partition adjacent to the initial search partition as the current search partition along a first preset search direction; then setting a current search center point within the current search partition based on the starting center point; then setting a current search sub-partition along a second preset search direction and its opposite direction with the current search center point as the center; then extracting the center of gravity position point of the current search sub-partition based on a grayscale centroid method, then marking the center of gravity position point of the current search sub-partition as the center point of the laser stripe within the current search sub-partition, then determining that the current search partition has been searched and that the center point of the laser stripe has been extracted from the current search partition; then executing step C; wherein the second preset search direction is perpendicular to the first preset search direction; Step C: updating the current search partition to the initial search partition, and updating the center point of the laser stripe in the current search sub-partition to the starting center point, and then repeating step B until all search partitions in the laser image have been searched, thereby extracting the center point of the laser stripe formed in each stripe partition; wherein the center point of the laser stripe in the current search sub-partition corresponds to the center point of the laser stripe formed in the corresponding stripe partition; The step B specifically includes: Step B1: Determine whether there is a valid pixel interval in the current search sub-region based on a preset grayscale value threshold. If yes, extract the valid pixel interval from the current search sub-region and then execute step B2; otherwise, execute step B3; Step B2: extracting the center of gravity of the effective pixel interval based on the grayscale centroid method, and determining the center of gravity of the current search sub-region, marking the center of gravity of the current search sub-region as the center of the laser stripe in the current search sub-region, then determining that the current search sub-region has been searched and the center of the laser stripe has been extracted from the current search sub-region, and then executing Step B5; Step B3: Determine whether the current search sub-partition currently covers the boundary of the current search partition in the second preset search direction. If so, determine that the boundary of the current search partition in the second preset search direction has been searched, and then execute step B4; otherwise, execute step B7; wherein, the boundary of the current search partition in the second preset search direction is the boundary of the current search partition pointed to by the second preset search direction; the boundary of the current search partition in the opposite direction of the second preset search direction is the boundary of the current search partition pointed to by the opposite direction of the second preset search direction; Step B4: Determine whether the boundary of the current search partition in the opposite direction of the second preset search direction has been searched. If so, execute step B5; otherwise, update the opposite direction of the second preset search direction to the second preset search direction, and then execute step B6; Step B5: determining whether the current search sub-area currently covers the boundary of the laser image in the first preset search direction; if so, determining that all search sub-areas in the laser image have been searched; otherwise, executing step C; Step B6: Update the search sub-area that is adjacent to the current search sub-area set for the first time within the current search area and covers the preset search step length to the current search sub-area, along the second preset search direction updated in step B4, and then execute step B1; Step B7: Update the search sub-area adjacent to the current search sub-area and covering the preset search step length to the current search sub-area along the second preset search direction, and then execute step B1; The preset search step size is the length of the search sub-region in the second preset search direction, and the preset search step size is measured in terms of the number of pixels.

2. The extraction method according to claim 1, wherein The step A specifically includes: Step A1: determining whether there is a valid pixel interval in the first search partition based on a preset grayscale value threshold; if so, setting the first search partition as the initial search partition, extracting the valid pixel interval from the initial search partition, and then executing step A2; otherwise, executing step A3; Step A2: extracting the centroid position point of the effective pixel interval based on the grayscale centroid method, and determining the centroid position point of the initial search partition; Step A3: determining whether the first search subarea covers the boundary of the laser image in the first preset search direction; if so, determining that all search subareas in the laser image have been searched and that the center point of the light stripe cannot be found in the laser image; otherwise, executing step A4; Step A4: Update the search partition adjacent to the first search partition along the first preset search direction to the first search partition, and then execute step A1; Wherein, when step A1 is performed for the first time, the first search partition is configured as a search partition in the laser image that is closest to a boundary of the laser image in the opposite direction of the first preset search direction; Among them, the boundary of the laser image in the opposite direction of the first preset search direction is the boundary in the laser image pointed to by the opposite direction of the first preset search direction; the boundary of the laser image in the first preset search direction is the boundary in the laser image pointed to by the first preset search direction.

3. The extraction method according to claim 2, characterized in that The method for determining whether there is a valid pixel interval within a pixel interval to be searched based on a preset grayscale value threshold includes: Step P1: Determine whether the grayscale values ​​of the pixels covered by the boundaries of the pixel interval to be searched in the second preset search direction and the opposite direction are all greater than a preset grayscale threshold. If so, determine that the pixel interval to be searched is the valid pixel interval; otherwise, execute step P2. The pixel interval to be searched in step A is the first search partition; the pixel interval to be searched in step B is the current search sub-partition; Step P2: Starting from one side boundary of the pixel interval to be searched, sequentially search for pixel points along a second preset search direction or in the opposite direction thereof, and detect the grayscale value of each pixel point; if the grayscale value of the searched pixel point is greater than a preset grayscale threshold, the searched pixel points whose grayscale value is greater than the preset grayscale threshold are used to form the valid pixel interval; if no pixel point whose grayscale value is greater than the preset grayscale threshold is found, it is determined that the valid pixel interval does not exist in the pixel interval to be searched; wherein, one side boundary of the pixel interval to be searched is the boundary of the pixel interval to be searched in the second preset search direction or the boundary of the pixel interval to be searched in the opposite direction of the second preset search direction; Alternatively, pixel points are searched in sequence starting from the two side boundaries of the pixel interval to be searched, and the grayscale value of each pixel point is detected; if a pixel point with a grayscale value greater than a preset grayscale threshold is searched for the first time starting from the boundary of the pixel interval to be searched in the second preset search direction, and a pixel point with a grayscale value greater than the preset grayscale threshold is searched for the first time starting from the boundary of the pixel interval to be searched in the opposite direction of the second preset search direction, then the pixel point searched for the first time in the second preset search direction and the pixel point searched for the first time in the opposite direction of the second preset search direction are respectively marked as the two endpoints of the valid pixel interval, and then the pixel points distributed in the pixel interval to be searched at the two endpoints together with the two endpoints form the valid pixel interval, and determine that the grayscale value of each pixel point in the pixel interval to be searched between the two endpoints is greater than the preset grayscale threshold.

4. The extraction method according to claim 3, wherein When the pixel distance covered by the current search sub-region in the laser image is less than the preset search step size, there exists: If the boundary of the current search sub-region in the second preset search direction is already set outside the laser image, updating the pixel points at the boundary of the laser image currently covered by the current search sub-region to the pixel points covered by the boundary of the current search sub-region in the second preset search direction; If the boundary of the current search sub-region in the opposite direction of the second preset search direction is already set outside the laser image, then updating the pixel points at the boundary of the laser image currently covered by the current search sub-region to the pixel points covered by the boundary of the current search sub-region in the opposite direction of the second preset search direction; There are no pixels outside the laser image.

5. The extraction method according to claim 1 or 2, characterized in that In step B, the method of setting the current search center point in the current search partition based on the starting center point includes: In the laser image, setting the coordinate value of the starting center point in the second preset search direction as the coordinate value of the current search center point in the second preset search direction; The coordinate value of the starting center point in the first preset search direction is not equal to the coordinate value of the current search center point in the first preset search direction; Among them, the coordinate value of the pixel point in the second preset search direction is the coordinate value in the first coordinate axis of the pixel coordinate system set in the laser image, and the coordinate value of the pixel point in the first preset search direction is the coordinate value in the second coordinate axis of the pixel coordinate system set in the laser image.

6. The extraction method according to claim 5, characterized in that The method for extracting the center of gravity position point of the effective pixel interval based on the grayscale center of gravity method includes: When the first preset search direction is pre-calibrated to be parallel to the horizontal axis of the pixel coordinate system, and the second preset search direction is pre-calibrated to be parallel to the vertical axis of the pixel coordinate system, the vertical coordinate value of the center of gravity position point of the valid pixel interval is: Wherein, i is the horizontal coordinate value of the pixel point in the effective pixel interval, which represents the order of the effective pixel point in the laser image along the first preset search direction; j is the vertical coordinate value of the pixel point in the effective pixel interval, which represents the order of the pixel point in the effective pixel interval along the second preset search direction; m is the number of all pixels in the effective pixel interval; I(i, j) is the grayscale value of the pixel point (i, j); the first coordinate axis is the vertical coordinate axis, and the second coordinate axis is the horizontal coordinate axis; Then set the coordinates of the center of gravity of the effective pixel interval to (i, v c ), where the vertical coordinate value v of the center of gravity point c Less than or equal to m.

7. The extraction method according to claim 5, characterized in that The method for extracting the center of gravity position point of the effective pixel interval based on the grayscale center of gravity method includes: When the first preset search direction is pre-calibrated to be parallel to the vertical axis of the pixel coordinate system, and the second preset search direction is pre-calibrated to be parallel to the horizontal axis of the pixel coordinate system, the horizontal coordinate value of the center of gravity position point of the valid pixel interval is: Wherein, j is the horizontal coordinate value of the pixel point in the effective pixel interval, which represents the order of the pixel point in the effective pixel interval along the second preset search direction; m is the number of all pixel points in the effective pixel interval; i is the vertical coordinate value of the pixel point in the effective pixel interval, which represents the order of the effective pixel interval in the laser image along the first preset search direction; I(j,i) is the grayscale value of the pixel point (j,i); the first coordinate axis is the horizontal coordinate axis, and the second coordinate axis is the vertical coordinate axis; Then set the coordinates of the center of gravity of the effective pixel interval to (v c ,i), where the horizontal coordinate value v of the center of gravity point c Less than or equal to m.

8. The extraction method according to claim 1, characterized in that In step B, the method of setting the current search sub-area along the second preset search direction and the reverse direction thereof with the current search center point as the center includes: Taking the current search center point as the center, divide the pixel intervals along the second preset search direction and the reverse direction thereof into a preset search distance, and then form the current search sub-area from the divided pixel intervals; Among them, the preset search distance is half of the preset search step size; the more pixel points represented by the preset search distance, the smaller the light bar jump of the laser stripe within a single search partition; the fewer pixel points represented by the preset search distance, the larger the light bar jump of the laser stripe within a single search partition.

9. A robot, including a line laser module for its body assembly, wherein the line laser module includes a line laser emitter and a camera; a laser line emitted by the line laser emitter is reflected back to the camera by a surface to be measured, forming a laser streak; the laser streak is displayed in an image captured by the camera, enabling the robot to capture a laser image; The laser stripe is divided into a plurality of light stripe partitions within the laser image; the laser image is evenly divided into a plurality of search partitions along the length direction of the laser stripe; It is characterized in that The robot is configured to perform the extraction method according to any one of claims 1 to 7; wherein the light stripe center point calculated within a search partition corresponds to the light stripe center point formed by the laser stripe within the corresponding light stripe partition; Before the robot executes the extraction method, the camera is pre-calibrated so that the distribution direction of pixel points in a search partition within the laser image is parallel to the width direction of the laser stripe; wherein, the length direction of the laser stripe is perpendicular to the width direction of the laser stripe, the width direction of the laser stripe is parallel to the second preset search direction, and the length direction of the laser stripe is parallel to the first preset search direction.

Images