Laser positioning method and robot based on image information

Abstract

The present invention discloses a laser positioning method and a robot based on image information. The execution subject of the laser positioning method is a robot equipped with a structured light module. The structured light module includes a line laser transmitter and a camera without an infrared filter, so that the imaging information of infrared light and the imaging information of visible light are retained in the image captured by the camera. The laser positioning method includes the following steps: the robot controls the camera to capture an image of light reflected back from the surface of an object to be measured by the line laser emitted by the line laser transmitter, and detects the brightness and darkness type of the image captured by the camera. When the robot detects that the current frame image captured by the camera is a bright frame image, the robot searches for the position of the line laser from the current frame image by executing an inter-frame tracking algorithm. When the robot detects that the current frame image captured by the camera is a dark frame image, the robot extracts the position of the line laser from the current frame image by executing a brightness centroid algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of laser data processing, and in particular to a laser positioning method and a robot based on image information. Background Art

[0002] The structured light module refers to any laser module that includes a line laser transmitter and a camera module. In the structured light module, the line laser transmitter is used to emit a line laser outward. The line laser emitted by the line laser transmitter can be located in front of the robot; the camera module can collect environmental images, and can also receive the reflected light returned by the line laser hitting the object, wherein the line laser emitted by the line laser transmitter is located within the field of view of the camera module; the line laser can help detect the contour, height and / or width of the object in the direction of travel of the robot, collectively referred to as the position information of the laser; wherein, the camera used to collect the line laser is generally provided with an infrared bandpass filter or an infrared high-pass filter, because after the line laser transmitter emits infrared light to the surface of the object, if the camera module accurately detects the position of the line laser in the imaging plane, the camera itself needs to be sensitive not only to the infrared light band reflected back by the line laser on the surface of the object, but also to light in bands such as visible light.

[0003] The camera module uses the aforementioned infrared bandpass filter or infrared high-pass filter to filter the reflected light of the laser emitted by the line laser transmitter on the surface of the obstacle, specifically, it transmits the infrared band carried by the laser, and absorbs or reflects the ambient light of other bands, so as to separate the infrared light carried by the reflected light from other interfering ambient light. However, while filtering out the non-infrared light bands, a large amount of environmental information is also lost. The infrared bandpass filter or infrared high-pass filter used in conjunction with the structured light module is relatively expensive, and is prone to introduce interference when the ambient light intensity is high, resulting in more interference points. Summary of the invention

[0004] In order to solve the above technical problems, the present invention discloses a laser positioning method and a robot based on image information under the condition that the camera is not equipped with an infrared filter, so as to receive the infrared light emitted by the linear laser transmitter reflected on the surface of the obstacle, and realize the robot tracking the reflected light of the laser on the surface of the obstacle. The specific technical solution is as follows:

[0005] A laser positioning method based on image information, wherein the execution subject of the laser positioning method is a robot equipped with a structured light module, the structured light module includes a line laser transmitter and a camera without an infrared filter, so that the imaging information of infrared light and visible light are retained in the image captured by the camera; the laser positioning method includes: the robot controls the camera to capture the image of the light reflected back from the surface of the object to be measured by the line laser emitted by the line laser transmitter, and detects the brightness type of the image captured by the camera; when the robot detects that the current frame image captured by the camera is a bright frame image, the robot searches for the position of the line laser from the current frame image by executing an inter-frame tracking algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image; when the robot detects that the current frame image captured by the camera is a dark frame image, the robot extracts the position of the line laser from the current frame image by executing a brightness centroid algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image.

[0006] Furthermore, the method in which the robot searches for the line laser position from the current frame image by executing an inter-frame tracking algorithm includes: step 1, the robot traverses the current frame image column by column, and obtains the initial pixel position in the corresponding column of the current frame image, and excludes the pixel points in the current frame image where the line laser position does not exist according to the pixel points in the corresponding column that meet the preset brightness distribution characteristics, wherein the line laser position is used to indicate the reflection position of the line laser on the surface of the object to be measured; step 2, except for the column where the pixel point where the line laser position does not exist is located, in the current column of the current frame image, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 3, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 4, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 5, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 6, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 7, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 8, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 9, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 10, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 11, the robot sets the initial pixel position in the current column to the position of the line laser reflected on the surface of the object to be measured; step 12, the robot sets the initial pixel position in the current column to the position of the line laser reflected The search center is then searched upward along the current column from the search center for pixels within a search radius, and the search center is searched downward along the current column from the search center for pixels within a search radius; then, according to the difference between the brightness values ​​of the pixels searched upward and the brightness values ​​of the pixels searched downward under the search states corresponding to the search centers determined twice adjacently, and the inter-frame matching relationship formed by the same type of values ​​of the pixels in the same column of the current frame image relative to the reference frame image, the convex hull center pixel point in the current column is screened out to update the convex hull center pixel point determined in the current column of the current frame image last time; wherein the reference frame image is configured to be Before the current frame image is collected, the robot finds the latest bright frame image where the line laser position is located; every time the search center in the current column is updated once, the convex hull center pixel point set in the current column is also updated once; step 3, according to the relationship between the brightness value of the line laser emitted by the line laser transmitter in the effective coverage area corresponding to the positioning coordinates of the previous dark frame image and the brightness value of the convex hull center pixel point in the current frame image, eliminate interference points from the convex hull center pixel points that have been screened out; after the robot traverses the convex hull center pixel points in all columns of pixel points in the current frame image to eliminate all interference points, the remaining The coordinates of the central pixel point of the convex hull are set as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, and the robot searches for the line laser position determined in each column in the current frame image to connect the laser line segments formed by the line laser emitted by the line laser transmitter on the surface of the object to be measured, and determines that the robot has searched for the line laser position from the current frame image by executing the inter-frame tracking algorithm; wherein, the line laser position determined in the same column is the position of the last updated central pixel point of the convex hull in the same column after the robot has traversed all the pixel points in the same column, and the coordinates of a line laser position are represented by the corresponding positioning coordinates.

[0007] Furthermore, in step 2, whenever a convex hull center pixel point is screened out for a search center, an adjacent pixel point searched upward or downward along the current column starting from the search center is updated as the search center, and step 2 is re-executed to obtain a new convex hull center pixel point and update the new convex hull center pixel point as the convex hull center pixel point; each of the search centers is within the coverage area of ​​a search radius relative to the initial pixel position, wherein the search radius is set to a first preset pixel distance; the screened convex hull center pixel point is the last updated convex hull center pixel point in each column of convex hull center pixel points in the current frame image; the screened convex hull center pixel point is the closest to the origin of the coordinate system of the current frame image in each column of convex hull center pixel points in the current frame image. convex hull center pixel points; the robot sets the set of pixel points that meet the convex hull feature on the current column of the current frame image to the pixel points whose brightness values ​​decrease from the center of the convex hull along the current column to the upper and lower sides respectively, and the set of pixel points composed of the convex hull center to form a convex hull, the convex hull center is the pixel point with the largest brightness value in the pixel point set, and sets the convex hull center pixel point to the pixel point belonging to the convex hull center; in the set of pixel points that meet the convex hull feature, starting from the center of the convex hull in the upward direction along the same column, the brightness value of the pixel point decreases upward along the current column and generates a first gradient value between the brightness values ​​of two adjacent pixel points, and, starting from the center of the convex hull in the downward direction along the same column, the brightness value of the pixel point decreases downward along the current column and generates a second gradient value between the brightness values ​​of two adjacent pixel points, so that the convex hull center belongs to the search center.

[0008] Further, in the step 2, the method for selecting the convex hull center pixel point according to the difference between the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward in the search states corresponding to the search centers determined twice adjacently, and the inter-frame matching relationship formed by the same type of numerical values ​​in the same column of the pixel points of the current frame image relative to the reference frame image, includes: in the current column of the current frame image, controlling the brightness value of the search center to compare with the brightness value of the convex hull center pixel point located in the same column searched last time; the convex hull center pixel point located in the same column searched last time is the convex hull center pixel point selected in the same column of the current frame image for the search center determined last time, the search center determined last time is a pixel point adjacent to the currently determined search center in the current column of the current frame image downward or upward, and the column order of the pixel points in the same column of the current frame image is the same as the search center determined last time. The column order of the current column of the current frame image is equal; if the brightness value of the currently determined search center is greater than the brightness value of the convex hull center pixel point located in the same column searched last time, then in the current column of the current frame image, search for pixels upward from the search center, and count the pixels whose brightness values ​​decrease according to the first gradient value until the upward counting stop condition is met, then mark the number of pixels whose brightness values ​​decrease according to the first gradient value as the upward gradient descent number, and stop searching for pixels upward until the next update of the search center; and search for pixels downward from the search center, and count the pixels whose brightness values ​​decrease according to the second gradient value until the downward counting stop condition is met, then mark the number of pixels whose brightness values ​​decrease according to the second gradient value as the downward gradient descent number, and stop searching for pixels downward until the next update of the search center;When the robot determines that the number of upward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents required to be counted for the last search for the center pixel point of the convex hull located in the same column, and / or determines that the number of downward gradient descents counted in the current column of the current frame image is greater than or equal to the number of downward gradient descents required to be counted for the last search for the center pixel point of the convex hull located in the same column, among the pixels traversed by the robot in the current column of the current frame image, if it is detected that the first gradient value and the second gradient value are not equal to the first preset gradient parameter, and the absolute value of the difference between the first gradient value and the second gradient value is less than the second preset gradient parameter, If the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched upward along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between the brightness values ​​of the same type formed by searching upward in the same column of pixels in the reference frame image, and the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downward along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference between the brightness values ​​of the same type formed by searching downward in the same column of pixels in the reference frame image, then the robot marks the currently determined search center as the convex hull center pixel; wherein the first preset gradient parameter is less than the second preset gradient parameter. ;

[0009] Further, the absolute value of the difference in brightness values ​​of the same type formed by an upward search among pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched upward in a column with the same column order as the current column and the brightness value of the pixel at the search center finally determined on the same column in the reference frame image, starting from the search center finally determined in a column with the same column order as the current column, and the distance between the pixel with the smallest brightness value searched upward and the search center finally determined on the same column is less than or equal to the search radius; the absolute value of the difference in brightness values ​​of the same type formed by a downward search among pixels in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downward in a column with the same column order as the current column and the brightness value of the pixel at the search center finally determined on the same column in the reference frame image, starting from the search center finally determined in a column with the same column order as the current column, and the distance between the pixel with the smallest brightness value searched downward and the search center finally determined on the same column is less than or equal to the search radius.

[0010] Furthermore, for a currently determined search center, step 2 also includes: if the brightness value of the pixel point at the search center is greater than the brightness value of the convex hull center pixel point located in the same column searched last time, then in the current column of the current frame image, the pixel point is searched upward from the search center, and the pixel point is searched downward from the search center; if the robot detects that the brightness value of the pixel point is not decreasing according to the first gradient value during the upward search from the search center, the preset upward gradient abnormality count number is counted once, and then the robot determines whether it has searched the pixel points covered within the search radius upward along the current column of the current frame image, and if so, the robot stops. Stop searching for pixels along the current column of the current frame image upward and determine whether the upward counting stop condition is met, otherwise when the upward gradient anomaly frequency is greater than the first preset error number, stop searching for pixels along the current column of the current frame image upward and determine whether the upward counting stop condition is met; and, in the process of searching downward from the search center, if it is detected that the brightness value of the pixel point is not decreasing according to the second gradient value, the preset downward gradient anomaly count number is counted once, and then the robot determines whether it has searched the pixels covered within the search radius downward along the current column of the current frame image, and if so, the robot stops searching for pixels along the current column of the current frame image downward and determines whether the upward counting stop condition is met. The robot determines whether the upward counting stop condition is met, otherwise, when the upward gradient abnormal frequency is greater than the second preset error number, the robot stops searching for pixels along the current column of the current frame image downward and determines that the downward counting stop condition is met; or, in the process of searching upward from the search center, the robot counts the pixels whose brightness value is 255 and whose positions are adjacent to each other upward along the current column of the current frame image, and marks the number of pixels whose brightness value is 255 and whose positions are adjacent to each other as the number of upward overexposures. When the robot detects that the number of upward overexposures is greater than the third preset error number, and / or the pixel points covered within the search radius are counted upward along the current column of the current frame image, the robot Stop searching for pixels along the current column of the current frame image upward and determine that the upward counting stop condition is met; and in the process of searching downward from the search center, the robot counts the pixels whose brightness value is 255 and whose positions are adjacent to each other downward along the current column of the current frame image, and marks the number of pixels whose brightness value is 255 and whose positions are adjacent to each other as the downward overexposure number. When the robot detects that the upward overexposure number is greater than the fourth preset error number, and / or the pixel points covered within the search radius are counted downward along the current column of the current frame image, the robot stops searching for pixels along the current column of the current frame image downward and determines that the downward counting stop condition is met.

[0011] Further, in the step 3, the method of removing interference points from the screened convex hull center pixel points according to the size relationship between the brightness value in the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image and the brightness value of the convex hull center pixel point in the current frame image includes: after the robot traverses all columns of pixel points in the current frame image and obtains the latest convex hull center pixel point in each column, and saves the positioning coordinates of the line laser emitted by the line laser emitter in the previous dark frame image, for each convex hull center pixel point in the current frame image The center pixel point of the convex hull is within a circular domain with the position of the positioning coordinates of the line laser emitted by the line laser transmitter in the previous frame of dark image as the center of the circle and the radius as the detection pixel distance. If the robot determines that there is at least one pixel point in the circular domain whose brightness value is greater than the brightness value of the center pixel point of the convex hull with the same coordinates as the center of the circle in the current frame image by a preset ambient light brightness threshold, the robot determines that the center pixel point of the convex hull with the same coordinates as the center of the circle in the current frame image is an interference point, the robot cannot find the line laser position at the interference point, and removes the interference point from the current frame image.

[0012] Furthermore, in step 1, the method for excluding pixel points where no line laser position exists in the current frame image based on pixel points in the corresponding column that meet the preset brightness distribution characteristics includes: if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel point at the line laser position in the same column found in the previous round by a first preset brightness threshold, or the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel point at the line laser position in the same column found in the previous round by a second preset brightness threshold, then starting from a position one reference pixel distance upward from the initial pixel position in the current column of the current frame image, searching for pixel points downward along the current column of the current frame image; if it is detected that the brightness value of a pixel point currently being searched is greater than the brightness value of the pixel point at the line laser position in the same column found in the previous round by a first preset brightness threshold. Brightness threshold, or detect that the brightness value of a pixel point currently being searched is equal to the value 255, then count the number of error position counts once, and determine that the pixel point currently searched is a pixel point that meets the preset brightness distribution characteristics; when the robot detects that the number of error position counts is greater than the reference pixel count threshold, it is determined that there is no line laser position in the current column of the current frame image, then the pixel points in the current column of the current frame image are set to the pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel search range of step 2, and at the same time determine that the light intensity of the robot's environment is greater than the first preset light intensity threshold; wherein, the reference pixel distance is expressed by the number of pixels, so that the reference pixel count threshold is equal to the reference pixel distance; wherein, the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image.

[0013] Furthermore, in step 1, the method for excluding pixel points where there is no line laser position in the current frame image according to pixel points in the corresponding column that meet the preset brightness distribution characteristics includes: taking the initial pixel position in the current column of the current frame image as the center of the ring, in the current column of the current frame image, marking the pixel points covered by the ring area below the center of the ring with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius as the first pixel points to be tested, and then calculating the average brightness value of the first pixel points to be tested. If the average brightness value of the first pixel points to be tested is greater than the average brightness value of the pixel points located in the previous round, the pixel points are marked as the first pixel points to be tested. The brightness value of the pixel point at the line laser position in the same column is determined, then the first pixel point to be tested is determined to be a pixel point that meets the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image, then the pixel points in the current column of the current frame image are set to the pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel point search range of step 2, and at the same time, it is determined that the light intensity of the environment in which the robot is located is greater than the first preset light intensity threshold; wherein the first positioning radius is smaller than the second positioning radius, and the line laser position in the same column found in the previous round belongs to the reference The position of the convex hull center pixel point finally determined in the same column of pixel points of the frame image; or, taking the initial pixel position in the current column of the current frame image as the center of the ring, in the current column of the current frame image, the pixel point covered by the ring area above the center of the ring, with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius is marked as the second pixel point to be tested, and then the average brightness value of the second pixel point to be tested is calculated. If the average brightness value of the second pixel point to be tested is greater than the brightness value of the pixel point at the line laser position in the same column found in the previous round, it is determined that the second pixel point to be tested is Pixel points that meet the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image, then the pixel points in the current column of the current frame image are set to pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel search range of step 2, and at the same time, it is determined that the light intensity of the robot's environment is greater than the first preset light intensity threshold; wherein the first positioning radius is smaller than the second positioning radius, and the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image.

[0014] Furthermore, the initial pixel position is the position of the original pixel point in the image captured by the camera after the line laser emitted by the line laser transmitter is reflected on the robot's travel plane back to the camera's field of view when there is no obstacle in front of the robot; each original pixel point corresponds to a reflection position on the robot's travel plane, and is used to represent the search starting point for searching the line laser position in each column of the same frame image; the reference frame image is configured as a bright frame image where the line laser position most recently found by the robot is located before the current frame image is captured, wherein the line laser position most recently found by the robot is derived from the convex hull center pixel point set in the corresponding column of the reference frame image.

[0015] Furthermore, in step 1, if the initial pixel position cannot be obtained in the current column of the current frame image, the line laser position located in the same column found in the previous round is updated to the initial pixel position, and the second preset pixel distance is updated as the search radius, and step 2 is repeated to search for the convex hull center pixel point in the corresponding column; wherein, the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image or the initial pixel position in the same column of pixels of the first bright frame image; if the robot still cannot search for the convex hull center pixel point in the same column during the process of repeatedly executing step 2, it is determined that the robot cannot find the line laser position in the same column.

[0016] Furthermore, the method in which the robot extracts the line laser position from the current frame image by executing the brightness centroid algorithm includes: the robot traverses the current frame image column by column; the robot searches for each pixel point in the current column in turn, and based on the magnitude relationship between the brightness value of the pixel point currently searched out in the current column of the current frame image and the brightness value of the pixel point at the corresponding position of the previous bright frame image and the brightness value of the pixel point at the corresponding position of the previous bright frame image, the robot selects legal pixel points from the current column of the current frame image; then, in the current column of the current frame image, at least two legal pixel points adjacent to each other are connected to form a positioning line segment; after all legal pixel points adjacent to each other are connected, the positioning line segment with the longest length is selected; if the length of the selected positioning line segment with the longest length is greater than a preset continuous length threshold, the center of the selected positioning line segment with the longest length is set as the line laser position.

[0017] Furthermore, based on the magnitude relationship between the brightness value of the pixel currently being searched in the current frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image, as well as the brightness value of the pixel at the corresponding position of the previous bright frame image, a method for screening out legal pixels from the current column of the current frame image includes: subtracting the brightness value of the pixel at the same row and column position of the previous bright frame image from the brightness value of the pixel currently being searched in the current frame image to obtain a relative difference of the dark frame image; when it is detected that the inverse of the relative difference of the dark frame image is greater than a preset brightness difference threshold, and the brightness value of the pixel at the same row and column position of the previous bright frame image is greater than a reference bright frame image brightness threshold, setting the pixel currently being searched in the current frame image as the legal pixel.

[0018] Furthermore, the image sequence formed by the light reflected back from the surface of the object to be measured by the line laser emitted by the line laser transmitter and captured by the camera is configured to generate bright frame images and dark frame images alternately in sequence, so that: when the current frame image captured by the camera is a bright frame image, the next frame image captured by the camera is a dark frame image; within the time interval between the camera capturing the current frame bright frame image and the camera capturing the next frame bright frame image, the camera captures the current frame dark frame image; after the camera captures the next frame bright frame image, the camera captures the next frame dark frame image; wherein, in the process of executing the laser positioning method, the first frame image of the image sequence is a bright frame image.

[0019] Furthermore, the laser positioning method also includes: when the robot detects that the light intensity of its environment is greater than a first preset light intensity threshold, the robot reduces the gain of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not overexposed; when the robot detects that the light intensity of its environment is greater than the first preset light intensity threshold, the robot reduces the exposure time of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not overexposed; when the robot detects that the light intensity of its environment is less than a second preset light intensity threshold, the robot increases the gain of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not underexposed; when the robot detects that the light intensity of its environment is less than the second preset light intensity threshold, the robot increases the exposure time of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not underexposed.

[0020] Furthermore, when the robot detects that the current exposure value of the camera is greater than a first preset exposure threshold, the power gear of the line laser emitter for emitting the line laser is increased so that the intensity of the line laser emitted by the line laser emitter is configured to be equal to the product of the smoothing coefficient and the current exposure value; when the robot detects that the current exposure value of the camera is less than a second preset exposure threshold, the power gear of the line laser emitter for emitting the line laser is lowered so that the intensity of the line laser emitted by the line laser emitter is configured to be equal to the product of the smoothing coefficient and the current exposure value; wherein the first preset exposure threshold is greater than the second preset exposure threshold, and the current exposure value of the camera is used to reflect the exposure of the camera in an environment with a current light brightness; and the smoothing coefficient is used to smooth the step size of the exposure value adjustment so that the robot can search for the line laser position from the current frame image.

[0021] A robot, the body of which is equipped with a structured light module, the structured light module comprising a line laser transmitter and a camera without an infrared filter, so that the imaging information of infrared light and visible light are retained in the image captured by the camera; a controller is arranged inside the robot, the controller is electrically connected to the structured light module, and the controller is configured to execute the laser positioning method to obtain the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image; wherein the line laser emitted by the line laser transmitter is within the field of view of the camera.

[0022] Furthermore, the horizontal viewing angle of the camera is configured to receive the light reflected back by the line laser within the width of the robot body in front of the robot; and / or the installation height of the structured light module on the robot body is configured to be positively correlated with the height of the obstacle to be measured, so that the obstacle to be measured occupies the effective field of view of the camera.

[0023] Furthermore, the coverage range of the upper viewing angle of the camera is configured to cover the bottom of the plane formed by the line laser emitted by the line laser transmitter; the coverage range of the lower viewing angle of the camera is configured to cover the light reflected by the line laser emitted by the line laser transmitter from the surface of the obstacle in front of the robot's body; and / or the heading angle formed by the deflection of the camera relative to the central axis of the robot is maintained within a preset error angle range, so that the optical axis of the camera is parallel to the direction of travel of the robot, and the camera receives the light reflected by the line laser within the width of the robot's body in front of the robot; and / or the roll angle generated by the rotation of the camera along its optical axis is maintained within a preset error angle range, so that the camera receives the light reflected by the line laser within the width of the robot's body in front of the robot, wherein the camera is rotatably mounted on the robot's body.

[0024] Furthermore, if the installation distance between the camera and the line laser module is larger, the coordinate offset of the pixel point representing the reflection position of the line laser on the surface of the obstacle relative to the center of the camera in the image captured by the camera increases.

[0025] Furthermore, the emission angle of the line laser transmitter and the receiving angle of the camera are set as follows: the line laser transmitter transmits a line laser to a preset detection position in front of the body, and the line laser is reflected back to the camera at the preset detection position, wherein the length of the laser line segment formed by the line laser at the preset detection position is greater than the body width of the robot; whenever the robot walks a preset travel distance in the direction from the current position to the preset detection position, the horizontal distance between the preset detection position and the robot becomes smaller, and the coordinate offset of the pixel point in the image captured by the camera for representing the same reflection position of the line laser in the preset detection position relative to the center of the camera increases.

[0026] The technical effect of the present invention is that: in the process of executing the laser positioning method to track the image of the reflected light of the line laser, there is no need to use an infrared filter to filter the ambient light, so that all the details of the infrared and visible light bands are retained in the collected image, which makes it convenient for the robot to search for the position information associated with the pixel points formed by the outgoing laser in the imaging plane of the camera from the current frame image, including adopting appropriate algorithms (such as pixel matching algorithms and pixel search algorithms) in the bright frame image and the dark frame image respectively to extract the outgoing laser position to achieve laser positioning, which can then be used for map navigation positioning and deep learning for identifying obstacles.

[0027] When the robot detects that the current frame image captured by the camera is a bright frame image, it chooses to input the current frame image into the processing rule model corresponding to the inter-frame tracking algorithm to output a valid laser position, effectively filtering out various ambient light interferences in scenes where the camera is not too close to obstacles, and reducing dependence on infrared filters; specifically, in the pixels that meet the convex hull features, based on the numerical relationship between the brightness value gradient generated in the upward searched pixel points and the brightness value gradient generated in the downward searched pixel points and the difference between the search states corresponding to the search centers determined twice adjacently, the relationship between the brightness value of the currently searched pixel point and the brightness value of the convex hull center pixel point determined in the same column of the same frame image last time, and the inter-frame matching formed by the same type of values ​​in the same column of the current frame image relative to the reference frame image According to the matching relationship, the convex hull center pixel of the current column is screened out, and each pixel within the search radius relative to the initial pixel position is traversed in the current column and the convex hull center pixel is updated. After excluding the interference of the pixel where the line laser position does not exist, the final convex hull pixel of the current column is determined. It can reduce the misjudgment of interference points under strong ambient light conditions, thereby searching for a more accurate convex hull center pixel by tracking the number and brightness values ​​of pixels that meet the convex hull features of the reference frame image. After eliminating all interference points, the coordinates of the remaining convex hull center pixels are set as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, so as to more accurately realize the robot's tracking of the reflected light of the laser on the obstacle surface. It is suitable for robot navigation walking scenes to achieve the effect of robot locating obstacles. On the other hand, when the robot detects that the current frame image captured by the camera is a dark frame image, it chooses to input the current frame image into the processing rule model corresponding to the brightness centroid algorithm to output a positioning line segment with a reasonable connection length, so as to overcome the problem of being sensitive to ambient light interference in the scene where the camera is too close to the obstacle, and correspondingly, prevent the robot from hitting the obstacle reflecting the laser light in front due to misjudgment. Therefore, the present invention combines the inter-frame tracking algorithm and the brightness centroid algorithm to complement each other in various ambient light intensity scenes, and realizes laser positioning in the bright frame image and dark frame image that are generated alternately and successively for reflecting the reflected light of the line laser.

[0028] In addition, the present invention also introduces a dynamic adjustment method for the exposure value of the camera, which adjusts the gain and exposure time of the camera according to the current environmental conditions, so that the image seen in the camera does not appear overexposed or underexposed; on this basis, according to the adjusted camera gain and exposure time, the power level of the line laser transmitter for emitting the line laser is adjusted to achieve the use of a stronger line laser emission power level in a high-brightness environment, so that the obstacle can also see the line laser when the ambient light is brighter, and the image will not be overexposed (for example, under strong outdoor ambient light, after the line laser is reflected back to the camera by a white obstacle, the gain or exposure time of the camera is reduced), avoiding overexposure of the image collected by the camera due to the overbright environment, thereby finding a more accurate line laser position; in a low-brightness environment, a weaker line laser emission power level is used, so that the obstacle image is not so strongly overexposed when the ambient light is darker, thereby not generating more reflection interference, and facilitating a more accurate line laser position (corresponding to the convex hull center). BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The present invention discloses a flow chart of a laser positioning method based on image information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0031] The embodiment of the present invention discloses a laser positioning method based on image information, which specifically locates the reflection position of the laser light on the surface to be measured, and adaptively screens out representative laser line position information based on the change in brightness value of pixel points in two related frames of images captured by a camera (corresponding to the change in ambient light intensity), thereby overcoming the interference of ambient light and improving the obstacle detection accuracy and the obstacle avoidance efficiency of the robot. The executor of the laser positioning method disclosed in the embodiment of the present invention is a robot that relies on structured light navigation and positioning. The robot is equipped with a structured light module, which includes a line laser transmitter and a camera without an infrared filter, so that the imaging information of infrared light and visible light are retained in the image collected by the camera; the line laser emitted by the line laser transmitter is located within the field of view of the camera, and the line laser emitted by the line laser sensor can be projected onto the surface of an obstacle. The field of view of the camera covers all or part of the outline of the obstacle. In general navigation and positioning scenarios, when the robot moves indoors or outdoors, the structured light module arranged on the robot can be used to detect whether there is an obstacle in front of the robot's travel direction. When the robot moves towards the obstacle, the detection accuracy of the reflection position of the obstacle surface is affected by executing the laser positioning method, thereby improving the positioning and obstacle avoidance accuracy.

[0032] It should be noted that the structured light module used in the embodiment of the present application generally refers to any sensor module including a line laser emitter and a camera. In the structured light module, the line laser emitter is used to emit a line laser outward. Among them, the line laser emitted by the line laser emitter can be located in the effective detection area in front of the robot, and the camera can sequentially collect multiple frames of images under various ambient light conditions, including infrared light imaging information and visible light imaging information, wherein the visible light imaging information can be directly used to construct a map and mark the position of obstacles in the map. In this embodiment, it is mainly based on receiving the image of the reflected light returned by the line laser hitting the object to be measured, and it is necessary to overcome the interference of ambient light of different bands. The line laser emitted by the line laser emitter is located within the field of view of the camera and forms a laser line segment on the surface of the object to be measured or on the horizontal ground. The line laser can help detect the contour, height and / or width of the object in the direction of travel of the robot. This embodiment mainly extracts the height information of the object to meet the needs of the inter-frame tracking algorithm. Compared with perception solutions based on image sensors, line laser transmitters can provide cameras with more accurate pixel height and direction information, which can reduce the complexity of perception operations and improve real-time performance.

[0033] Specifically, the working principle of the structured light module is as follows: the line laser emitter emits a line laser outward, and after reaching the surface of the obstacle, a part of the emitted line laser is reflected back and forms a pixel point on the image through the optical imaging system in the camera. Since the distance from the surface of the object to the return point is different, the flight time of the reflected light is different. By measuring the flight time of the reflected light, each pixel point can obtain independent distance information and direction information, and then use the trigonometric conversion relationship to obtain height information and width information, and mark it as the coordinate information of the pixel point on the image, which is collectively referred to as position information. During the robot's movement, on the one hand, the line laser emitter in the structured light module can be controlled to emit a line laser outward, and the line laser will be reflected back after encountering an obstacle on the path of travel, at least covering the ground medium and the low obstacles on the ground; on the other hand, the camera in the structured light module is controlled to collect the environmental image in the front area. During this period, if the obstacle on the path of travel detected by the line laser will form a laser line segment on the surface of the object, and the laser line segment can be collected by the camera, that is, the image collected by the camera will contain the laser line segment formed by the line laser emitted by the line laser emitter after encountering the object. As for the angle between the laser line segment formed by the line laser on the surface of the object and the horizontal plane, there is no limitation. For example, it can be parallel or perpendicular to the horizontal plane, or it can be at any angle with the horizontal plane, which can be determined according to the application requirements. Among them, each laser line segment contains multiple pixels, each pixel corresponds to a point on the obstacle surface, and the points on the obstacle surface represented by the pixels on the laser line segment in a large number of environmental images can form obstacle point cloud data. The coordinate system used by these obstacle point cloud data can be the coordinate system where the robot is located. The robot can convert the pixel coordinates on the laser line segment to the coordinate system where the robot is located according to the conversion relationship between the image coordinate system where the camera is located and the coordinate system where the robot is located, and obtain the obstacle point cloud data. Alternatively, the coordinate system used by these obstacle point cloud data can also be the world coordinate system. The robot can convert the pixel coordinates on the laser line segment to the coordinate system where the robot is located according to the conversion relationship between the coordinate system where the camera is located, the coordinate system where the robot is located, and the world coordinate system. Obstacle point cloud data; obstacle point cloud data can include but not limited to the three-dimensional coordinate information, color information, reflection intensity information, etc. of the point. After obtaining the obstacle point cloud data, the height information and length and width information of the obstacle are obtained, and then the type of obstacle can be identified based on the obstacle point cloud data. In the embodiment of the present application, it is not limited to identifying the type of obstacle and the area it occupies through the obstacle point cloud data. For example, the obstacle point cloud data can be input into a deep learning model to identify the type of obstacle. Alternatively, the obstacle can be depicted based on the obstacle point cloud data to obtain the obstacle point and obstacle outline, and the type of obstacle can be determined based on the obstacle outline, or based on cluster analysis of the obstacle points, threshold filtering, and confidence judgment.

[0034] Specifically, the implementation form of the line laser transmitter is not limited, and it can be any device / product form that can emit line laser. For example, the line laser transmitter can be but is not limited to a laser tube. Similarly, the implementation form of the camera is not limited, and all visual devices that can collect environmental images are applicable to the embodiments of the present application. For example, the camera can include but is not limited to a monocular camera, a binocular camera, etc. In the embodiments of the present application, the wavelength of the line laser emitted by the line laser transmitter can be limited to the wavelength of infrared light, for example, it can be an infrared laser. In the process of implementing the laser positioning method, the camera can receive various wavelengths of line laser light emitted by the line laser transmitter without assembling a filter (such as an infrared filter) on the lens; of course, in some embodiments, the installation position, installation angle, etc. of the line laser transmitter, as well as the installation position relationship between the line laser transmitter and the camera module, etc. are not limited. In the embodiments of the present application, the number of line laser transmitters is not limited, for example, it can be one, or two or more. Similarly, the number of cameras is not limited, for example, it can be one, or two or more.

[0035] In some embodiments, the field of view of the camera includes a vertical field of view angle and a horizontal field of view angle. In this embodiment, a camera with a suitable field of view angle can be selected according to application requirements, as long as the line laser emitted by the line laser transmitter is within the field of view of the camera. As for the angle between the laser line segment formed by the line laser on the surface of the object and the horizontal plane, it is not limited. For example, it can be parallel or perpendicular to the horizontal plane, or it can be at any angle with the horizontal plane, which can be determined according to application requirements.

[0036] In some embodiments, the installation height of the structured light module composed of a line laser emitter and a camera needs to be determined according to the size of the obstacle to be detected. If the installation height of the structured light module in the robot is higher, the longitudinal space covered in front of the robot will be larger, which will make the detection accuracy of smaller obstacles worse; if the installation height of the structured light module in the robot is lower, the longitudinal space covered in front of the robot will be smaller, which will improve the detection accuracy of smaller obstacles. Preferably, the line laser emitter is installed above the camera without an infrared filter, and the center line of the line laser emitter intersects with the center line of the camera at one point.

[0037] See also Figure 1It can be seen that the laser positioning method based on image information disclosed in the present invention includes: a robot controls a camera to collect an image of light reflected back from the surface of an object to be measured by a line laser emitted by a line laser transmitter, and detects the brightness and darkness type of the image collected by the camera, specifically distinguishing between a bright frame image and a dark frame image, that is, the robot detects whether each frame of image collected by the camera in sequence is a bright frame image or a dark frame image; in some embodiments, the robot controls the line laser transmitter to emit a line laser and controls the camera to collect an image of light reflected back from the surface of the object to be measured while moving toward a predetermined target position, the structured light module works in a certain manner, and the line laser transmitter emits the line laser externally according to a preset modulation period and emission power level; the camera periodically collects images to obtain a set of image sequences, a set of image sequences includes data of at least one frame of image, each frame of image includes a laser line segment formed when the line laser hits the surface of an object or the ground, a laser line segment includes multiple coordinate data, and the coordinate data on the laser line segments in a large number of environmental images can form point cloud data.

[0038] Specifically, the method for detecting whether the image captured by the camera is a bright frame image or a dark frame image includes: controlling the line laser emitter to emit a line laser according to a preset modulation period, when the line laser is an infrared laser modulation signal, the infrared laser modulation signal outputs a first level (corresponding to a logic high level) in the first modulation sub-period, and after being reflected by the object to be measured, it is captured by the camera to form a bright frame image; the infrared laser modulation signal outputs a second level (corresponding to a logic low level) in the first modulation sub-period, and after being reflected by the object to be measured, it is captured by the camera to form a dark frame image; then, the imaging plane reflected to the camera within one sampling period is sequentially a frame of bright frame images and a frame of dark frame images, and the robot will set an image structure (structure information of the image data) for each frame of the image obtained from the camera, and then cache it and mark the brightness and darkness attributes of the line laser accordingly, wherein each frame of the image can be saved as the previous frame of the image for tracking and matching. Under the configuration of the robot, the image sequence formed by the light reflected back from the surface of the object to be measured by the line laser emitted by the line laser transmitter in the imaging plane of the camera is configured to be alternately generated as bright frame images and dark frame images, so that: when the current frame image captured by the camera is a bright frame image, the next frame image captured by the camera is a dark frame image; within the time interval between the camera capturing the current frame bright frame image and the camera capturing the next frame bright frame image, the camera captures the current frame dark frame image, and the time interval is equal to a sampling period of the camera; after the camera captures the next frame bright frame image, the camera captures the next frame dark frame image. In which, during the execution of the laser positioning method, the first frame image of the light reflected by the line laser on the surface of the object to be measured and captured by the camera is a bright frame image, which is recorded as the first frame image captured after the line laser transmitter emits the line laser, and can also be recorded as the first frame bright frame image; then the second frame image of the light reflected by the line laser on the surface of the object to be measured and captured by the camera is a dark frame image, which is recorded as the second frame image after the line laser transmitter emits the line laser, and can also be recorded as the first frame dark frame image; then, the third frame image of the light reflected by the line laser on the surface of the object to be measured and captured by the camera is a bright frame image, which is recorded as the third frame image after the line laser transmitter emits the line laser, and can also be recorded as the second frame bright frame image, and then the next frame image captured by the camera is a dark frame image, then the robot distinguishes and marks the bright frame image and the dark frame image in sequence from a group of image sequences captured by the camera based on the above-mentioned alternating generation method according to the sampling period of the camera.

[0039] Preferably, the distinction between bright frame images and dark frame images can be achieved based on the average grayscale value of the image. Specifically, each time a frame of image is collected, all pixels of the frame image are first traversed, the sum of the brightness values ​​of all pixels is accumulated, and then the quotient of the sum of the brightness values ​​and the number of pixels is calculated as the average brightness value of the frame image. When the robot detects that the brightness values ​​of the preset threshold number of pixels in the frame image are all greater than the average brightness value, the frame image is set as a bright frame image, which can be used to search for the pixels that meet the optimal convex hull condition in the positioning scene with strong ambient light, allowing the camera exposure to increase; when the robot detects that the brightness values ​​of the preset threshold number of pixels in the frame image are all less than the average brightness value, the frame image is set as a dark frame image, which can be used for the pixels that meet the optimal convex hull condition in the positioning scene with weak ambient light, allowing the camera exposure to decrease; improve the adaptability of the laser positioning method to ambient light intensity. Wherein, the preset threshold number is greater than or equal to the number of all pixels in the frame image.

[0040] When the robot detects that the current frame image captured by the camera is a bright frame image, the robot searches for the line laser position from the current frame image by executing the inter-frame tracking algorithm, and then sets the coordinates of the line laser position to the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, so as to realize the positioning of the pixel position of the line laser; wherein, the robot sequentially inputs each frame image captured by the camera into the processing rule model corresponding to the inter-frame tracking algorithm, and outputs the line laser position in the corresponding bright frame image, so as to obtain the laser line segment connected by each line laser position, so as to facilitate the positioning of the object to be measured; input the inter-frame tracking algorithm The image in the corresponding processing rule model can be divided into the current frame image and the previous frame image, or the current frame image and the next frame image. The matching relationship between the previous bright frame image and the current bright frame image, and / or the matching relationship between the previous dark frame image and the current bright frame image can be used to track the reflection position of the line laser, and effectively filter out various ambient light interferences, especially strong ambient light interference, in scenes where the camera is not too close to obstacles, overcome the influence of the longitudinal jump of the pixel points caused by the change in the distance between the camera and the object to be measured, tend to obtain a more accurate line laser position, and reduce dependence on infrared filters.

[0041] When the robot detects that the current frame image captured by the camera is a dark frame image, the robot extracts the line laser position from the current frame image by executing the brightness centroid algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, thereby realizing the positioning of the complete pixel position of the line laser in the process of alternatingly executing the inter-frame tracking algorithm and the brightness centroid algorithm; wherein, the robot selects to input each frame image into the processing rule model corresponding to the brightness centroid algorithm, and outputs the effective line laser position in the corresponding dark frame image, and the effective line laser position is the line laser position with predictive significance determined at the moment when the dark frame image is captured, so as to assist the line laser position in the corresponding bright frame image to connect a relatively accurate laser line segment. Since the robot only pays attention to the sensitivity of the change of the ambient light intensity and ignores the distance of the robot relative to the obstacle during the execution of the inter-frame tracking algorithm, the robot is prone to collide with the obstacle during walking. Therefore, after the current frame image acquired is switched from the bright frame image to the dark frame image, the robot switches to execute the brightness centroid algorithm, so that in the scene where the camera is too close to the obstacle, it can not only notice the existence of the obstacle, but also timely identify the reflection position of the line laser to prevent the robot from colliding with the obstacle in front. As for the misjudgment problem of the line laser position caused by the interference of ambient light, after the current frame image acquired is switched from the dark frame image to the bright frame image, the robot overcomes the corresponding misjudgment problem by executing the inter-frame tracking algorithm. Therefore, the embodiment of the present invention switches the execution of the inter-frame tracking algorithm and the brightness centroid algorithm to cope with the interference of the ambient light intensity, enhances the stability of the robot's laser positioning in various walking environments, and realizes the stable completion of laser positioning in the bright frame image and the dark frame image that are alternately generated to reflect the reflected light of the line laser.

[0042] In summary, in the process of executing the laser positioning method to track the image of the reflected light of the line laser, there is no need to use an infrared filter to filter the ambient light, so that all the details of the infrared and visible light bands are retained in the captured image, which makes it easy for the robot to search for the position information associated with the pixel points formed by the line laser in the imaging plane of the camera from the current frame image, including adopting appropriate algorithms (such as pixel matching algorithms and pixel search algorithms) in the bright frame image and the dark frame image respectively, and complementarily extracting the positions of each line laser to realize laser positioning of obstacles of different distances in various ambient light intensity scenes. The robot can overcome the interference of ambient light of different intensities in various walking environments, and can be used for map navigation positioning and deep learning for identifying obstacles.

[0043] As an embodiment, the method in which the robot searches for the position of the outgoing laser from the current frame image by executing an inter-frame tracking algorithm includes:

[0044] Step 1: The robot traverses the current frame image column by column, and obtains the initial pixel position in the current column of the current frame image. Generally, an initial pixel position can be obtained in each column of the current frame image, and an initial pixel position is used as the search starting point for searching for pixel points that meet the optimal convex hull condition in the column where it is located; it should be noted that the initial pixel position is the position of the original pixel point in the image collected by the camera after the line laser emitted by the line laser transmitter is reflected back to the camera's field of view on the robot's travel plane (generally the ground) when there is no obstacle in front of the robot (or an obstacle within the field of view of the camera). At this time, the line laser transmitter or the camera has been calibrated; preferably, the robot's travel plane can be represented by the surface of the object to be measured; each original pixel point is a reflection position on the corresponding robot's travel plane, which is used to represent the search starting point for searching the line laser position in each column of the same frame image, and each original pixel point obtained in the same frame image is preferably located on the same row, and may include adjacent pixel points in the same row. Wherein, the object to be measured can be an obstacle protruding on the robot's travel plane.

[0045] At the same time, the pixel points where the line laser position does not exist in the current frame image are excluded according to the pixel points in the corresponding column that meet the preset brightness distribution characteristics, so as to exclude the interference of the pixel points with strong ambient light interference before starting to search for the pixel points that meet the convex hull characteristics in the current frame image, wherein the line laser position is used to indicate the reflection position of the line laser searched by the robot in the current frame image on the surface of the object to be measured. In this embodiment, the pixel points where the line laser position does not exist in the current frame image are the pixel points with strong ambient light interference.

[0046] Step 2. Except for the column where the pixel point where the line laser position does not exist is determined in step 1, the robot traverses the relevant columns of the current frame image in sequence, specifically traverses the pixel points in the column where the initial pixel position exists; in the current column of the current frame image, the robot sets the initial pixel position in the current column as the search center, and then searches for pixel points within a search radius upward along the current column from the search center. Optionally, the robot starts from the initial pixel position in the current column, sets the first preset pixel distance as the search radius, and searches for pixel points that meet the convex hull feature upward and downward along the column direction respectively; wherein the current column is a column currently traversed by the robot, and the search radius is also applicable to defining the coverage area in two opposite column directions of the same column; the two opposite column directions of the same column include the direction of searching upward along the same column and the direction of searching downward along the same column starting from the initial pixel position. Then, for a currently determined search center, the convex hull center pixel point in the current column is screened out according to the difference in the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward under the search states corresponding to the two adjacent search centers determined, and the inter-frame matching relationship formed by the same type of numerical values ​​in the same column of pixels of the current frame image relative to the reference frame image, and then the convex hull center pixel point in the current column that has been screened out is updated with the convex hull center pixel point determined in the current column of the current frame image last time; whenever the search center in the current column is updated once, the convex hull center pixel point set in the current column is also updated once, and each pixel point within the search radius relative to the initial pixel position is traversed in the current column and the convex hull center pixel point is updated to determine the final convex hull center pixel point of the current column. In some embodiments, due to multiple reflections of the line laser, multiple pixel points that meet the convex hull features will be generated in the same column of the current frame image, and two or more convex hull center pixel points will be updated at the same time, and finally two or more convex hull center pixel points are determined in the current column, and then the offsets of these convex hull center pixel points relative to the origin of the coordinate system of the current frame image are compared, and the convex hull center pixel point with the smallest absolute value of the longitudinal coordinate offset is selected to be updated as the convex hull center pixel point finally determined in the current column. If the longitudinal coordinate of the origin of the coordinate system of the current frame image represents the longitudinal coordinate of the robot's traveling plane, then the convex hull center pixel point finally determined in the current column is the convex hull center pixel point in the current column of the current frame image that is closest to the ground.

[0047] Wherein, the reference frame image is configured as a bright frame image where the line laser position found by the robot most recently before the current frame image is acquired; the line laser position found by the robot most recently is selected from the convex hull center pixel points of the corresponding column. The difference between the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward can be extended to the numerical relationship between the brightness value gradient generated in the pixel point searched upward and the brightness value gradient generated in the pixel point searched downward, which can be compared in the search state corresponding to the search center determined twice adjacently, which is conducive to selecting the convex hull center pixel point from the searched pixel points that meet the convex hull characteristics, and the brightness value of the currently determined search center is compared with the brightness value of the search center determined last time in the same column; the search centers determined twice adjacently are respectively the currently determined search center and the last determined search center in the same column, which can be the two adjacent search centers determined by searching the pixels within the search radius in two rounds in the upward direction of the current column from the initial pixel position, selecting and updating the convex hull center pixel points in the same column; or it can be the two adjacent search centers determined by searching the pixels within the search radius in two rounds in succession along the current column upward direction; Starting in the downward direction along the current column, two rounds of searching are performed successively to determine two adjacent search centers of the pixel points within the search radius, and the convex hull center pixel points of the same column are screened and updated, wherein one round of searching corresponds to one search center, and more corresponds to a search state in the pixel point area of ​​different columns, and the update range of the search center is within the coverage area of ​​a search radius relative to the initial pixel position, including the initial pixel position, so as to update the convex hull center pixel points of the same column, and continuously screen out more accurate convex hull centers to represent the line laser position; the inter-frame matching relationship includes matching in the number of pixels and brightness values ​​between the two frames of images, and the two frames of images can be two adjacent frames of images, or two bright frames of images separated by one or more frames of images, and the specific matching involved can be based on the change in brightness value and vertical coordinate of the pixel points at the same reflection position of the line laser on the surface of the obstacle in the image collected in real time during the walking process of the robot.

[0048] On this basis, whenever the robot traverses each pixel point within a search radius upward along the current column from the search center in accordance with step 2 and filters and updates the convex hull center pixel point, and also traverses each pixel point within a search radius downward along the current column from the search center and filters and updates the convex hull center pixel point, the robot starts to traverse the pixel points within the search radius from the initial pixel position of the next column of the current frame image.

[0049] In step 1, when the current frame image is a bright frame image and the previous frame image is a dark frame image, the robot saves the previous frame image. If the robot has searched for the corresponding line laser position (including the line laser position on the corresponding column or the line laser position on all columns) from the previous frame image before executing the current step 1, the previous frame image is marked as a reference frame image; preferably, the reference frame image is configured as a bright frame image where the line laser position found by the robot most recently before the current frame image is acquired, wherein the line laser position found by the robot most recently is a convex hull center pixel point from the corresponding column, specifically, in the corresponding column, with the initial pixel position as the center, within a search radius coverage area in the upward direction of the corresponding column and within a search radius coverage area in the downward direction of the corresponding column, all pixel points are updated to the search center in sequence, and then a new convex hull center pixel point is set by step 2.

[0050] Step 3. For the convex hull center pixel points in each column of pixel points that have been screened out by the robot, the robot first determines the convex hull center pixel points that are interference points based on the relationship between the brightness value of the line laser emitted by the line laser transmitter in the effective coverage area corresponding to the positioning coordinates of the previous dark frame image and the brightness value of the convex hull center pixel points in the current frame image, and then removes the interference points from the screened convex hull center pixel points. The robot may select a column-by-column traversal method to remove the interference points in the current frame image to eliminate the interference of ambient light. After the robot has traversed the convex hull center pixel points among all the columns of pixel points in the current frame image to eliminate all interference points, the coordinates of the remaining convex hull center pixel points are set as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, and the robot searches for the line laser position determined in each column in the current frame image to connect the laser line segments formed by the line laser emitted by the line laser transmitter on the surface of the object to be measured, and determines that the robot has searched for the line laser position from the current frame image by executing the inter-frame tracking algorithm; wherein, the line laser position determined in the same column is the position of the convex hull center pixel point last updated in the same column after the robot has traversed all the pixel points in the same column, and the coordinates of a line laser position are represented by the corresponding positioning coordinates. Since the robot takes a column-by-column traversal method to obtain the line laser position belonging to the corresponding column for the current frame image during the execution of steps 1 to 3, after determining the column number currently traversed, the coordinates of each line laser position can be represented by only the ordinate value to identify the height information of the reflection position of the line laser on the obstacle surface, which can also be used for robot obstacle avoidance.

[0051] Specifically, in step 2, except for the column where the pixel point where the line laser position does not exist, the robot sets the initial pixel position obtained in step 1 in the current column as the search center, that is, the initial pixel position in step 2 in the aforementioned embodiment. Whenever a convex hull center pixel point is screened out for a search center, an adjacent pixel point searched upward or downward along the current column from the search center is updated as the search center, and step 2 is re-executed to obtain a new convex hull center pixel point and update the new convex hull center pixel point as the convex hull center pixel point; wherein each of the search centers is within a coverage area of ​​a search radius relative to the initial pixel position, and the search radius is set to a first preset pixel distance, preferably, the first preset pixel distance is less than the maximum pixel distance covered by the current frame image, and is within the detection range of the camera, so as to limit the search for the convex hull center pixel point to the vicinity of the initial pixel position. In this embodiment, the convex hull center pixel point is a pixel point that meets the convex hull feature within the coverage range of the search radius. Preferably, the convex hull feature here is a feature used to represent a figure formed by a line laser hitting the surface of an obstacle, and the pixel point feature of the coverage area of ​​the figure can be the brightness value of the pixel points within the coverage area of ​​the figure itself, or the brightness value of the pixel points within the coverage area of ​​the inscribed circle of the figure, or the brightness value of the pixel points within the coverage area of ​​the circumscribed circle of the figure. It is worth noting that the filtered convex hull center pixel point is the last updated convex hull center pixel point in each column of convex hull center pixel points existing (able to be searched) in the current frame image; the filtered convex hull center pixel point is the convex hull center pixel point that is closest to the origin of the coordinate system of the current frame image in each column of convex hull center pixel points existing in the current frame image. Preferably, the convex hull center pixel point is not necessarily updated in each column in the current frame image, so the finally connected laser line segment is not continuous, which can be used to represent the raised obstacles on the ground where the robot is moving.

[0052] In this embodiment, the robot sets the set of pixel points that meet the convex hull feature on the current column of the current frame image to the pixel points whose brightness values ​​decrease from the center of the convex hull along the current column to the upper and lower sides respectively, and the set of pixel points composed of the convex hull center to form a convex hull, which can be regarded as forming a figure surrounding a laser line segment, which is used to detect the local effective detection area on the obstacle surface where the reflection position of the line laser is located. The convex hull center is the pixel point with the largest brightness value in the pixel point set, and the convex hull center pixel point is set as the pixel point belonging to the convex hull center; in the set of pixel points that meet the convex hull feature, starting from the center of the convex hull in the upward direction along the same column, the brightness value of the pixel point decreases upward along the current column and generates a first gradient value between the brightness values ​​of two adjacent pixel points, and starting from the center of the convex hull in the downward direction along the same column, the brightness value of the pixel point decreases downward along the current column and generates a second gradient value between the brightness values ​​of two adjacent pixel points, so that the convex hull center belongs to the search center. Among them, in the neighborhood of the convex hull center, there is at least one pixel whose brightness value is equal to the brightness value of the search center. Therefore, in the set of pixel points that meet the convex hull feature, starting from the convex hull center, the brightness value decreases upward along the current column to generate a first gradient value, and the brightness value decreases downward along the current column to generate a second gradient value. Among them, the pixel distance required to traverse upward from the convex hull center can be less than or equal to the search radius, and the pixel distance required to traverse downward from the convex hull center can also be less than or equal to the search radius, so as to form a predetermined brightness value gradient change law in the convex hull. If the change law of the brightness values ​​of multiple pixel points traversed in the same convex hull does not conform to the predetermined brightness value gradient change law.

[0053] Preferably, the brightness value of the search center is a value of 255, that is, the brightness value of the image is the maximum gray value (maximum gray level) divided by the binarization method. It should be noted that the brightness of the pixel is used to indicate the brightness of the light irradiated on the surface of the object to be measured. When the gray value is used to indicate its brightness value, if the gray value is higher, the image is brighter, and the brightness value is larger. The grayscale image formed by the binarization of the image contains only brightness information, not color information, just like a black and white picture, the brightness changes from dark to bright, and the change is continuous. Therefore, to represent the grayscale image, it is necessary to quantize the brightness value, which is usually divided into 256 levels from 0 to 255, wherein the value 255 is used in this embodiment to represent a brightness value. When the range of the gray value is from 0 to 255, this embodiment also represents the value range of the brightness value of the pixel as a value of 0 to 255, then each frame of the image captured by the camera can be regarded as converted into a grayscale image, wherein the brightness is the grayscale, and the larger the grayscale value, the larger the brightness value, wherein the value 0 can represent the blackest pixel, and the value 255 can represent the whitest pixel. The pixel mentioned in this embodiment is an indivisible unit in a frame of image. Each frame of image is composed of many pixels. It exists as a small grid of a single color and can be mapped to a cell (grid) in a grid map. The grayscale image uses a capacity of one byte to store a pixel.

[0054] As an embodiment, in step 2, for each column of each frame image with an initial pixel position, excluding the column where the pixel point where the line laser position does not exist, the method of screening out the convex hull center pixel point according to the difference between the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward in the search state corresponding to the search center determined twice adjacently, and the inter-frame matching relationship formed by the same type of values ​​of the current frame image in the same column of the pixel point relative to the reference frame image includes:

[0055] In the current column of the current frame image, the robot starts from the search center and searches for the pixel points that meet the convex hull feature upward or downward along the column direction. The brightness value of the search center is controlled to be compared with the brightness value of the convex hull center pixel point located in the same column searched last time. The difference in brightness values ​​can be used but is not limited to determine the size relationship between the two; the convex hull center pixel point located in the same column searched last time is the convex hull center pixel point screened out in the same column of the current frame image for the search center determined last time, and the search center determined last time is a pixel point adjacent to the currently determined search center downward or upward in the current column of the current frame image. Furthermore, the column ordering of the pixels in the same column of the reference frame image is equal to the column ordering of the current column of the current frame image, and the convex hull center pixels in a column with the same ordering of the reference frame image do not necessarily have the same row ordering.

[0056] If the robot detects that the brightness value of the search center is greater than the brightness value of the pixel point at the center of the convex hull located in the same column searched last time, then in the current column, the pixel points are searched upward from the search center, and the number of pixel points whose brightness values ​​decrease according to the first gradient value is counted, that is, whenever a pixel point with a decreasing brightness value is searched upward, and the brightness value of the currently searched pixel point is reduced by one of the first gradient values ​​relative to the brightness value of the pixel point searched last time (the brightness value of a pixel point below the currently searched pixel point), the number of pixel points whose brightness values ​​decrease according to the first gradient value is counted by one. This can be understood as the robot searching upward along the current column of the current frame image for pixel points that meet the convex hull features until the upward counting stop condition is met.

[0057] Preferably, the first gradient value changes adaptively with the change of the number of searches. For example, the closer the currently searched pixel is to the upper edge of the convex hull, the larger the first gradient value becomes. In the same convex hull, the closer the pixel is to the upper edge of the convex hull, the more drastic the decrease in brightness value, satisfying a predetermined brightness value gradient change rule. When the upward counting stop condition is met, the counting of the number of pixel points whose brightness values ​​decrease according to the first gradient value is stopped. At this time, the robot also stops searching for pixel points along the column direction, and then marks the number of pixel points whose brightness values ​​decrease according to the first gradient value as the upward gradient descent number.

[0058] The robot also searches for pixels downward from the search center, and counts the number of pixels whose brightness values ​​decrease according to the second gradient value, that is, whenever a pixel whose brightness value decreases is searched downward, and the brightness value of the currently searched pixel is reduced by one second gradient value relative to the brightness value of the pixel searched last (the brightness value of a pixel above the currently searched pixel), the number of pixels whose brightness values ​​decrease according to the second gradient value is counted plus one. This can be understood as the robot searching downward along the current column of the current frame image for pixels that meet the convex hull feature until the downward counting stop condition is met.

[0059] Preferably, the second gradient value changes adaptively with the change of the number of searches. For example, the closer the currently searched pixel is to the lower edge of the convex hull, the larger the second gradient value becomes. In the same convex hull, the closer the pixel is to the lower edge of the convex hull, the more drastic the decrease in brightness value, satisfying a predetermined brightness value gradient change rule. When the downward counting stop condition is met, stop counting the number of pixel points whose brightness values ​​decrease according to the second gradient value, and then mark the number of pixel points whose brightness values ​​decrease according to the second gradient value as the downward gradient descent number.

[0060] After determining to stop searching and counting upward from the search center along the current column of the current frame image, and determining to stop searching and counting downward from the search center along the current column of the current frame image, when the robot determines that the number of upward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents counted in the same column of the reference frame image (the column order of a column involved in comparison in the reference frame image is equal to the column order of the current column of the current frame image, which can also be counted as the current column of the reference frame image), and / or determines that the number of downward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents counted in the same column of the reference frame image When the number is greater than or equal to the number of downward gradient descents counted in the same column of the reference frame image (the column order of a column involved in the comparison in the reference frame image is equal to the column order of the current column of the current frame image, which can also be counted as the current column of the reference frame image), it indicates that the robot is approaching the obstacle to be detected, and in this process, the number of pixel points that meet the convex hull features used to characterize the same local area of ​​the obstacle has increased relative to before the installation height increases. Then, the number of pixel points that meet the convex hull features that can be searched in the current frame image increases, and the robot can detect more details of the obstacle. Although there are The risk of collision is eliminated, but after the current frame image is switched to a dark frame image or after the next frame image (dark frame image) is acquired, it will switch to executing the brightness centroid algorithm to avoid obstacles; on this basis, among the pixels traversed by the current column of the current frame image, if it is detected that the first gradient value and the second gradient value are not equal to the first preset gradient parameter, and the absolute value of the difference between the first gradient value and the second gradient value is less than the second preset gradient parameter, and the absolute value of the difference between the brightness value of the pixel point with the smallest brightness value searched upward along the current column and the brightness value of the pixel point at the currently determined search center is greater than the brightness value of the same column of the reference frame image If the absolute value of the difference in brightness values ​​of the same type formed by searching upward among the pixels in the pixel points, and the absolute value of the difference between the brightness value of the pixel point with the smallest brightness value searched downward along the current column and the brightness value of the pixel point at the currently determined search center is greater than the absolute value of the difference in brightness values ​​of the same type formed by searching downward among the pixels in the same column of the reference frame image, the robot marks the currently determined search center as the convex hull center pixel point, and determines that the matching relationship formed by the same type of values ​​of the current frame image relative to the reference frame image in the same column of pixels meets the expected change in the position of the pixel points in the convex hull during the robot's walking process.

[0061] Specifically, the absolute value of the difference of the same type of brightness values ​​formed by the upward search in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched upward in the same column with the same column order as the current column in the reference frame image and the brightness value of the search center finally determined on the same column, wherein the distance between the pixel with the smallest brightness value searched upward and the search center finally determined on the same column is less than or equal to the search radius. And, the absolute value of the difference of the same type of brightness values ​​formed by the downward search in the same column of the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downward in the same column with the same column order as the current column in the reference frame image and the brightness value of the search center finally determined on the same column, wherein the distance between the pixel with the smallest brightness value searched downward and the search center finally determined on the same column is less than or equal to the search radius.

[0062] Since the absolute value of the difference between the pixels with the smallest brightness values ​​in the two opposite directions is increasing, the pixel offset of the pixel used to characterize the same reflection position of the line laser relative to the center of the image in the same frame image (which can also be understood as the coordinate offset relative to the origin of the coordinate system in the image coordinate system) increases, and it is further determined that the robot is approaching the obstacle to be measured, and in this process, the number of pixels used to characterize the same local area of ​​the obstacle increases relative to before the installation height increases, then the number of pixels that can be searched in the current frame image increases, and the robot can detect more details of the obstacle, which proves that the convex hull center pixel point among the pixels searched in the current column of the current frame image is a point that relatively accurately represents the laser line segment of the line laser hitting the surface of the obstacle to be measured, until the convex hull center pixel point of all columns of the same frame image is traversed and updated, the line laser position in each column is obtained and connected or fitted as a laser line segment representing the line laser, so as to realize the positioning of the obstacle where the laser line segment is located, so as to facilitate the robot to avoid obstacles in time.

[0063] It should be noted that, in the process of executing step 2, the robot first starts from the search center and searches for pixels in sequence along a column of the current frame image upwards until all pixels within a search radius are searched along a column of the current frame image upwards, and then starts from the same search center and searches for pixels in sequence along a column of the current frame image downwards until all pixels within a search radius are searched along a column of the current frame image downwards. Alternatively, the robot first starts from the search center and searches for pixels in sequence along a column of the current frame image downwards until all pixels within a search radius are searched along a column of the current frame image downwards, and then starts from the same search center and searches for pixels in sequence along a column of the current frame image downwards until all pixels within a search radius are searched along a column of the current frame image upwards.

[0064] Preferably, the reference frame is the first frame image of the light reflected by the line laser on the surface of the object to be measured captured by the camera during the execution of the laser positioning method, when the convex hull center pixel point in the same column of pixels of the reference frame image is located at the initial pixel position in the same column of the reference frame image, the initial pixel position in the same column of the reference frame image is the line laser position in the same column of the reference frame image, which best represents a point of the laser line segment of the line laser hitting the surface of the object to be measured. Wherein, in the process of executing the laser positioning method, the first frame image of the light reflected by the line laser on the surface of the object to be measured captured by the camera is a bright frame image, which is recorded as the first frame image after the line laser transmitter emits the line laser, and can also be recorded as the first bright frame image.

[0065] It should be noted that the first preset gradient parameter is smaller than the second preset gradient parameter; the first preset gradient parameter is preferably a value of 0 to avoid selecting pixels that meet the convex hull characteristics in a pixel area with a constant brightness value (such as a local overexposed area, although the pixels inside it may be the center of the convex hull); the second preset gradient parameter is preferably a value of 25 to control the coordinate jump of the pixel points at the same reflection position used to characterize the obstacle within a controllable range, avoid introducing pixels with drastic changes in brightness values, and only focus on the effective detection area of ​​the obstacle to be detected; it can reduce the search amount and calculation amount, and also improve the detection accuracy.

[0066] For a currently determined search center, step 2 corresponding to the above embodiment also includes a stop condition for searching for pixels along the column direction (the upward counting stop condition and the downward counting stop condition), specifically including:

[0067] If the brightness value of the pixel at the search center is greater than the brightness value of the convex hull center pixel located in the same column searched last time, it indicates that the brightness value of the pixel at the search center (initially the initial pixel position) of the current column of the current frame image is not equal to the brightness value of the reasonable convex hull center searched last time, and as the robot approaches the obstacle, the difference in brightness values ​​between the two increases, wherein the convex hull center pixel located in the same column searched last time is the convex hull center pixel selected in the same column of the current frame image for the search center determined last time, and the search center determined last time is a pixel point adjacent to the currently determined search center in the current column of the current frame image downward or upward. The robot starts searching for pixels upward from the search center in the current column of the current frame image, with the goal of searching for pixels upward from the search center in order to filter out the convex hull center pixels in the current column by counting the pixels that meet the convex hull features; and searches for pixels downward from the search center in order to filter out the convex hull center pixels in the current column by counting the pixels that meet the convex hull features. Specifically, in the current column of the current frame image, the number of pixel points whose brightness values ​​decrease according to the first gradient value is counted upward from the search center. Preferably, starting from the search center, each time a pixel point that meets the convex hull feature is searched upward along the current column, the count is increased by one to obtain the number of pixel points whose brightness values ​​decrease according to the first gradient value; and in the current column of the current frame image, the number of pixel points whose brightness values ​​decrease according to the second gradient value is counted downward from the search center to realize the search for pixel points that meet the convex hull feature upward and downward along the column direction starting from the corresponding initial pixel position. Preferably, starting from the search center, each time a pixel point that meets the convex hull feature is searched downward along the current column, the count is increased by one to obtain the number of pixel points whose brightness values ​​decrease according to the second gradient value.

[0068] In some embodiments, if the robot detects that the brightness value of the pixel point is not decreasing according to the first gradient value during the upward search from the search center, the preset upward gradient anomaly count number is counted once, and then the robot determines whether it has searched all the pixels covered within the search radius upward along the current column of the current frame image. If so, the robot stops searching for pixels upward along the current column of the current frame image and determines that the upward counting stop condition is met, and then performs step 2 to filter out the convex hull center pixel point, and then starts to update an adjacent pixel point searched upward from the search center along the current column as the search center, and then repeats step 2 to update the convex hull center pixel point; otherwise, when the upward gradient anomaly frequency is greater than the first preset error number, stop searching for pixels upward along the current column of the current frame image and determine that the upward counting stop condition is met, and then continue to perform step 2 to filter out the convex hull center pixel point, and then starts to update an adjacent pixel point searched upward from the search center along the current column as the search center, and then repeats step 2 to update the convex hull center pixel point; until all the pixels covered within the search radius are searched upward relative to the initial pixel position.

[0069] In some embodiments, during the downward search from the search center, if it is detected that the brightness value of the pixel point is not decreasing according to the second gradient value, the preset downward gradient abnormality count number is counted once, and then the robot determines whether it has searched the pixel points covered within the search radius downward along the current column of the current frame image. If so, the robot stops searching the pixel points downward along the current column of the current frame image and determines that the downward counting stop condition is reached, and then according to the difference between the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward under the search states corresponding to the search centers determined twice adjacently, and the inter-frame matching relationship formed by the same type of values ​​in the same column of the current frame image relative to the reference frame image, the convex hull center pixel point is screened out, and then the adjacent pixel point searched downward from the search center along the current column is started to be more accurately searched. The new search center is set as the search center, and step 2 is repeated to update the central pixel point of the convex hull; otherwise, when the frequency of the upward gradient anomaly is greater than the second preset error number, stop searching for pixels downward along the current column of the current frame image and determine that the downward counting stop condition is satisfied, and then according to the difference in the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward under the search states corresponding to the search centers determined twice adjacently, and the inter-frame matching relationship formed by the same type of values ​​of the pixel points in the same column of the current frame image relative to the reference frame image, the central pixel point of the convex hull is screened out, and then an adjacent pixel point searched upward along the current column starting from the search center is updated as the search center, and step 2 is repeated to update the central pixel point of the convex hull until all the pixels covered within the search radius are searched upward relative to the initial pixel position.

[0070] For the first preset error number and the second preset error number, it is necessary to add that if the pixel points searched within the search radius do not meet the convex hull feature within a certain error tolerance range, after all, the search center is not necessarily the convex hull center, then the preset error number needs to be set for judgment, wherein the source of the error is that the reflection position formed by the line laser of the same emission angle collected during the robot's walking process on the same surface of the object to be measured will jump longitudinally, which is reflected in the pixel points used to represent the same reflection position in different frame images. The upward displacement along the ordinate axis. Therefore, if the robot detects that the brightness value of the pixel point is not decreasing according to the first gradient value in the process of counting upward from the search center, and / or detects that the brightness value of the pixel point is not decreasing according to the second gradient value in the process of counting downward from the search center, it is determined that the gradient value between two adjacent pixel points searched along one of the column directions is abnormal, and the preset gradient abnormality counting number is counted once; when the robot detects that the gradient abnormality frequency is greater than the preset error number, and / or the pixel points covered within the search radius are counted, the robot stops counting and stops searching for pixel points that meet the convex hull feature.

[0071] In some embodiments, during the process of searching upward from the search center, the robot counts the pixel points whose brightness value is 255 and whose positions are adjacent to each other along the current column of the current frame image upward, and marks the number of pixel points whose brightness value is 255 and whose positions are adjacent to each other as the number of upward overexposures, thereby forming a count of the number of overexposed areas where the pixel points at the search center in the upward direction have continuous brightness values ​​of 255. Then the robot determines whether it has searched all the pixels covered within the search radius upward along the current column of the current frame image. If so, the robot stops searching all the pixels covered within the search radius upward along the current column of the current frame image and determines that the upward counting stop condition is met, and then performs step 2 to filter out the convex hull center pixel, and then starts to update an adjacent pixel searched upward along the current column from the search center as the search center, and then repeats step 2 to update the convex hull center pixel; otherwise, when the robot detects that the number of upward overexposures is greater than the third preset error number, it stops searching all the pixels covered within the search radius upward along the current column of the current frame image and determines that the upward counting stop condition is met, and then continues to perform step 2 to filter out the convex hull center pixel, and then starts to update an adjacent pixel searched upward along the current column from the search center as the search center, and then repeats step 2 to update the convex hull center pixel; until all the pixels covered within the search radius are searched upward relative to the initial pixel position. Implementation: When the robot detects that the amount of upward overexposure is greater than the third preset error number, and / or the pixel points covered within the search radius are counted upward along the current column of the current frame image, the robot stops searching for pixel points upward along the current column of the current frame image and determines that the upward counting stop condition is met.

[0072] In some embodiments, while searching downward from the search center, the robot counts the pixel points whose brightness value is 255 and whose positions are adjacent to each other along the current column of the current frame image downward, and marks the number of pixel points whose brightness value is 255 and whose positions are adjacent to each other as the downward overexposure number, thereby forming a count of the overexposed area where the pixel points at the search center in the downward direction have a continuous brightness value of 255. Then the robot determines whether it has searched all the pixels covered within the search radius downward along the current column of the current frame image. If so, the robot stops searching all the pixels covered within the search radius downward along the current column of the current frame image and determines whether the downward counting stop condition is met, and then performs step 2 to filter out the convex hull center pixel, and then starts to update an adjacent pixel searched downward along the current column from the search center as the search center, and then repeats step 2 to update the convex hull center pixel; otherwise, when the robot detects that the number of downward overexposures is greater than a fourth preset error number, it stops searching all the pixels covered within the search radius downward along the current column of the current frame image and determines whether the downward counting stop condition is met, and then continues to perform step 2 to filter out the convex hull center pixel, and then starts to update an adjacent pixel searched downward along the current column from the search center as the search center, and then repeats step 2 to update the convex hull center pixel; until all the pixels covered within the search radius are searched downward relative to the initial pixel position. Implementation: When the robot detects that the amount of upward overexposure is greater than the fourth preset error number, and / or the pixel points covered within the search radius are counted down along the current column of the current frame image, the robot stops searching for pixel points downward along the current column of the current frame image and determines that the downward counting stop condition is met. Implementation: When the robot detects that the amount of upward overexposure is greater than the fourth preset error number, and / or the pixel points covered within the search radius are counted down along the current column of the current frame image, the robot stops searching for pixel points downward along the current column of the current frame image and determines that the downward counting stop condition is met.

[0073] In summary, in each round of searching for the convex hull center pixel near the search center, the robot determines the stop search condition by comparing the brightness of a pair of pixels searched in opposite directions starting from the search center and counting the overexposure and gradient anomalies.

[0074] As an embodiment, in the step 3, the method for removing interference points from the screened convex hull center pixel points according to the size relationship between the brightness value in the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser transmitter in the previous dark frame image and the brightness value of the convex hull center pixel point in the current frame image includes: after the robot traverses all columns of pixel points in the current frame image and obtains the convex hull center pixel point from the current frame image, and also saves the positioning coordinates of the line laser emitted by the line laser transmitter in the previous dark frame image, for each convex hull center pixel point in the current frame image, The positioning coordinates of the line laser emitted by the line laser transmitter in the previous frame of dark image are located at a position as the center of the circle, and the radius is the detection pixel distance in a circular domain. If the robot determines that there is at least one pixel point in the circular domain whose brightness value is greater than the brightness value of the convex hull center pixel point with the same coordinates as the center of the circle in the current frame image by a preset ambient light brightness threshold, then the robot determines that the convex hull center pixel point with the same coordinates as the center of the circle in the current frame image is an interference point, and there is ambient light interference in the vicinity of the convex hull center pixel point with the same coordinates as the center of the circle in the current frame image, causing the robot to be unable to find the line laser position at the interference point, then it is necessary The interference point is removed from the current frame image to overcome the interference of ambient light and reduce positioning misjudgment; wherein the circular area is the effective coverage area corresponding to the positioning coordinates, and preferably, the radius of the circular area (detection pixel distance) is not equal to the search radius; in the process of executing the inter-frame tracking algorithm, the current frame image is a bright frame image, and the previous frame image is a dark frame image, that is, the previous dark frame image. At this time, the line laser position in the previous dark frame image has been output by the brightness centroid algorithm, and the positioning coordinates in the previous dark frame image (the coordinates of the line laser position determined in one column) are used; in this embodiment, the robot selects the position in the previous dark frame image. The coordinates of the pixel point at the center of the circle are equal to the coordinates of a convex hull center pixel point obtained in the current frame image, and the brightness value of the convex hull center pixel point with the same coordinates as the center of the circle in the current frame image can be used for comparison, wherein the preset ambient light brightness threshold is specifically associated with the robot's travel speed or rotation speed. Preferably, the greater the robot's travel speed or rotation speed, the more drastic the position jump of the pixel points used to represent the same reflection position in the image collected by the robot in real time, and the greater the gradient difference between the brightness values ​​of the two pixel points, and the preset ambient light brightness threshold is set to be larger to adapt to the denoising accuracy.

[0075] As an embodiment, in the step 1, the method of excluding pixel points where there is no line laser position in the current frame image based on pixel points in the corresponding column that meet the preset brightness distribution characteristics includes: if the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel point at the line laser position in the same column found in the previous round by a first preset brightness threshold, or the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel point at the line laser position in the same column found in the previous round by a second preset brightness threshold, then starting from a position one reference pixel distance away from the initial pixel position in the current column of the current frame image upwards, searching for pixel points along the current column of the current frame image downwards; wherein the first preset brightness threshold is less than The second preset brightness threshold, the first preset brightness threshold is preferably a value of 10, and the second preset brightness threshold is preferably a value of 235. When the brightness value of the initial pixel point in the current column of the current frame image changes little relative to the brightness value generated by the line laser position located in the same column found in the previous round, or the brightness value generated by the line laser position located in the same column found in the previous round is large enough to be close to the value 255 (the highest grayscale value), the current column may be affected by ambient light, and a reference position is required to start searching for pixels along the current column of the current frame image to exclude pixels with abnormal brightness values ​​or the column where they are located, and the sum of the first preset brightness threshold and the second preset brightness threshold is less than the value 255 (the highest grayscale value), then the first preset brightness threshold and the second preset brightness threshold are used as the brightness value judgment conditions for the columns to be excluded in the rough screening.

[0076] Then, in the process of searching for pixels, if it is detected that the brightness value of a pixel currently being searched is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round by a first preset brightness threshold, or it is detected that the brightness value of a pixel currently being searched is equal to the value 255 (the highest grayscale value), the number of error position counts is counted once, and it is determined that the pixel currently being searched is a pixel that meets the preset brightness distribution characteristics, and the vicinity of the initial pixel in the current column of the current frame image may have a brightness value greater than the brightness value of the pixel at the line laser position in the same column found in the previous round by the first preset brightness threshold, or may have a brightness value equal to 255 (the highest grayscale value), which is easily affected by ambient light; wherein the reference pixel distance is represented by the number of pixels so that the reference pixel count threshold is equal to the reference pixel distance.

[0077] When the robot detects that the error position count amount is greater than the reference pixel count threshold, it is determined that there is no line laser position in the current column of the current frame image, and the pixel points in the current column of the current frame image are set as pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel point search range of step 2, and at the same time, it is determined that the light intensity of the environment where the robot is located is greater than the first preset light intensity threshold, indicating that there is strong ambient light interference in the area from the position of the reference pixel distance upward from the initial pixel position in the current column of the current frame image to the position where the pixel point at the bottom of the current column of the current frame image is located; the reference pixel count threshold is preferably a value of 25, and is set to be equal to the reference pixel distance, then when the error position count amount is greater than the position offset of the search starting point of the pixel point that meets the preset brightness distribution characteristics in this embodiment relative to the initial pixel position of the same column, it is determined that the line laser position cannot be searched in the current column of the current frame image, and there is ambient light interference. In this way, the reference test area set in the same column is traversed row by row to compare the pixel points that meet the brightness requirements and record the number of times to determine the strong ambient light.

[0078] Preferably, the reference pixel distance is equal to the pixel distance composed of 25 pixels. Then, starting from a position 25 pixels upward from the initial pixel position in the current column of the current frame image, pixels are searched downward along the current column of the current frame image until the current column of the current frame image is traversed to the bottom, and a reference test area is formed in the current column, wherein the area formed by starting from a position one reference pixel distance upward from the initial pixel position in the current column of the current frame image and extending downward along the current column of the current frame image to the bottom of the current column of the current frame image is the reference test area; in the process of traversing each pixel of the reference test area, whenever it is detected that the brightness value of a pixel currently traversed is 10 greater than the brightness value of the pixel at the line laser position found in the previous round and located in the same column, or it is detected that the brightness value of a pixel currently traversed is equal to 255 (the highest grayscale value), it is counted once, and it is determined that the pixel currently searched is a pixel that meets the preset brightness distribution characteristics, until the number of pixel points that meet the preset brightness distribution characteristics is greater than 25. Among them, the sum of the reference pixel distance (or reference pixel count threshold) and the second preset brightness threshold is greater than the value 255 (the highest grayscale value), and the sum of the first preset brightness threshold and the second preset brightness threshold is less than the value 255. The changes in the brightness values ​​generated by the initial pixel position in the current column of the current frame image relative to the line laser position found in the same column in the previous round and the changes in the brightness values ​​of the pixels searched in the current column of the current frame image are compared, reflecting the ambient light intensity of the area corresponding to the current column.

[0079] It should be noted that the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image, that is, the line laser position is determined in the same column of pixels of the reference frame image (the position of the convex hull center pixel point set after removing the interference points in the aforementioned embodiment), and the same column of pixels of the reference frame image is a column of pixels in the reference frame image with the same column order as the current column of the current frame image; each time the same column of line laser positions is set in a bright frame image, it is recorded as a round of finding the line laser position located in the same column of the corresponding image, and each round of corresponding search images are different frame images.

[0080] As an embodiment, in step 1, the method of excluding pixels at which no line laser position exists in the current frame image according to the pixels in the corresponding column that meet the preset brightness distribution characteristics includes:

[0081] Taking the initial pixel position in the current column of the current frame image as the center of the ring, in the current column of the current frame image, the pixel points covered by the ring area with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius located below the center of the ring are marked as the first pixel points to be tested; which is equivalent to: taking the initial pixel position in the current column of the current frame image as the center of the circle, setting a first circle with a first radius of the first positioning radius; at the same time, taking the initial pixel position in the current column of the current frame image as the center of the circle, setting a second circle with a second radius of the second positioning radius, wherein the first positioning radius is smaller than the second positioning radius; and then below the initial pixel position (in the downward direction of the current column), marking the pixel points covered by the ring area enclosed by the second circle and the first circle in the current column of the current frame image as the first pixel points to be tested.

[0082] Then, the average value of the brightness value of the first pixel to be tested is calculated, that is, the ratio of the sum of the brightness values ​​of all the first pixel to be tested in the current column of the current frame image to the total number of the first pixel to be tested in the current column of the current frame image is obtained, as the average value of the brightness value of the first pixel to be tested; wherein, the circular ring coverage area formed between the first circle and the second circle serves as a transition area for judging the change in light intensity, which depends on the settings of the first positioning radius and the second positioning radius. The first positioning radius is preferably 3, and the second positioning radius is preferably 12. It can be expressed by pixel distance, and its unit is the number of pixels, to form a sufficiently large transition area for judging the change in light intensity.

[0083] If the average brightness value of the first pixel to be tested is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round, it is determined that the first pixel to be tested is a pixel that meets the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image, and there is strong ambient light interference in the reflection area corresponding to the current column of the current frame image, then the pixel points in the current column of the current frame image are set to pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel search range in step 2, and at the same time, it is determined that the light intensity of the environment in which the robot is located is greater than the first preset light intensity threshold; wherein, the line laser position in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image, preferably the initial pixel position in the same column of pixels of the reference frame image, and the same column of pixels of the reference frame image is a column with the same column order as the current column of the current frame image.

[0084] Similarly, taking the initial pixel position in the current column of the current frame image as the center of the ring, in the current column of the current frame image, the pixel points covered by the ring area above the center of the ring with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius are marked as second pixel points to be tested; which is equivalent to: taking the initial pixel position in the current column of the current frame image as the center of the circle, setting a first circle with a first radius as the first positioning radius; at the same time, taking the initial pixel position in the current column of the current frame image as the center of the circle, setting a second circle with a second radius as the second positioning radius, wherein the first positioning radius is smaller than the second positioning radius; then above the initial pixel position (in the upward direction of the current column), the pixel points covered by the ring area enclosed by the second circle and the first circle in the current column of the current frame image are marked as second pixel points to be tested, which are different from the first pixel points to be tested.

[0085] Then, the average value of the brightness value of the second pixel to be tested is calculated, that is, the ratio of the sum of the brightness values ​​of all the second pixel to be tested in the current column of the current frame image to the total number of the second pixel to be tested in the current column of the current frame image is obtained, as the average value of the brightness value of the second pixel to be tested; wherein, the circular ring coverage area formed between the first circle and the second circle serves as a transition area for judging the change in light intensity, which depends on the settings of the first positioning radius and the second positioning radius. The first positioning radius is preferably 3, and the second positioning radius is preferably 12. It can be expressed by pixel distance, and its unit is the number of pixels, to form a sufficiently large transition area for judging the change in light intensity.

[0086] If the average brightness value of the second pixel to be tested is greater than the brightness value of the pixel at the line laser position in the same column found in the previous round, it is determined that the second pixel to be tested is a pixel that meets the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image, then the pixel points in the current column of the current frame image are set to pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel search range of step 2, and at the same time, it is determined that the light intensity of the environment in which the robot is located is greater than the first preset light intensity threshold; wherein, the line laser position in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image, preferably the initial pixel position in the same column of pixels of the reference frame image, and the same column of pixels of the reference frame image is a column with the same column order as the current column of the current frame image.

[0087] In some embodiments, in step 1, if the initial pixel position cannot be obtained in the current column of the current frame image, the line laser position located in the same column found in the previous round is updated to the initial pixel position, and the second preset pixel distance is updated to the search radius, and step 2 is repeated. Specifically, the line laser position located in the same column found in the previous round is updated to the search center in the current column of the current frame image, and the search radius is set to the second preset pixel distance, the second preset pixel distance is not equal to the first preset pixel distance, and then starting from the most recently set search center, a search is performed upward along the current column for a pixel point within a search radius, and starting from the search center, a search is performed downward along the current column for a pixel point within a search radius, and each pixel point within the search radius is traversed until the convex hull center pixel point in the corresponding column is searched; specifically, the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward are compared according to the brightness values ​​of the search centers determined twice adjacently. The difference in the search state, and the inter-frame matching relationship formed by the same type of numerical values ​​in the same column of pixels of the current frame image relative to the reference frame image, after the convex hull center pixel point in the current column is screened out, an adjacent pixel point searched upward or downward from the search center along the current column is updated as the search center, and step 2 is re-executed to obtain a new convex hull center pixel point and update the new convex hull center pixel point as the convex hull center pixel point, wherein each of the search centers is within the coverage area of ​​a search radius relative to the initial pixel position, wherein the search radius is set to a second preset pixel distance, preferably, the second preset pixel distance is not equal to the first preset pixel distance; then whenever the search center in the current column is updated once, the convex hull center pixel point set in the current column is also updated once; wherein the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image. In addition, if the robot cannot find the central pixel point of the convex hull in the same column (the current column of the current frame image) while repeatedly executing step 2, it is determined that the robot cannot find the position of the line laser in the same column, and then the pixel points in the current column of the current frame image are excluded from the pixel search range of step 2. At the same time, it is determined that the light intensity of the environment in the current column of the current frame image is so large that the reflection position of the line laser in the object to be measured cannot be identified.

[0088] In summary, when the robot detects that the current frame image captured by the camera is a bright frame image, it chooses to input the current frame image into the processing rule model corresponding to the inter-frame tracking algorithm to output a valid laser position, and effectively filters out various ambient light interferences in scenes where the camera is not too close to obstacles, reducing dependence on infrared filters; specifically, in the pixels that meet the convex hull features, based on the numerical relationship between the brightness value gradient generated in the upward searched pixel points and the brightness value gradient generated in the downward searched pixel points, and the difference between the search states corresponding to the search centers determined twice adjacently, the relationship between the brightness value of the currently searched pixel point and the brightness value of the convex hull center pixel point determined in the same column of the same frame image last time, and the inter-frame formed by the same type of numerical values ​​in the same column of the current frame image relative to the reference frame image. Matching relationship, filter out the convex hull center pixel of the current column, and traverse each pixel within the search radius relative to the initial pixel position in the current column and update the convex hull center pixel, and exclude the interference of pixels where there is no line laser position, and determine the final convex hull pixel of the current column. It can reduce the misjudgment of interference points under strong ambient light conditions, thereby searching for a more accurate convex hull center pixel by tracking the number and brightness values ​​of pixels that meet the convex hull features of the reference frame image, and then after eliminating all interference points, set the coordinates of the remaining convex hull center pixel points as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, so as to more accurately realize the robot's tracking of the reflected light of the laser on the obstacle surface, which is suitable for robot navigation walking scenes to achieve the effect of robot locating obstacles.

[0089] As an embodiment, in order to find the position of the line laser in the dark frame image to overcome the scenes that the inter-frame tracking algorithm cannot cope with and ensure the obstacle avoidance effect, it is necessary to switch to executing the brightness centroid algorithm after the dark frame image is collected, and allow the previous frame image (belonging to the bright frame image, corresponding to the bright frame image of the line laser emitted by the line laser transmitter reflected back on the surface of the object to be measured) for use by the brightness centroid algorithm, so as to achieve the effect of complementary advantages with the inter-frame tracking algorithm, which can not only overcome the interference of ambient light, but also continuously track the reflection position of the line laser.

[0090] Specifically, the method in which the robot extracts the line laser position from the current frame image by executing the brightness centroid algorithm includes: the robot traverses the current frame image column by column, wherein the current frame image is a dark frame image and is configured to be divided by columns, then the brightness values ​​of the pixels in each column can be taken out in order by column sorting and the number of pixels in a certain search area can be counted, so as to screen out the line laser position of the predicted reflection position of the line laser, which may be different from the properties of the aforementioned convex hull center pixel, including the brightness value and the ordinate position. Since it is a dark frame image, the brightness value of the pixel is not large, so the initial pixel position disclosed in the aforementioned embodiment cannot be collected.

[0091] The robot searches for each pixel point of the current column in sequence, specifically starting from the pixel points in the top row of the current column and traversing to the pixel points in the bottom row of the current column in sequence; or starting from the pixel points in the bottom row of the current column and traversing to the pixel points in the top row of the current column in sequence, so as to complete the search of each pixel point in the same column, the detection of the brightness value and the counting of the number of pixels; in the process of searching the pixel points in the current column, according to the size relationship between the brightness value of the pixel point currently being searched in the current column of the current frame image and the brightness value of the pixel point at the corresponding position of the previous bright frame image and the brightness value of the pixel point at the corresponding position of the previous bright frame image, legal pixels are screened out from the current column of the current frame image, wherein the coordinate position of the pixel point at the corresponding position of the previous bright frame image is equal to the coordinate position of the pixel point currently being searched in the current frame image, at least the relative position relationship formed with respect to the origin of the coordinate system is equivalent, and the size relationship described in this embodiment is determined by setting a threshold value. Then, in the current column of the current frame image, at least two legal pixel points that are adjacent in position are connected to form a positioning line segment; after all the legal pixel points that are adjacent in position are connected, the positioning line segment with the longest length is selected, wherein, after all the legal pixel points that are adjacent in position are connected, there are multiple positioning line segments, each positioning line segment is a line segment formed by connecting legal pixel points that are continuously arranged in pixel positions in the same column in sequence, and then the positioning line segment is selected for length comparison to obtain the positioning line segment with the longest length, because the difference between the brightness value of the legal pixel point and the brightness value of the pixel point at the same position of the previous bright frame image is controlled within a reasonable threshold range, and the brightness value of the pixel point at the same position of the previous bright frame image is controlled within a certain brightness value range to prevent strong light interference, then the brightness value of the continuously arranged legal pixel points can predict the approximate range of the reflection position of the outgoing laser on the surface of the object to be measured, and can also enhance the anti-interference ability of ambient light. The longest positioning line segment is preferably a straight line segment and is parallel to the ordinate axis of the image coordinate system; preferably, the selected longest positioning line segment is set as the predicted laser line segment, and then the coordinates of the pixel point at the center of the predicted laser line segment are set as the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image. If the length of the selected longest positioning line segment is greater than the preset continuous length threshold, the center of the selected longest positioning line segment is set as the line laser position, which is equivalent to the convex hull center mentioned in the embodiment of executing the inter-frame tracking algorithm to improve the accuracy of obstacle detection.

[0092] It should be added that the coordinates of a line laser position are represented by the corresponding positioning coordinates. For the current frame image, the line laser position on the corresponding column is obtained by traversing column by column. Therefore, after determining the column number of the current traversal, only the ordinate value can be selected to represent the coordinates of each line laser position to identify the height information of the reflection position of the line laser on the obstacle surface, which can also be used for robot obstacle avoidance.

[0093] On the basis of the above-mentioned embodiment, the method for selecting legal pixels from the current column of the current frame image according to the magnitude relationship between the brightness value of the pixel currently searched in the current frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image comprises: subtracting the brightness value of the pixel at the same row and column position of the previous bright frame image from the brightness value of the pixel currently searched in the current frame image to obtain a relative difference of the dark frame image; when it is detected that the opposite number of the relative difference of the dark frame image is greater than a preset brightness difference threshold, and the brightness of the pixel at the same row and column position of the previous bright frame image When the value is greater than the reference bright frame image brightness threshold, the pixel currently being searched in the current frame image is set as a legal pixel, as a legal pixel screened out from the current column of the current frame image, indicating that the pixel in the current frame image is not disturbed by strong ambient light; wherein the preset brightness difference threshold is an empirical value set based on the difference in brightness values ​​of the pixel at the same position in two adjacent frame images so that the pixel information collected by the current frame image is less disturbed by the ambient light intensity; and the reference bright frame image brightness threshold is an empirical value set based on the brightness value of the pixel in the bright frame image so that the pixel in the bright frame image is less disturbed by the ambient light intensity.

[0094] In summary, when the robot detects that the current frame image captured by the camera is a dark frame image, it chooses to input the current frame image into the processing rule model corresponding to the brightness centroid algorithm to output a positioning line segment with a reasonable connection length, so as to overcome the problem of being sensitive to ambient light interference in the scene where the camera is too close to the obstacle, and correspondingly, prevent the robot from colliding with the obstacle in front that reflects the laser light due to misjudgment. Therefore, the present invention combines the inter-frame tracking algorithm and the brightness centroid algorithm to complement each other in various ambient light intensity scenes, and realizes laser positioning in the bright frame image and dark frame image that are alternately generated to reflect the reflected light of the line laser.

[0095] In some embodiments, the method of filtering out legal pixels from the current column of the current frame image based on the magnitude relationship between the brightness value of the pixel currently searched in the current frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image can also be expressed as: subtracting the brightness value of the pixel with the same row and column position in the current frame image from the brightness value of the pixel currently traversed in the previous bright frame image to obtain a relative difference of the dark frame image; when it is detected that the relative difference of the dark frame image is greater than a preset brightness difference threshold, and the brightness value of the pixel currently traversed in the previous bright frame image is greater than the reference bright frame image brightness threshold, the pixel with the same row and column position in the current frame image is set as a legal pixel, as the legal pixel filtered out from the current frame image, indicating that the current dark frame image is not disturbed by strong ambient light. It should be added that the image sequence formed by the light reflected back from the surface of the object to be measured by the line laser emitted by the line laser transmitter in the imaging plane of the camera is configured to generate bright frame images and dark frame images alternately in sequence, so that: when the current frame image captured by the camera is a bright frame image, the next frame image captured by the camera is a dark frame image; within the time interval between the camera capturing the current frame bright frame image and the camera capturing the next frame bright frame image, the camera captures the current frame dark frame image; after the camera captures the next frame bright frame image, the camera captures the next frame dark frame image.

[0096] As an embodiment, the laser positioning method further includes adjusting exposure information of the camera, specifically including:

[0097] When the robot detects that the light intensity of its environment is greater than the first preset light intensity threshold, it means that the robot detects that the intensity of visible light in the current environment is relatively large and the exposure of the camera becomes relatively large. The robot then reduces the gain of the camera (image signal amplification parameter) to obtain a first gain, so that the image of the light reflected back by the line laser on the surface of the object to be measured and captured by the camera is not overexposed, especially the image information of the visible light part is not easy to be overexposed, so as to improve the accuracy of extracting the position of the aforementioned line laser in scenes with strong ambient light; the first preset light intensity threshold is mainly a strong light threshold set according to the degree of overexposure of the image captured by the camera due to the strong visible light in the environment.

[0098] When the robot detects that the light intensity of its environment is greater than the first preset light intensity threshold, it means that the robot detects that the intensity of visible light in the current environment is relatively large and the exposure of the camera becomes relatively large, then the robot reduces the exposure time of the camera to obtain the first exposure time, so that the image of the light reflected back by the line laser on the surface of the object to be measured collected by the camera is not overexposed, especially the image information of the visible light part is not easy to be overexposed, so as to improve the accuracy of extracting the position of the aforementioned line laser in the scene with strong ambient light; the first preset light intensity threshold is mainly a strong light threshold set according to the degree of overexposure of the image collected by the camera due to the strong visible light in the environment.

[0099] When the robot detects that the light intensity of its environment is less than the second preset light intensity threshold, it means that the robot detects that the intensity of visible light in the current environment is small and the exposure of the camera becomes relatively small, then the robot increases the gain of the camera (image signal amplification parameter) to obtain a second gain, so that the image of the light reflected back by the line laser on the surface of the object to be measured collected by the camera does not appear underexposed (too dark). Preferably, the first gain is less than the second gain; but if the gain before adjusting the first gain in the aforementioned embodiment is already very large to cope with the light intensity of the environment, then the first gain is not necessarily less than the second gain; thereby improving the accuracy of extracting the aforementioned line laser position in a scene with weak ambient light; the second preset light intensity threshold is mainly a strong light threshold set according to the exposure degree of the image collected by the camera to the darker visible light in the environment, and the second preset light intensity threshold is much smaller than the first preset light intensity threshold.

[0100] When the robot detects that the light intensity of its environment is less than the second preset light intensity threshold, it means that the robot detects that the intensity of visible light in the current environment is small and the exposure of the camera becomes relatively small, then the robot increases the exposure time of the camera to obtain a second exposure time, so that the image of the light reflected back by the line laser on the surface of the object to be measured collected by the camera does not appear underexposed. Preferably, the first exposure time is less than the second exposure time; but if the exposure time before the first exposure time adjusted in the aforementioned embodiment is itself very large to cope with the light intensity of the environment, then the first exposure time is not necessarily less than the second exposure time.

[0101] Therefore, this embodiment adjusts the gain and exposure time of the camera according to the current environmental conditions, so that the image seen by the camera is not overexposed or underexposed, thereby realizing dynamic exposure adjustment of the camera.

[0102] It should be added that the camera gain should be adjusted within a reasonable range to avoid noise; noise generally refers to the camera gain being adjusted too high in a low exposure environment.

[0103] As an embodiment, when the exposure information of the camera is used to adjust the power level of the line laser transmitter, the following situations exist:

[0104] When the robot detects that the current exposure value of the camera is greater than the first preset exposure threshold, the power level of the line laser transmitter for emitting the line laser is increased so that the intensity of the line laser emitted by the line laser transmitter is configured to be equal to the product of the smoothing coefficient and the current exposure value;

[0105] In preferred example 1, the current exposure value of the camera includes the third gain and / or the third exposure time. When the light intensity of the environment in which the robot is located is greater, the third gain and / or the third exposure time are adjusted to be greater to adapt to the exposure amount of the current ambient light intensity. At this time, the smoothing coefficient is set to a reasonable value to smooth the step size of the exposure value adjustment, thereby suppressing overexposure.

[0106] In preferred example 2, the current exposure value of the camera includes the first gain and / or the first exposure time. Then, when the light intensity of the environment in which the robot is located is greater, the first gain and / or the first exposure time adjusted according to the above embodiment become smaller. At this time, the smoothing coefficient is set to a reasonable value for smoothing the step size of the exposure value adjustment. When the first gain and / or the first exposure time become smaller, the intensity of the line laser emitted by the line laser transmitter can be suppressed to become smaller to adapt to the exposure amount of the current ambient light intensity.

[0107] On the basis of the above-mentioned preferred example 1 or preferred example 2, the power level of the line laser emitter for emitting the line laser is automatically adjusted until the intensity of the line laser emitted by the line laser emitter (the emission power of the line laser emitter) is equal to the product of the smoothing coefficient and the current exposure value, thereby achieving the use of a stronger line laser level in a high-brightness environment, and also avoiding a more drastic change in the current exposure value and avoiding overexposure of the image of the line laser emitted by the line laser emitter on the surface of the object to be measured that is captured by the camera, so that the robot can accurately search for the position of the line laser from the current frame image according to the inter-frame tracking algorithm disclosed in the above-mentioned embodiment, at least ensuring that the brightness value of the pixel point and the gradient value between two pixel points are within a reasonable range, and the obstacle can also be captured by the camera with the image of the reflected light of the line laser on its surface when the ambient light is bright.

[0108] When the robot detects that the current exposure value of the camera is less than the second preset exposure threshold, the power level of the line laser transmitter for emitting the line laser is lowered so that the intensity of the line laser emitted by the line laser transmitter is configured to be equal to the product of the smoothing coefficient and the current exposure value. The first preset exposure threshold is greater than the second preset exposure threshold to reflect that the current ambient light is darker.

[0109] In preferred example three, the current exposure value of the camera includes the fourth gain and / or the fourth exposure time; when the light intensity of the robot's environment is smaller, the pre-adjusted third gain and / or the third exposure time will be reduced to adapt to the exposure required by the current ambient light intensity. At this time, the smoothing coefficient is set to a reasonable value to smooth the step size of the exposure value adjustment, thereby suppressing underexposure.

[0110] Preferred example four, the current exposure value of the camera includes the second gain and / or the second exposure time. Then, when the light intensity of the environment in which the robot is located is smaller, the second gain and / or the second exposure time adjusted according to the above embodiment become larger. At this time, the smoothing coefficient is set to a reasonable value for smoothing the step size of the exposure value adjustment, which can suppress the intensity of the line laser emitted by the line laser emitter from becoming larger when the second gain and / or the second exposure time become larger, so as to adapt to the exposure amount of the current ambient light intensity.

[0111] On the basis of the above-mentioned preferred example three or preferred example four, the power level of the line laser emitter for emitting the line laser is automatically adjusted until the intensity of the line laser emitted by the line laser emitter (the emission power of the line laser emitter) is equal to the product of the smoothing coefficient and the current exposure value, so as to avoid the current exposure value from changing more drastically and the image of the light reflected back from the surface of the object to be measured by the line laser emitted by the line laser emitter captured by the camera from being underexposed, so that the robot can accurately search for the line laser position from the current frame image in a darker environment, so that the image of the obstacle captured by the camera under dark ambient light is not overexposed so strongly, thereby not generating more reflection interference, and facilitating the use of the laser positioning method to find a more accurate line laser position.

[0112] Preferably, the current exposure value of the camera is used to reflect the exposure of the camera in an environment with a current light brightness; the current exposure value of the camera can be a positive reflection of the intensity of the ambient light; it can also be a negative reflection of the intensity of the current ambient light. For example, the smaller the current exposure value of the camera is adjusted, the stronger the current ambient light is, which will guide the line laser transmitter to increase the power level for emitting the line laser, so that even when the ambient light is brighter, the reflected light of the line laser on the surface of the obstacle can be collected.

[0113] In summary, the intensity of the line laser emitted by the line laser transmitter can describe the mapping relationship between the power level of the line laser transmitter for emitting the line laser and the current exposure value. Combined with the adjustment effect of the smoothing coefficient, it is adapted to the acquisition of image data of the reflected light of the line laser by the structured light module under a series of different exposures, so that according to the adjusted camera gain and exposure time, the power level of the line laser transmitter for emitting the line laser is adjusted to achieve the use of a stronger line laser emission power level in a high-brightness environment, so that the obstacle can also see the line laser when the ambient light is brighter, and the image will not be overexposed (for example, in strong outdoor ambient light, after the line laser is reflected back to the camera by the white obstacle, the camera gain or exposure time is reduced), avoiding overexposure of the image collected by the camera due to the overbright environment, thereby finding a more accurate line laser position; using a weaker line laser emission power level in a low-brightness environment makes the obstacle image less overexposed when the ambient light is darker, thereby not generating more reflection interference, and facilitating a more accurate line laser position (corresponding to the convex hull center).

[0114] Based on the various embodiments of the aforementioned laser positioning method, the present invention also discloses a robot, the body of which is equipped with a structured light module, the structured light module includes a line laser transmitter and a camera without an infrared filter, the camera receives light of various wavelengths of the line laser emitted by the line laser transmitter without a filter (such as an infrared filter) in the lens, so that the imaging information of infrared light and visible light are retained in the image captured by the camera. A controller is provided inside the robot, the controller is electrically connected to the structured light module, and the controller is configured to execute the laser positioning method to obtain the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, that is, to obtain the line laser position in the bright frame image and the line laser position in the dark frame image; wherein the line laser emitted by the line laser transmitter is within the field of view of the camera.

[0115] In this embodiment, the controller can control the line laser emitter and the camera to work. Optionally, the controller can control the exposure of the camera on the one hand, and on the other hand, can control the line laser emitter to emit a line laser to the outside during the exposure of the camera, so that the camera can collect the environmental image detected by the line laser. Among them, the controller can control the camera and the line laser emitter to work simultaneously, or alternately, and there is no limitation on this. It should be noted that the laser light reflected by the photographic object (the surface of the object to be measured) is projected onto the photosensitive film through the lens of the camera, causing it to undergo a chemical change and produce an image. This process is called exposure.

[0116] Preferably, the controller may use an image processing hardware device with FPGA and DSP. In order to achieve faster processing speed, considering that FPGA has obvious advantages in processing streaming parallel computing, some operations involving morphological processing of images are performed in FPGA, and the rest are performed in DSP. Even if the resolution of the image is increased, it will not consume more processing time. When the image size is 2048x2048 pixels, the processing speed of the image processing system can reach 6 frames / s, which can meet the accuracy and real-time requirements of robot obstacle avoidance at the same time.

[0117] In some embodiments, the horizontal viewing angle of the camera is configured to receive the light reflected by the line laser within the width of the robot body in front of the robot to obtain an image of the environment detected by the line laser. In order to obtain the corresponding horizontal viewing angle of the camera, a wide-angle lens or a non-wide-angle lens can be used. The specific use depends on the width of the robot body. It only needs to be able to collect the line laser within the entire body range.

[0118] And / or the installation height of the structured light module on the robot body is configured to be positively correlated with the height of the obstacle to be measured, so that the obstacle to be measured occupies the effective field of view of the camera. Among them, the installation heights of the line laser emitter and the camera need to be determined according to the size of the obstacle to be measured. The larger the installation height of the structured light module on the robot body, the larger the longitudinal space that can be covered, but it deviates further from the small-sized obstacle to be measured, so the local details collected are less, and the detection accuracy of the small-sized obstacle to be measured is reduced; the smaller the installation height of the structured light module on the robot body, the smaller the longitudinal space that can be covered, but it is closer to the small-sized obstacle to be measured, so the local details of the obstacle collected increase, and the detection accuracy of the small-sized obstacle to be measured is improved.

[0119] As for the installation height, at the installation height of the structured light module, the line laser emitter and the camera can be located at different heights. For example, on the top of the robot, the line laser emitter is higher than the camera; or, the camera is higher than the line laser emitter; of course, the line laser emitter and the camera can also be located at the same height. In actual use, the structured light module will be installed on a self-moving robot (such as a sweeping robot, patrol robot and other autonomous mobile devices). In this case, the distances between the line laser emitter and the camera to the working surface of the robot (such as the ground) are different. For example, the distance from the camera to the working surface is 32mm, and the distance from the line laser emitter to the working surface is 47mm.

[0120] As an embodiment, the coverage range of the upper viewing angle of the camera is configured to cover the bottom of the plane formed by the line laser emitted by the line laser transmitter, specifically, a laser plane composed of multiple beams of line laser, which extends and spreads along the emission direction of the line laser transmitter and shoots to the ground where the robot is traveling. Preferably, the angle between the laser plane and the working surface of the robot is 15 degrees; the coverage range of the lower viewing angle of the camera is configured to cover the light reflected back from the surface of the obstacle in front of the robot's body by the line laser emitted by the line laser transmitter; therefore, the pitch angle of the camera can be appropriately adjusted according to the requirements of the map image required for navigation; the lower viewing angle of the camera is the angle formed by detecting from top to bottom, forming the camera's top-down viewing angle; the upper viewing angle of the camera is the angle formed by detecting from bottom to top, forming the camera's top-down viewing angle; wherein the pitch angle of the camera is divided into the lower viewing angle of the camera and the upper viewing angle of the camera. Preferably, the upper viewing angle of the camera is set to 24 degrees, and the lower viewing angle of the camera is set to 18 degrees.

[0121] And / or the heading angle formed by the deflection of the camera (optical axis of the lens) relative to the central axis of the robot is maintained within a preset error angle range, so that the optical axis of the camera is parallel to the travel direction of the robot, and the camera receives the light reflected back by the line laser within the width of the robot in front of the robot, so as to detect obstacles directly in front of the robot in real time while the robot is walking.

[0122] And / or the roll angle of the camera generated by the rotation of its optical axis is maintained within a preset error angle range, so that the camera receives the light reflected back by the line laser within the width of the robot body in front of the robot, wherein the camera is rotatably mounted on the body of the robot, and the preset error angle range is set to -0.01 to 0.01 degrees, so that the heading angle formed by the deflection of the camera (optical axis of the lens) relative to the central axis of the robot is maintained at about 0 degrees, and the roll angle of the camera generated by the rotation of its optical axis is also maintained at about 0 degrees.

[0123] After the structured light module is installed, the angle between the center line of the line laser emitted by the line laser emitter and the installation baseline of the line laser emitter is equivalent to the angle between the laser surface and the working surface of the robot, preferably 15 degrees; the installation baseline refers to a straight line where the line laser emitter is located when the line laser emitter is located at an installation height above the robot, or a straight line where the line laser emitter and the camera are located when the line laser emitter and the camera are located at the same installation height.

[0124] In this embodiment, the emission angle of the line laser emitter is not limited. The emission angle is related to the detection distance that the robot where the structured light module is located needs to meet, the body width of the robot, and the mechanical distance between the line laser emitter and the camera. When the detection distance that the robot where the structured light module is located needs to meet, the body width of the robot, and the mechanical distance between the line laser emitter and the camera are determined, the emission angle of the line laser emitter can be directly obtained through the trigonometric function relationship, that is, the emission angle is a fixed value.

[0125] Of course, if a specific emission angle is required, it can be achieved by adjusting the detection distance that the robot where the structured light module is located needs to meet and the mechanical distance between the line laser emitter and the camera. In some application scenarios, when the detection distance that the robot needs to meet and the body width of the robot are determined, the emission angle of the line laser emitter can be changed within a certain angle range by adjusting the mechanical distance between the line laser emitter and the camera.

[0126] As an embodiment, if the installation distance between the camera and the line laser module is larger, the coordinate offset of the pixel point used to represent the reflection position of the line laser on the surface of the obstacle relative to the center of the camera in the image captured by the camera increases, wherein the pixel point used to represent the reflection position of the line laser on the surface of the obstacle includes but is not limited to the convex hull center pixel point obtained in the above embodiment, the pixel point that meets the convex hull feature, and the line laser position. When the robot approaches an obstacle, the larger the installation distance between the camera and the line laser module, the larger the longitudinal jump of the pixel point in the image captured by the camera, thereby obtaining more local detail information and improving the detection accuracy of the obstacle.

[0127] It should be noted that the installation distance refers to the mechanical distance between the line laser emitter and the camera (or called the baseline distance). The mechanical distance between the line laser emitter and the camera can be flexibly set according to the application requirements of the structured light module. Among them, the mechanical distance between the line laser emitter and the camera, the detection distance that the robot where the structured light module is located needs to meet, and the body width of the robot can determine the size of the measurement blind area to a certain extent. For the robot where the structured light module is located, its body width is fixed, and the measurement range and the mechanical distance between the line laser emitter and the camera can be flexibly set according to demand, which means that the mechanical distance and the blind area range are not fixed values. Under the premise of ensuring the measurement range (or performance) of the robot, the blind area range should be minimized as much as possible. However, the larger the mechanical distance between the line laser emitter and the camera, the larger the controllable distance range, which is conducive to better control of the blind area size to improve the detection accuracy of obstacles.

[0128] In some application scenarios, the structured light module is applied to a sweeping robot, for example, it can be installed on the impact plate of the sweeping robot or on the robot body. For the sweeping robot, the following example gives a relatively reasonable range of mechanical distances between the line laser emitter and the camera. For example, the mechanical distance between the line laser emitter and the camera can be greater than 20mm; further optionally, the mechanical distance between the line laser emitter and the camera is greater than 30mm. Furthermore, the mechanical distance between the line laser emitter and the camera is greater than 41mm. It should be noted that the range of mechanical distances given here is not only applicable to the application of the structured light module in a sweeping robot, but also to the application of the structured light module in other devices with specifications and dimensions that are close to or similar to those of the sweeping robot.

[0129] As an embodiment, the emission angle of the line laser transmitter and the receiving angle of the camera are set as follows: the line laser transmitter transmits the line laser to a preset detection position in front of the body, and the line laser is reflected back to the camera at the preset detection position to form the pixel point that meets the convex hull feature or the convex hull center pixel point in the image captured by the camera, wherein the length of the laser line segment formed by the line laser at the preset detection position is greater than the body width of the robot; the reflection position of the line laser after hitting the ground depends on the lateral emission angle of the line laser (i.e., the emission angle of the line laser reflector) and the lateral pixel viewing angle of the camera (i.e., the receiving angle of the camera, corresponding to the horizontal viewing angle), and the line laser hits the front so that the horizontal length of the line laser extracted by the camera is slightly wider than the body width of the robot.

[0130] Whenever the robot walks a preset distance in the direction from the current position to the preset detection position, the horizontal distance between the preset detection position and the robot becomes smaller. In some embodiments, the robot approaches the obstacle at the preset detection position; the coordinate offset of the pixel point in the image captured by the camera for representing the same reflection position of the line laser in the preset detection position relative to the center of the camera increases, that is, the closer the line laser hits the ground in front of the machine, the larger the longitudinal jump of the pixel point in the image captured by the camera for representing the reflection position of the line laser at the same travel distance of the robot, the more local information is captured, and the higher the detection accuracy of the obstacle.

[0131] In summary, during the walking process of the robot, when the distance between the obstacle and the camera (or the structured light module as a whole in combination with the laser emitter) decreases, the vertical coordinate position jump of the pixel point used to reflect the same reflection position of the line laser in the image captured by the camera increases, and the number of pixel points used to characterize the same local area of ​​the obstacle increases, so that the gradient value of the brightness between the pixel points may decrease, and the number of pixel points in the same frame image may remain unchanged, thereby improving the detection accuracy of the obstacle. Among them, the pixel points used to reflect the reflection position of the line laser include pixel points that meet the convex hull characteristics. As the robot approaches the obstacle, the position of the pixel points that meet the convex hull characteristics will jump longitudinally (the vertical coordinate changes), so that the obstacle in the image captured by the robot's camera changes from the original overall outline to a local outline, at least the outline height of the obstacle covered in the longitudinal direction changes, then the number of pixel points required for the local outline collected before approaching the obstacle will increase, thereby improving the accuracy of obstacle detection. Furthermore, the greater the installation distance between the camera and the line laser emitter in the robot, for example, the greater the installation height of the line laser emitter relative to the camera, the greater the change in the vertical coordinate of the pixel point used to represent the reflection position of the line laser on the surface of the obstacle, and the obstacle in the same frame image captured by the camera changes from the original overall outline to a local outline, then the number of pixel points used to characterize the same local area of ​​the obstacle increases relative to before the installation height becomes larger, or each time the robot walks a test distance, the number of pixel points used to characterize the same local area of ​​the obstacle in the current frame image collected in real time increases relative to before the installation height becomes larger, thereby improving the accuracy of obstacle detection, and the size of obstacles that can be detected by robots relative to the prior art can become smaller.

[0132] It should be noted that the pixel points that meet the convex hull feature are configured to simulate part or all of the point information of the laser line segment formed by projecting a line laser onto the surface of the object to be measured; the robot sets the set of pixel points that meet the convex hull feature of the image to the pixel points whose brightness values ​​decrease from the center of the convex hull along the current column to the upper and lower sides respectively, and the set of pixel points composed of the center of the convex hull, and the set of pixel points forms a convex hull, wherein the center of the convex hull is the pixel point with the largest brightness value in the set of pixel points; within the set of pixel points that meet the convex hull feature, starting from the center of the convex hull, the brightness value decreases upward along the current column to generate the first gradient value, and the brightness value decreases downward along the current column to generate the second gradient value.

[0133] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0134] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable technicians familiar with the technical field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A laser positioning method based on image information, characterized in that: The execution subject of the laser positioning method is a robot equipped with a structured light module, which includes a line laser transmitter and a camera without an infrared filter, so that the image collected by the camera retains the imaging information of infrared light and the imaging information of visible light; The laser positioning method comprises: The robot controls the camera to collect images of light reflected from the surface of the object to be measured by the line laser emitted by the line laser transmitter, and detects the brightness and darkness type of the image collected by the camera; When the robot detects that the current frame image captured by the camera is a bright frame image, the robot searches for the line laser position from the current frame image by executing the inter-frame tracking algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image; The method for the robot to search for the position of the outgoing laser from the current frame image by executing the inter-frame tracking algorithm includes: Step 1, the robot traverses the current frame image column by column, and obtains the initial pixel position in the corresponding column of the current frame image, and excludes the pixel points where there is no line laser position in the current frame image according to the pixel points in the corresponding column that meet the preset brightness distribution characteristics, wherein the line laser position is used to indicate the reflection position of the line laser on the surface of the object to be measured; Step 2, except for the column where the pixel point where the line laser position does not exist, in the current column of the current frame image, the robot sets the initial pixel position in the current column as the search center, and then searches for the pixel points within a search radius upward along the current column from the search center, and searches for the pixel points within a search radius downward along the current column from the search center; then, according to the difference between the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward under the search states corresponding to the search centers determined twice adjacently, and the inter-frame matching relationship formed by the same type of numerical values ​​of the pixel points in the same column of the current frame image relative to the reference frame image, the convex hull center pixel point in the current column is screened out to update the convex hull center pixel point determined in the current column of the current frame image last time; wherein the reference frame image is configured as a bright frame image where the line laser position is located that the robot has found most recently before the current frame image is acquired; whenever the search center in the current column is updated once, the convex hull center pixel point set in the current column is also updated once; Step 3: according to the magnitude relationship between the brightness value in the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser transmitter in the previous dark frame image and the brightness value of the convex hull center pixel point in the current frame image, interference points are eliminated from the screened convex hull center pixel points; after the robot traverses the convex hull center pixel points in all columns of pixel points in the current frame image to eliminate all interference points, the coordinates of the remaining convex hull center pixel points are set as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image, and the robot is determined to search for the line laser position determined in the corresponding column in the current frame image to connect the laser line segment formed by the line laser emitted by the line laser transmitter on the surface of the object to be measured, and it is determined that the robot has searched for the line laser position from the current frame image by executing the inter-frame tracking algorithm; Among them, the line laser position determined in the same column is the position of the convex hull center pixel point that was last updated in the same column after the robot traverses all the pixels in the same column. The coordinates of a line laser position are expressed using the corresponding positioning coordinates; When the robot detects that the current frame image captured by the camera is a dark frame image, the robot extracts the line laser position from the current frame image by executing the brightness centroid algorithm, and then sets the coordinates of the line laser position as the positioning coordinates of the line laser emitted by the line laser transmitter in the current frame image; The method for the robot to extract the line laser position from the current frame image by executing the brightness centroid algorithm includes: The robot traverses the current frame image column by column; The robot searches for each pixel point in the current column in sequence, and selects legal pixel points from the current column of the current frame image according to the magnitude relationship between the brightness value of the currently searched pixel point in the current column of the current frame image and the brightness value of the pixel point at the corresponding position of the previous bright frame image and the brightness value of the pixel point at the corresponding position of the previous bright frame image; Then, in the current column of the current frame image, at least two legal pixel points that are adjacent in position are connected to form a positioning line segment; after all the legal pixel points that are adjacent in position are connected, the positioning line segment with the largest length is selected; If the length of the selected positioning line segment with the longest length is greater than the preset continuous length threshold, the center of the selected positioning line segment with the longest length is set as the line laser position.

2. The laser positioning method according to claim 1, characterized in that: In step 2, whenever a convex hull center pixel point is screened out for a search center, an adjacent pixel point searched upward or downward along the current column starting from the search center is updated as the search center, and step 2 is re-executed to obtain a new convex hull center pixel point and update the new convex hull center pixel point as the convex hull center pixel point; each of the search centers is within the coverage area of ​​a search radius relative to the initial pixel position, wherein the search radius is set to a first preset pixel distance; the screened convex hull center pixel point is the last updated convex hull center pixel point in each column of convex hull center pixel points in the current frame image; the screened convex hull center pixel point is a convex hull center pixel point that is closest to the origin of the coordinate system of the current frame image in each column of convex hull center pixel points in the current frame image; Among them, the robot sets the set of pixel points that meet the convex hull feature on the current column of the current frame image to the pixel points whose brightness values ​​decrease from the center of the convex hull along the current column to the upper and lower sides respectively, and the pixel point set composed of the convex hull center to form a convex hull, the convex hull center is the pixel point with the largest brightness value in the pixel point set, and the convex hull center pixel point is set as the pixel point belonging to the convex hull center; in the set of pixel points that meet the convex hull feature, starting from the center of the convex hull in the upward direction along the same column, the brightness value of the pixel point decreases upward along the current column and generates a first gradient value between the brightness values ​​of two adjacent pixel points, and starting from the center of the convex hull in the downward direction along the same column, the brightness value of the pixel point decreases downward along the current column and generates a second gradient value between the brightness values ​​of two adjacent pixel points, so that the convex hull center belongs to the search center.

3. The laser positioning method according to claim 2, characterized in that: In step 2, the method for selecting the central pixel point of the convex hull according to the difference between the brightness value of the pixel point searched upward and the brightness value of the pixel point searched downward in the search states corresponding to the search centers determined twice adjacently, and the inter-frame matching relationship formed by the same type of values ​​of the current frame image in the same column of pixels relative to the reference frame image, includes: In the current column of the current frame image, the brightness value of the search center is controlled to be compared with the brightness value of the convex hull center pixel point located in the same column searched last time; the convex hull center pixel point located in the same column searched last time is the convex hull center pixel point screened out in the same column of the current frame image for the search center determined last time, the search center determined last time is a pixel point adjacent to the currently determined search center downward or upward in the current column of the current frame image, and the column order of the pixels in the same column of the current frame image is equal to the column order of the current column of the current frame image; If the brightness value of the search center currently determined is greater than the brightness value of the pixel point at the center of the convex hull located in the same column searched last time, then in the current column of the current frame image, search for pixels upward from the search center, and count the pixels whose brightness values ​​decrease according to the first gradient value until the upward counting stop condition is met, then mark the number of pixels whose brightness values ​​decrease according to the first gradient value as the upward gradient descent number, and stop searching for pixels upward until the next update of the search center; and search for pixels downward from the search center, and count the pixels whose brightness values ​​decrease according to the second gradient value until the downward counting stop condition is met, then mark the number of pixels whose brightness values ​​decrease according to the second gradient value as the downward gradient descent number, and stop searching for pixels downward until the next update of the search center; When the robot determines that the number of upward gradient descents counted in the current column of the current frame image is greater than or equal to the number of upward gradient descents required to be counted for the last search for the center pixel point of the convex hull located in the same column, and / or determines that the number of downward gradient descents counted in the current column of the current frame image is greater than or equal to the number of downward gradient descents required to be counted for the last search for the center pixel point of the convex hull located in the same column, among the pixels traversed by the robot in the current column of the current frame image, if it is detected that the first gradient value and the second gradient value are not equal to the first preset gradient parameter, and the absolute value of the difference between the first gradient value and the second gradient value is less than the second preset gradient parameter, And the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched upward along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference in brightness values ​​of the same type formed by searching upward among the pixels in the same column of the reference frame image, and the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downward along the current column and the brightness value of the pixel at the currently determined search center is greater than the absolute value of the difference in brightness values ​​of the same type formed by searching downward among the pixels in the same column of the reference frame image, then the robot marks the currently determined search center as the convex hull center pixel; wherein the first preset gradient parameter is less than the second preset gradient parameter.

4. The laser positioning method according to claim 3, characterized in that: The absolute value of the difference of the brightness values ​​of the same type formed by the upward search in the same column of the pixel points of the reference frame image is the absolute value of the difference between the brightness value of the pixel point with the smallest brightness value searched upward in a column with the same column order as the current column and the brightness value of the pixel point at the search center finally determined in the same column in the reference frame image, starting from the search center finally determined in the column order as the current column, wherein the distance between the pixel point with the smallest brightness value searched upward and the search center finally determined in the same column is less than or equal to the search radius; The absolute value of the difference in brightness values ​​of the same type formed by searching downward in the same column of pixels in the reference frame image is the absolute value of the difference between the brightness value of the pixel with the smallest brightness value searched downward along a column with the same column order as the current column, starting from the search center finally determined in a column with the same column order as the current column, and the brightness value of the pixel at the search center finally determined on the same column, wherein the distance between the pixel with the smallest brightness value searched downward and the search center finally determined on the same column is less than or equal to the search radius.

5. The laser positioning method according to claim 3, characterized in that: For a currently determined search center, step 2 further includes: If the brightness value of the pixel at the search center is greater than the brightness value of the pixel at the center of the convex hull located in the same column searched last time, then in the current column of the current frame image, search for pixels upward from the search center and search for pixels downward from the search center; If the robot detects that the brightness value of the pixel point is not decreasing according to the first gradient value during the upward search from the search center, the preset upward gradient abnormality count number is counted once, and then the robot determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image upward. If so, the robot stops searching for pixels along the current column of the current frame image upward and determines that the upward counting stop condition is met. Otherwise, when the upward gradient abnormality frequency is greater than the first preset error number, the robot stops searching for pixels along the current column of the current frame image upward and determines that the upward counting stop condition is met. And, if the robot detects that the brightness value of the pixel point is not decreasing according to the second gradient value during the downward search from the search center, the preset downward gradient abnormality count number is counted once, and then the robot determines whether it has searched all the pixels covered within the search radius along the current column of the current frame image downward. If so, the robot stops searching for pixels along the current column of the current frame image downward and determines that the downward counting stop condition is met. Otherwise, when the upward gradient abnormality frequency is greater than the second preset error number, the robot stops searching for pixels along the current column of the current frame image downward and determines that the downward counting stop condition is met. Alternatively, in the process of searching upward from the search center, the robot counts the pixel points whose brightness value is 255 and whose positions are adjacent to each other upward along the current column of the current frame image, and marks the number of pixel points whose brightness value is 255 and whose positions are adjacent to each other as the number of upward overexposures. When the robot detects that the number of upward overexposures is greater than the third preset error number, and / or the pixel points covered within the search radius are counted upward along the current column of the current frame image, the robot stops searching for pixel points upward along the current column of the current frame image and determines that the upward counting stop condition is satisfied; and in the process of searching downward from the search center, the robot counts the pixel points whose brightness value is 255 and whose positions are adjacent to each other downward along the current column of the current frame image, and marks the number of pixel points whose brightness value is 255 and whose positions are adjacent to each other as the number of downward overexposures. When the robot detects that the number of upward overexposures is greater than the fourth preset error number, and / or the pixel points covered within the search radius are counted downward along the current column of the current frame image, the robot stops searching for pixel points downward along the current column of the current frame image and determines that the downward counting stop condition is satisfied.

6. The laser positioning method according to claim 2, characterized in that: In step 3, the method of removing interference points from the screened convex hull center pixel points according to the magnitude relationship between the brightness value in the effective coverage area corresponding to the positioning coordinates of the line laser emitted by the line laser transmitter in the previous dark frame image and the brightness value of the convex hull center pixel point in the current frame image includes: After the robot traverses all the columns of the current frame image and obtains the latest convex hull center pixel in each column, and saves the positioning coordinates of the line laser emitted by the line laser transmitter in the previous dark image, for each convex hull center pixel in the current frame image, within a circular domain with the positioning coordinates of the line laser emitted by the line laser transmitter in the previous dark image as the center and the radius as the detection pixel distance, if the robot determines that there is at least one pixel in the circular domain whose brightness value is greater than the brightness value of the convex hull center pixel with the same coordinates as the center of the circle in the current frame image by a preset ambient light brightness threshold, then the robot determines that the convex hull center pixel with the same coordinates as the center of the circle in the current frame image is an interference point, the robot cannot find the line laser position at the interference point, and removes the interference point from the current frame image.

7. The laser positioning method according to claim 1, characterized in that: In step 1, the method of excluding the pixel points where there is no line laser position in the current frame image according to the pixel points in the corresponding column that meet the preset brightness distribution characteristics includes: If the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel point at the line laser position found in the previous round and located in the same column by a first preset brightness threshold, or the brightness value of the initial pixel position in the current column of the current frame image is greater than the brightness value of the pixel point at the line laser position found in the previous round and located in the same column by a second preset brightness threshold, then starting from a position one reference pixel distance upward from the initial pixel position in the current column of the current frame image along the current column of the current frame image, search for pixel points downward along the current column of the current frame image; if it is detected that the brightness value of a pixel point currently being searched is greater than the brightness value of the line laser position found in the previous round and located in the same column, then the initial pixel position in the current column of the current frame image is searched for pixel points downward; if it is detected that the brightness value of a pixel point currently being searched is greater than the brightness value of the line laser position found in the previous round and located in the same column, then the initial pixel position in the current column of the current frame image is searched for pixel points downward; When the brightness value of the pixel point at which the robot is located is greater than the first preset brightness threshold, or when the brightness value of a pixel point currently being searched is detected to be equal to the value 255, the error position count amount is counted once, and it is determined that the pixel point currently being searched is a pixel point that meets the preset brightness distribution characteristics; when the robot detects that the error position count amount is greater than the reference pixel count threshold, it is determined that there is no line laser position in the current column of the current frame image, then the pixel points in the current column of the current frame image are set as pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel point search range of step 2, and at the same time, it is determined that the light intensity of the environment where the robot is located is greater than the first preset light intensity threshold; The reference pixel distance is represented by the number of pixels, so that the reference pixel count threshold is equal to the reference pixel distance; Among them, the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixel points belonging to the reference frame image.

8. The laser positioning method according to claim 1, characterized in that: In step 1, the method of excluding the pixel points where there is no line laser position in the current frame image according to the pixel points in the corresponding column that meet the preset brightness distribution characteristics includes: Taking the initial pixel position in the current column of the current frame image as the center of the ring, in the current column of the current frame image, the pixel points covered by the ring area with an inner diameter of the first positioning radius and an outer diameter of the second positioning radius located below the center of the ring are marked as the first pixel points to be tested, and then the average brightness value of the first pixel points to be tested is calculated, and if the average brightness value of the first pixel points to be tested is greater than the brightness value of the pixel points at the line laser position located in the same column found in the previous round, it is determined that the first pixel points to be tested are pixels that meet the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image, then the pixel points in the current column of the current frame image are set as pixel points where there is no line laser position, and then the pixel points in the current column of the current frame image are excluded from the pixel point search range of step 2, and at the same time, it is determined that the light intensity of the environment where the robot is located is greater than the first preset light intensity threshold; wherein the first positioning radius is smaller than the second positioning radius, and the line laser position located in the same column found in the previous round is the position where the convex hull center pixel point finally determined in the same column of the reference frame image is located; Alternatively, the initial pixel position in the current column of the current frame image is taken as the center of the ring. In the current column of the current frame image, the pixel points covered by the ring area above the center of the ring, with the inner diameter being the first positioning radius and the outer diameter being the second positioning radius, are marked as second pixels to be tested. Then, the average brightness value of the second pixels to be tested is calculated. If the average brightness value of the second pixels to be tested is greater than the brightness value of the pixel points at the line laser positions in the same column found in the previous round, it is determined that the second pixels to be tested are pixels that meet the preset brightness distribution characteristics, and it is determined that there is no line laser position in the current column of the current frame image. Then, the pixels in the current column of the current frame image are set as pixels where there is no line laser position. The pixels in the current column of the current frame image are excluded from the pixel search range of step 2, and it is determined that the light intensity of the environment in which the robot is located is greater than the first preset light intensity threshold value. The first positioning radius is smaller than the second positioning radius, and the line laser position in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixels belonging to the reference frame image.

9. The laser positioning method according to claim 2, characterized in that: The initial pixel position is the position of the original pixel point in the image captured by the camera after the line laser emitted by the line laser transmitter is reflected back to the field of view of the camera by the robot's traveling plane when there is no obstacle in front of the robot; Each original pixel point corresponds to a reflection position on the traveling plane of the robot, and is used to represent the search starting point for searching the line laser position in each column of the same frame image; The reference frame image is configured as a bright frame image where the line laser position found by the robot most recently before the current frame image is acquired, wherein the line laser position found by the robot most recently is derived from the convex hull center pixel point set in the corresponding column of the reference frame image.

10. The laser positioning method according to claim 9, characterized in that: In step 1, if the initial pixel position cannot be obtained in the current column of the current frame image, the line laser position located in the same column found in the previous round is updated as the initial pixel position, and the second preset pixel distance is updated as the search radius, and step 2 is repeated to search for the convex hull center pixel point in the corresponding column; wherein the line laser position located in the same column found in the previous round is the position of the convex hull center pixel point finally determined in the same column of pixel points belonging to the reference frame image or the initial pixel position in the same column of pixel points of the first bright frame image; If the robot cannot find the convex hull center pixel point in the same column while repeatedly executing step 2, it is determined that the robot cannot find the line laser position in the same column.

11. The laser positioning method according to claim 1, characterized in that: According to the magnitude relationship between the brightness value of the pixel currently being searched in the current frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image and the brightness value of the pixel at the corresponding position of the previous bright frame image, the method for selecting legal pixels from the current column of the current frame image includes: The brightness value of the pixel currently being searched in the current frame image is subtracted from the brightness value of the pixel at the same row and column position in the previous bright frame image to obtain a relative difference of the dark frame image; when it is detected that the inverse of the relative difference of the dark frame image is greater than a preset brightness difference threshold, and the brightness value of the pixel at the same row and column position in the previous bright frame image is greater than the reference bright frame image brightness threshold, the pixel currently being searched in the current frame image is set as the legal pixel.

12. The laser positioning method according to any one of claims 1 to 11, characterized in that: The image sequence formed by the light reflected back from the surface of the object to be measured by the line laser emitted by the line laser transmitter is collected by the camera, and is configured to generate bright frame images and dark frame images alternately in sequence, so that: when the current frame image collected by the camera is a bright frame image, the next frame image collected by the camera is a dark frame image; within the time interval between the camera collecting the current frame bright frame image and the camera collecting the next frame bright frame image, the camera collects the current frame dark frame image; after the camera collects the next frame bright frame image, the camera collects the next frame dark frame image; Wherein, during the execution of the laser positioning method, the first frame image of the image sequence is a bright frame image.

13. The laser positioning method according to any one of claims 1 to 11, characterized in that: The laser positioning method also includes: When the robot detects that the light intensity of the environment in which it is located is greater than the first preset light intensity threshold, the robot reduces the gain of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not overexposed; When the robot detects that the light intensity of the environment in which it is located is greater than the first preset light intensity threshold, the robot reduces the exposure time of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not overexposed; When the robot detects that the light intensity of the environment in which it is located is less than a second preset light intensity threshold, the robot increases the gain of the camera so that the image of the light reflected by the line laser from the surface of the object to be measured collected by the camera does not appear underexposed; When the robot detects that the light intensity of its environment is less than the second preset light intensity threshold, the robot increases the exposure time of the camera so that the image of the light reflected by the line laser on the surface of the object to be measured collected by the camera is not underexposed.

14. The laser positioning method according to any one of claims 1 to 11, characterized in that: When the robot detects that the current exposure value of the camera is greater than the first preset exposure threshold, the power level of the line laser transmitter for emitting the line laser is increased so that the intensity of the line laser emitted by the line laser transmitter is configured to be equal to the product of the smoothing coefficient and the current exposure value; When the robot detects that the current exposure value of the camera is less than a second preset exposure threshold, the power level of the line laser transmitter for emitting the line laser is lowered so that the intensity of the line laser emitted by the line laser transmitter is configured to be equal to the product of the smoothing coefficient and the current exposure value; Among them, the first preset exposure threshold is greater than the second preset exposure threshold, and the current exposure value of the camera is used to reflect the exposure of the camera in the current light brightness environment; the smoothing coefficient is used to smooth the step size of the exposure value adjustment, so that the robot can search for the outgoing laser position from the current frame image.

15. A robot, characterized in that: The robot body is equipped with a structured light module, which includes a line laser transmitter and a camera without an infrared filter, so that the image collected by the camera retains the imaging information of infrared light and the imaging information of visible light; A controller is provided inside the robot, the controller is electrically connected to the structured light module, and the controller is configured to execute the laser positioning method according to any one of claims 1 to 14 to obtain the positioning coordinates of the line laser emitted by the line laser emitter in the current frame image; Among them, the line laser emitted by the line laser transmitter is within the field of view of the camera.

16. The robot according to claim 15, characterized in that: The horizontal viewing angle of the camera is configured to receive the light reflected back by the line laser within the width of the robot body in front of the robot; And / or the installation height of the structured light module on the body of the robot is configured to be positively correlated with the height of the obstacle to be measured, so that the obstacle to be measured occupies the effective field of view of the camera.

17. The robot according to claim 16, characterized in that: The coverage range of the upper viewing angle of the camera is configured to cover the bottom of the plane formed by the line laser emitted by the line laser transmitter; the coverage range of the lower viewing angle of the camera is configured to cover the light reflected back by the line laser emitted by the line laser transmitter from the obstacle surface in front of the robot body; and / or the heading angle formed by the deflection of the camera relative to the central axis of the robot is maintained within a preset error angle range, so that the optical axis of the camera is parallel to the travel direction of the robot, and the camera receives the light reflected back by the line laser within the width of the robot in front of the robot; And / or the roll angle of the camera generated by rotating along its optical axis is maintained within a preset error angle range, so that the camera receives the light reflected back by the line laser within the width of the robot in front of the robot.

18. The robot according to claim 15, characterized in that: If the installation distance between the camera and the line laser module is larger, the coordinate offset of the pixel point representing the reflection position of the line laser on the surface of the obstacle relative to the center of the camera in the image captured by the camera increases.

19. The robot according to claim 15, characterized in that: The emission angle of the line laser transmitter and the receiving angle of the camera are set as follows: the line laser transmitter transmits the line laser to a preset detection position in front of the robot body, and the line laser is reflected back to the camera at the preset detection position, wherein the length of the laser line segment formed by the line laser at the preset detection position is greater than the body width of the robot; Whenever the robot walks a preset distance in the direction from the current position to the preset detection position, the horizontal distance between the preset detection position and the robot becomes smaller, and the coordinate offset of the pixel point in the image captured by the camera that represents the same reflection position of the line laser in the preset detection position relative to the center of the camera increases.

Images