Obstacle recognition method, device, electronic device, and storage medium

Abstract

The present invention provides an obstacle recognition method, device, electronic device and storage medium, which includes: collecting a laser image of a target area, extracting candidate laser light bars in the laser image, obtaining theoretical positions of reference laser lines in the laser image, sorting valid laser light bars from the candidate laser light bars based on the theoretical positions, and obtaining a position of an obstacle based on the positions of the valid laser light bars in a world coordinate system.

Description

[Technical Field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) The present invention claims priority to a Chinese patent application filed with the China Patent Office on December 30, 2022, bearing application number 202211741448.X and entitled "Obstacle Recognition Method, Apparatus, Electronic Device, and Storage Medium," and a Chinese patent application filed with the China Patent Office on January 3, 2023, bearing application number 202310009410.1 and entitled "Method, Apparatus, Medium, and Electronic Device for Detecting the Distance Between a Robot and a Wall," the entire contents of which are incorporated herein by reference.

[0002] The present invention relates to the technical field of obstacle recognition, and more particularly to an obstacle recognition method, device, electronic device, and storage medium. [Background technology]

[0003] When a mobile robot performs a cleaning task, it must accurately recognize the relative position of the robot and obstacles, and mark the obstacles on a 2D grid map to achieve obstacle avoidance. The environment in which a mobile robot operates is filled with a large number of obstacles with unique materials and shapes. Obstacles with attributes such as reflective materials, light-absorbing materials, and low shapes typically place high demands on the mobile robot's environmental sensing capabilities. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention provides an obstacle recognition method, device, mobile robot electronic device, and storage medium for improving the accuracy of obstacle recognition. [Means for solving the problem]

[0005] According to a first aspect of the present invention, there is provided an obstacle recognition method, the obstacle recognition method comprising: collecting a laser image of the target area; extracting candidate laser light bars in the laser image; obtaining a theoretical position of a reference laser line in the laser image; sorting valid laser light bars from the candidate laser light bars based on the theoretical positions; and obtaining a position of an obstacle based on the position of the active laser light bar in a world coordinate system.

[0006] In some embodiments, obtaining a theoretical position of a reference laser line in the laser image comprises: Obtaining a light plane equation of the laser in a camera coordinate system; Obtaining a reference plane equation in the camera coordinate system; and obtaining an intersection expression between the light plane equation and the reference plane equation to obtain the theoretical position.

[0007] In some embodiments, obtaining the reference plane equation in the camera coordinate system may be specifically: A transformation matrix T from the camera coordinate system to the world coordinate system wc Transforming coordinates of a plurality of points on the reference plane in the world coordinate system into coordinates in the camera coordinate system based on the and establishing a reference plane equation in the camera coordinate system based on the transformed coordinates of the plurality of points.

[0008] JPEG2026503250000002.jpg68157

number

[0009] JPEG2026503250000004.jpg75157

number

[0010] In some embodiments, collecting a laser image of the target area includes: Emitting a horizontal line laser beam to a target area, the included angle with respect to a reference plane being greater than 0° and less than 90°; and acquiring a laser image of the target area.

[0011] In some embodiments, the method further comprises obtaining a background image of the target area; After acquiring the laser image of the target area, further performing background subtraction on the laser image based on the background image.

[0012] In some embodiments, the obstacle perception method is used in a mobile robot, before performing background subtraction on the laser image based on the background image, Obtaining motion information of a mobile robot; and performing motion compensation on the background image based on the motion information.

[0013] In some embodiments, extracting candidate laser light bars in the laser image may include: The method includes extracting pixel regions from the laser image in which pixel gradation values ​​are greater than a preset gradation value as the candidate laser light bars.

[0014] In some embodiments, sorting valid laser light bars from the candidate laser light bars based on the theoretical positions specifically includes: obtaining a distance between each of the candidate laser light bars and the theoretical position; and determining the candidate laser light bar with the smallest distance as the valid laser light bar.

[0015] In some embodiments, obtaining the position of the obstacle based on the position of the active laser light bar in a world coordinate system specifically includes: Obtaining the position of the central pixel point of the effective laser light bar in a world coordinate system to obtain the position of the obstacle.

[0016] In some embodiments, the obstacle is a wall; The method further comprises: Acquiring three-dimensional point cloud data of the wall body in a first coordinate system, which is a coordinate system established by the robot at its current position with the robot as the origin; fusing the 3D point cloud data into a robot-centered grid map to identify wall grid points in the grid map; fitting wall grid points corresponding to the wall in the grid map to obtain a wall contour; and calculating a distance between the robot and the wall based on the wall contour.

[0017] In some embodiments, acquiring three-dimensional point cloud data of the wall in a first coordinate system comprises: Acquiring light bar information projected onto the wall via a line laser, which is collected at the current position by a robot; and identifying three-dimensional point cloud data of the wall in a first coordinate system based on the light bar information.

[0018] In some embodiments, acquiring light bar information projected onto the wall via a line laser, the light bar information being collected by the robot at its current position, comprises: Acquiring a wall image collected by the robot at its current position as a first image while the line laser is turned on; Acquiring, as a second image, a wall image collected by the robot at the current position while the line laser is in an off state; and performing a differential process on the first image and the second image to obtain light bar information projected onto the wall via a line laser.

[0019] In some embodiments, determining three-dimensional point cloud data of the wall in a first coordinate system based on the light bar information includes: Identifying wall point cloud coordinates in the camera coordinate system of the light bar pixel center in the light bar information based on a mapping relationship from the camera coordinate system to the pixel coordinate system; and identifying three-dimensional point cloud data of the wall in the first coordinate system based on the wall point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system.

[0020] In some embodiments, acquiring three-dimensional point cloud data of the wall in a first coordinate system further comprises: Obtaining position change information indicating that the robot has moved from a previous position to a current position; Acquiring three-dimensional point cloud data of the wall body in a second coordinate system, which is a coordinate system established by the robot at the previous position with the robot as the origin; and converting the three-dimensional point cloud data of the wall in the second coordinate system into three-dimensional point cloud data in the first coordinate system based on the position change information.

[0021] In some embodiments, fitting wall grid points corresponding to the wall in the grid map to obtain a wall contour comprises: Identifying noise grid points from the wall grid points of the grid map, and identifying wall grid points other than the noise grid points as target grid points; and fitting the target grid points in the grid map to obtain a wall contour.

[0022] In some embodiments, identifying noise grid points from wall grid points of the grid map comprises: traversing each wall grid point in the grid map and identifying a probability value for the wall grid point that indicates a confidence that the wall grid point can be used to reflect a wall; if the probability value is equal to or less than a probability threshold, identifying the wall grid point as a first noise grid point, and identifying wall grid points other than the first noise grid point as candidate grid points; traversing each candidate grid point in the grid map and determining the number of candidate grid points in a predetermined grid region in which the candidate grid point is located; identifying the candidate grid points as second noise grid points if the number of the candidate grid points is equal to or less than a number threshold; and identifying the first noise grid points and the second noise grid points as the noise grid points.

[0023] According to a second aspect of the present invention, there is provided an obstacle recognition device, comprising: an acquisition module for acquiring a laser image of the target area; an extraction module for extracting candidate laser light bars in the laser image; an acquisition module for acquiring a theoretical position of a reference laser line in the laser image; a sorting module for sorting valid laser light bars from the candidate laser light bars based on the theoretical positions; and a transformation module for obtaining a position of an obstacle based on the position of the effective laser light bar in a world coordinate system.

[0024] According to a third aspect of the present invention, there is provided a mobile robot, the mobile robot comprising: a robot body, a horizontal laser module, an inertial measurement unit, and a control module; the horizontal laser module is installed on the robot body, the horizontal laser module includes an infrared laser transmitter and a camera, the horizontal laser module is used to collect a laser image of a target area and transmit it to the control module; the inertial measurement unit is provided in the robot body and is used to acquire the posture of the mobile robot in a world coordinate system and transmit the acquired posture to the control module; The control module is used to execute the obstacle recognition method described in any of the above embodiments.

[0025] According to a fourth aspect of the present invention, there is provided an electronic device, the electronic device comprising: a processor; a memory in which a program is stored, The program includes instructions that, when executed by the processor, cause the processor to perform the method described in any of the above embodiments.

[0026] According to a fifth aspect of the present invention, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions, the computer instructions being used to cause the computer to perform a method according to any of the preceding embodiments. [Effects of the Invention]

[0027] The obstacle recognition method according to an embodiment of the present invention can improve the accuracy of recognizing valid laser light bars by selecting valid laser light bars from candidate laser light bars with reference to the theoretical position of the reference laser line in the laser image, and further improve the accuracy of obstacle recognition. [Brief explanation of the drawings]

[0028] [Figure 1] 2 is a flowchart of an obstacle recognition method according to an embodiment of the present invention. [Figure 2] 1 is a schematic diagram of a mobile robot according to an embodiment of the present invention; [Figure 3] FIG. 2 is a schematic diagram of motion compensation according to an embodiment of the present invention; [Figure 4] This is a laser image when no obstacles are present. [Figure 5] This is a laser image when an obstacle is present. [Figure 6] 1A and 1B are schematic diagrams illustrating a scene in which distance measurement is performed using a line laser to which an embodiment of the present invention can be applied. [Figure 7] 1 shows a flowchart of a method for detecting a distance between a robot and a wall in an embodiment of the present invention. [Figure 8]10 shows a detailed flowchart for acquiring three-dimensional point cloud data of a wall body in a first coordinate system in an embodiment of the present invention. [Figure 9] 10 shows another detailed flowchart for acquiring three-dimensional point cloud data of a wall body in a first coordinate system in an embodiment of the present invention. [Figure 10] 10 shows an explanatory diagram for identifying wall grid points in the grid map according to an embodiment of the present invention; [Figure 11] 10 shows a detailed flowchart of fitting wall grid points corresponding to the walls in the grid map in an embodiment of the present invention. [Figure 12] FIG. 10 is an explanatory diagram illustrating fitting wall grid points corresponding to the wall in the grid map according to an embodiment of the present invention. [Figure 13] 10 shows a flowchart for controlling a robot to move along a wall in an embodiment of the present invention. [Figure 14] 4 shows a schematic diagram of a flow chart of an obstacle detection method according to another embodiment of the present invention; [Figure 15] 1 is a schematic diagram of an obstacle recognition device according to an embodiment of the present invention; [Figure 16] 1 shows a block diagram of a device for detecting a distance between a robot and a wall in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, the embodiments of the present invention will be described in more detail with reference to the drawings. Although several embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein; on the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are merely illustrative and do not limit the protection scope of the present invention.

[0030] It should be understood that the steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit performing steps as described. The scope of the present invention is not limited in this respect.

[0031] As used herein, the term "comprises" and its variations are open inclusive, i.e., "including, but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided in the following description. Note that concepts such as "first," "second," etc., referred to in the present invention are merely intended to distinguish different devices, modules, or units, and are not intended to limit the order or interdependence of functions performed by these devices, modules, or units.

[0032] It should be understood that the modifications "one" and "multiple" referred to in the present invention are illustrative rather than limiting, and that those skilled in the art should understand "one or more" unless the context clearly indicates otherwise.

[0033] The names of messages or information exchanged between devices in embodiments of the present invention are for illustrative purposes only and do not limit the scope of these messages or information.

[0034] 1 is a flowchart of an obstacle recognition method according to an embodiment of the present invention. As shown in FIG. 1, an embodiment of the present invention provides an obstacle recognition method, which includes the following steps: In S101, a laser image of the target area is acquired. In S102, candidate laser light bars are extracted from the laser image. In S103, the theoretical position of the reference laser line in the laser image is obtained. In S104, valid laser light bars are sorted out from the candidate laser light bars based on their theoretical positions. In S105, the position of the obstacle is obtained based on the position of the effective laser light bar in the world coordinate system.

[0035] The obstacle recognition method of this embodiment selects candidate laser light bars by referring to the theoretical position of the reference laser line in the laser image, thereby effectively eliminating the influence of reflection, refraction, etc. present in the scene on the laser image, improving the accuracy of valid laser light bar recognition and further improving the accuracy of obstacle recognition.

[0036] In the present invention, an obstacle includes any object on the floor in the environment in which the mobile robot is located, such as a table, chair, or sofa, but is not limited to these, and may also include a wall, a reinforcing chest of drawers, etc.

[0037] Specifically, the floor surface can be used as the reference plane, and the ground line can be used as the reference laser line.

[0038] In some embodiments, the obstacle perception method is used in a mobile robot, which may be a cleaning robot. The mobile robot includes a robot body and a horizontal laser module; The horizontal laser module is mounted on the robot body and includes an infrared laser transmitter and a camera. The horizontal laser module is used to collect laser images of a target area and transmit them to the control module. In this embodiment, the target area is the target movement area of ​​the robot body. The infrared laser transmitter is used to emit a horizontal line laser beam in the direction of movement of the robot body. The angle between the light plane of the horizontal line laser beam and the reference plane is greater than 0° and less than 90°, and the angle between the light plane of the horizontal line laser beam and the reference plane may be 45°, 50°, or 60°. When the optical path of the horizontal line laser beam hits an obstacle, the position of the laser beam bar in the laser image changes. This phenomenon can be utilized to effectively locate the effective laser beam bar representing the obstacle in the laser image through imaging in the camera. The position of the effective laser beam bar in the camera coordinate system can be used to calculate the position information of the obstacle surface in the world coordinate system.

[0039] The inertial measurement unit is provided in the robot body and is used to acquire the posture of the mobile robot in the world coordinate system and transmit it to the control module, which is used to execute the obstacle recognition method according to any of the embodiments.

[0040] In this embodiment, the horizontal line laser light is actively emitted by an infrared laser transmitter, thereby effectively avoiding the influence of ambient light sources. The intersection of the horizontal line laser light light plane and the reference plane is close to the mobile robot, reducing secondary reflection and refraction of the light beam and preventing adverse effects on the extraction of subsequent effective laser light bars. In addition, compared to a vertical line laser, the blind spot of the configured obstacle avoidance is smaller, allowing for effective recognition of low obstacles at close range.

[0041] JPEG2026503250000006.jpg24156

[0042] JPEG2026503250000007.jpg50156

[0043] In the camera coordinate system, the light plane equation of the laser is:

number

[0044] In the camera coordinate system, the reference plane equation is:

number

[0045] JPEG2026503250000010.jpg48156

number

[0046] The reference plane equation and the light plane equation in the camera coordinate system specify the position of the intersection of the two planes in the camera coordinate system, i.e., an expression for the theoretical position of the reference laser line in the laser image.

[0047] JPEG2026503250000012.jpg25157

number

number

[0048] JPEG2026503250000016.jpg22163

number

number

[0049] JPEG2026503250000021.jpg85155

number

[0050] In the obstacle recognition method according to this embodiment, the inertial measurement unit corrects the transformation matrix from the camera coordinate system to the world coordinate system in real time to adapt to the movement of the mobile robot.

[0051] In some embodiments, step S101 specifically includes: Emitting a horizontal line laser beam onto a target area, the angle between the horizontal line laser beam and a reference plane is greater than 0° and less than 90°; and acquiring a laser image of the target area.

[0052] moreover, acquiring a background image of the target area, the background image being collected when the horizontal line laser light is turned off; After obtaining the laser image of the target area, further This includes performing background subtraction on the laser image based on the background image.

[0053] The obstacle recognition method according to this embodiment can filter the influence of ambient light by turning on the horizontal line laser light and then turning off the horizontal line laser light to perform background subtraction, thereby improving the accuracy of obstacle recognition.

[0054] 3 is a schematic diagram of motion compensation according to an embodiment of the present invention. As shown in FIG. 3, when the robot has angular velocity, the time when the horizontal line laser beam is turned on differs from the time when the horizontal line laser beam is turned off, and there is a horizontal pixel shift between the laser image and the background image. Therefore, before performing background subtraction on the laser image based on the background image, the method further includes performing motion compensation on the background image using the rotation angle measured by the inertial measurement unit.

[0055] The obstacle recognition method according to an embodiment of the present invention takes into account the shaking that may occur in the mobile robot during the movement process, acquires movement data using an inertial measurement unit, corrects the position of the line laser light plane during the movement process, and performs motion compensation on the background image required for background subtraction, thereby aligning the background image and the laser image.

[0056] JPEG2026503250000023.jpg51157

number

[0057] In some embodiments, step S102 specifically includes extracting pixel regions from the laser image on which background subtraction has been performed, the pixel regions having pixel gradation values ​​greater than a preset gradation value, as candidate laser light bars.

[0058] Specifically, the pixel points of each column of the laser image are traversed, and pixel regions whose width is greater than a preset width and whose brightness is greater than a preset brightness are extracted as candidate laser light bars.

[0059] FIG. 4 is a laser image when no obstacle is present, and FIG. 5 is a laser image when an obstacle is present. As shown in FIGS. 4 and 5, when a single-line laser light surface is emitted, only one line laser light bar can be formed in a part of a row, and the remaining bright spots are reflections or noise. Therefore, there is a possibility that only some candidate laser light bars exist in the same row. The closer the candidate laser light bar is to the theoretical position of the reference laser line, the higher the probability that the candidate laser light bar is a valid laser light bar, and at the same time, the larger the gray value of the central pixel, the higher the probability that the candidate laser light bar is a valid laser light bar. In some embodiments, step S104 specifically includes: Obtaining a distance between each candidate laser light bar and a theoretical position; Obtaining the grayscale value of the center pixel point of each candidate laser light bar; calculating a score for each candidate laser light bar based on distance and gray value; determining the candidate laser light bar with the highest score as the valid laser light bar; Here, the higher the tone value, the higher the score, and the smaller the distance, the higher the score.

[0060] JPEG2026503250000025.jpg55156

number

number

number

[0061] JPEG2026503250000029.jpg45166

number

[0062] First, a brief description will be given of an application scenario of the present invention in which the obstacle is a wall, with reference to FIG.

[0063] Referring to FIG. 6, there is shown a schematic diagram of a scene in which distance measurement is performed using a line laser, to which an embodiment of the present invention can be applied.

[0064] As described above, in the present invention, the robot may be a cleaning robot 101, which moves back and forth within a cleaning area 102 while performing a cleaning task. In this case, the robot needs to sense obstacles present in the cleaning area 102 to avoid the robot colliding with the obstacles. Here, a wall 103 is one of the obstacles that the robot needs to sense. Typically, the robot can move along a wall, and in the process of moving along the wall, the robot needs to determine the distance between itself and the wall in real time, thereby improving the control accuracy of the robot's movement along the wall.

[0065] In the present invention, a robot can use a line laser to assist itself in moving along a wall. Specifically, the robot uses one or more line laser modules attached to it to emit a line laser outward and collect the projected laser light bar 104 at an obstacle (e.g., a wall obstacle), thereby assisting itself in moving along the wall. When the line laser module projects a laser outward, there are several projection methods. For example, as shown in FIG. 1(a), it is a vertical line laser projection method; as shown in FIG. 1(b), it is a horizontal line laser light projection method; and as shown in FIG. 1(c), it is a combination method of vertical line laser projection and horizontal line laser light projection.

[0066] 7, there is shown a flowchart of a method for detecting the distance between a robot and a wall according to an embodiment of the present invention. The method for detecting the distance between a robot and a wall may be performed by a device having a computing function. As shown in FIG. 2, the method for detecting the distance between a robot and a wall includes at least steps 710 to 770, which will be described in detail as follows: In step 710, three-dimensional point cloud data of the wall in a first coordinate system is acquired, and the first coordinate system is a coordinate system established by the robot at its current position with the robot as the origin.

[0067] In the present invention, the three-dimensional point cloud data is used to indicate the position distribution of the wall body in a first coordinate system, where the first coordinate system is a coordinate system established by the robot at its current position with the robot as the origin. Note that since the robot is moving in the cleaning area, the coordinate system established with the robot as the origin (i.e., the robot coordinate system) moves relative to the cleaning area. Furthermore, although the wall body does not move in the cleaning area, the three-dimensional point cloud data reflected in the robot coordinate system of the wall body (i.e., the position coordinates of the wall body in the robot coordinate system) changes as the robot moves in the cleaning area.

[0068] In the present invention, when a robot moves from one position to another, the line laser module attached to the robot emits a line laser outward, and based on the laser light bar projected onto the wall, three-dimensional point cloud data in the robot coordinate system of the position on the wall where the laser is projected (i.e., the coordinates in the robot coordinate system of the position on the wall where the laser is projected onto the wall) can be identified.

[0069] In one embodiment of the present invention, obtaining three-dimensional point cloud data of a wall in a first coordinate system can be performed according to the steps shown in FIG.

[0070] Referring to Figure 8, there is shown a detailed flowchart of acquiring 3D point cloud data of a wall in a first coordinate system in an embodiment of the present invention. Specifically, the process includes steps 711 and 712, In step 711, the light bar information projected onto the wall via a line laser is acquired, which is collected by the robot at its current position. In step 712, three-dimensional point cloud data of the wall in a first coordinate system is identified based on the light bar information.

[0071] In one specific example of this embodiment, acquiring the light bar information projected onto the wall via a line laser collected by the robot at its current position can be performed according to the following steps 7111 to 7113. In step 7111, with the line laser turned on, a wall image collected by the robot at the current position is acquired as the first image. In step 7112, with the line laser off, a wall image collected by the robot at its current position is acquired as a second image. In step 7113, a difference process is performed on the first image and the second image to obtain the light bar information projected onto the wall via a line laser.

[0072] In the present invention, both the wall image with the line laser turned on and the wall image with the line laser turned off can be acquired by collecting them using a camera attached to the robot.

[0073] In the present invention, a difference image is obtained by performing differential processing on the first image and the second image, and light bars whose brightness values ​​are greater than a predetermined brightness threshold are extracted from the difference image, thereby obtaining light bar information projected onto the wall via a line laser.

[0074] In another specific example of this embodiment, acquiring the light bar information projected onto the wall via a line laser, which is collected at the current position by the robot, may be performed by acquiring an image of the wall collected at the current position by the robot with the line laser turned on, and then recognizing the light bar information projected onto the wall from the image using an image recognition algorithm (e.g., an image recognition algorithm based on artificial intelligence).

[0075] In one specific example of this embodiment, determining the 3D point cloud data of the wall body in a first coordinate system based on the light bar information can be performed according to the following steps 7121 to 7122: In step 7121, based on the mapping relationship from the camera coordinate system to the pixel coordinate system, the wall point cloud coordinates in the camera coordinate system of the center of the light bar pixel in the light bar information are identified. In step 7122, three-dimensional point cloud data of the wall in the first coordinate system is identified based on the wall point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system.

[0076] In the present invention, after obtaining the light bar information projected onto the wall, first identify the light bar pixel center p=(u,v) in the light bar information, and then identify the wall point cloud coordinates pC=(x,y,z) of the pixel center in the camera coordinate system. Specifically, based on the mapping relationship from the camera coordinate system to the pixel coordinate system below, JPEG2026503250000031.jpg215155

number

[0077] In one embodiment of the present invention, acquiring the three-dimensional point cloud data of the wall in the first coordinate system may be performed according to the steps shown in FIG.

[0078] 9, there is shown another detailed flowchart of acquiring 3D point cloud data of a wall in a first coordinate system in an embodiment of the present invention, which specifically includes steps 713 to 715, In step 713, position change information is acquired when the robot moves from the previous position to the current position. In step 714, three-dimensional point cloud data of the wall in a second coordinate system is acquired, the second coordinate system being a coordinate system established by the robot at the previous position with the robot as the origin. In step 715, the three-dimensional point cloud data of the wall in the second coordinate system is converted into three-dimensional point cloud data in the first coordinate system based on the position change information.

[0079] In the present invention, the wall does not move within the cleaning area, but the robot coordinate system changes as the robot's position in the cleaning area changes, so the three-dimensional point cloud data of the wall reflected in the robot coordinate system (i.e., the position coordinates of the wall in the robot coordinate system) also changes. Based on this, it is necessary to convert the three-dimensional point cloud data in the robot coordinate system at the wall's previous position into the robot coordinate system at the current position. Specifically, based on the position change information when the robot moves from a previous position to a current position, the three-dimensional point cloud data of the wall in the second coordinate system (the coordinate system established by the robot at the previous position with the robot as the origin) is converted into three-dimensional point cloud data in the first coordinate system.

[0080] Continuing to refer to FIG. 7, in step 730, the 3D point cloud data is fused into a robot-centered grid map to identify wall grid points in the grid map.

[0081] In the present invention, in order to fuse the 3D point cloud data of the wall in the robot coordinate system, a grid map (a 2D grid map of the local wall) with the robot center as the origin is maintained, and the coordinate positions of the wall points observed in the robot coordinate system at the current position can be updated on the grid map in real time as the robot moves along the wall. As the robot moves, the wall coordinate positions observed at the previous positions are transformed into the robot coordinate system at the current position, and the point cloud data of the line laser from multiple frames is continuously fused on this grid map, thereby forming a scan of the wall.

[0082] To help those skilled in the art better understand the present invention, reference is now made to FIG.

[0083] Referring to FIG. 10, an illustration 500 is shown for identifying wall grid points in the grid map in accordance with an embodiment of the present invention.

[0084] As shown in Figure 10(d), when the robot is at the first position, the robot is located at the center of the grid map, and grid point 501 is a reflection in the grid map of the wall observed by the line laser at the robot's previous position, and grid point 502 is a reflection in the grid map of the wall observed by the line laser at the robot's first position (previous position). After the robot moves from the first position to the second position, the grid map is maintained to ensure that the robot is still positioned at the center of the grid map, as shown in FIG. 10(e). At this time, compared to FIG. 10(d), the movement of the robot changes the relative position between the actual robot and the wall. In FIG. 10(e), the relative position between the robot and the wall grid point in the grid map also changes. Specifically, when the robot moves to the upper left in the figure, the wall grid point in the grid map corresponds to the robot moving to the lower right in the figure. Specifically, grid point 503 is the reflection on the grid map of the wall observed by the line laser at the historical position after the robot moved to the second position (i.e., the current position). Grid point 504 is the reflection on the grid map of the wall observed by the line laser at the first position after the robot moved to the second position (i.e., the current position). Grid point 505 is the reflection on the grid map of the wall observed by the line laser at the second position (i.e., the current position) after the robot moved to the second position (i.e., the current position).

[0085] Still referring to FIG. 7, in step 750, wall grid points corresponding to the wall in the grid map are fitted to obtain a wall contour.

[0086] In the present invention, the wall grid points reflect the position of the wall in the cleaning area on a two-dimensional grid map, and the grid center also reflects the position of the robot in the cleaning area on a two-dimensional grid map.Therefore, the wall contour obtained by fitting the wall grid points corresponding to the wall in the grid map can be considered to be an abstract reflection of the position of the entire wall on the two-dimensional grid map.

[0087] In one embodiment of the present invention, fitting the wall grid points corresponding to the wall in the grid map to obtain the wall contour line can be performed according to the steps shown in FIG.

[0088] 11, there is shown a detailed flowchart of fitting wall grid points corresponding to the wall in the grid map in an embodiment of the present invention, specifically including steps 751 and 752. In step 751, noise grid points are identified from the wall grid points of the grid map, and wall grid points other than the noise grid points are identified as target grid points. In step 752, the target grid points are fitted in the grid map to obtain a wall contour.

[0089] Ideally, if a wall is straight, the points on the wall scanned by the line laser module attached to the robot should also be located on a straight line. However, when actually observing a wall using a line laser, various uncertainties can inevitably cause jitter or noise in the distance measurement values. When the robot's distance along the wall has high accuracy requirements and the control of the robot's movement along the wall is sensitive, the robot will constantly rotate a certain angle left or right to adjust its pose to maintain the distance from its center point to the wall, which appears to cause frequent fluctuations in the robot's behavior along the wall. To avoid the impact of wall noise on the stability and smoothness of the robot's behavior along the wall, it is necessary to perform wall fitting on a two-dimensional grid map of local obstacles and filter the distance measurement noise.

[0090] Based on this, in the present invention, noise grid points are identified from the wall grid points of the grid map, wall grid points other than the noise grid points are identified as target grid points, and finally the target grid points are fitted to the grid map to obtain a wall contour, thereby improving the accuracy of the wall contour.

[0091] In one specific example of the present invention, identifying noise grid points from wall grid points of the grid map can be performed according to the following steps 7511 to 7515. In step 7511, each wall grid point in the grid map is traversed to determine a probability value for the wall grid point, the probability value indicating the confidence that the wall grid point can be used to reflect a wall. In step 7512, if the probability value is less than or equal to a probability threshold, the wall grid point is identified as a first noise grid point, and wall grid points other than the first noise grid point are identified as candidate grid points. In step 7513, each candidate grid point in the grid map is traversed to identify the number of candidate grid points in the predetermined grid region in which the candidate grid point lies. In step 7514, if the number of the candidate grid points is less than or equal to a number threshold, the candidate grid points are identified as second noise grid points. In step 7515, the first noise grid point and the second noise grid point are identified as the noise grid points.

[0092] Specifically, in the present invention, each wall grid point in the grid map indicates a probability value of the reliability with which it is used to reflect the wall, and the larger the probability value, the higher the reliability. The magnitude of the probability value is related to the number of times each position on the wall has been scanned by a line laser. For a certain position on the wall, the more times that position has been scanned by a line laser currently and historically, the more frequently that position is reflected in the 3D point cloud data, and therefore the larger the probability value of the corresponding wall grid point in the grid map for that position, and the higher the reliability of the corresponding wall grid point.

[0093] In the present invention, wall grid points in a two-dimensional grid map of local obstacles (i.e., the grid map) are traversed to determine whether the probability value of the wall grid point is greater than a probability threshold, and a set of candidate grid points that meet the wall determination criteria (i.e., the first noise grid point is eliminated) are filtered. Furthermore, for each candidate grid point in the set of candidate grid points, it is determined whether the number of similarly determined candidate grid points within a certain size window of its neighboring area is greater than a count threshold. For example, for each candidate grid point, it is determined whether the number of similarly determined candidate grid points within its 3x3 neighboring area (i.e., out of nine grid points) is greater than three. If the count condition is met, proceed to the next step; otherwise, the candidate grid point is considered to be an independent noise grid point (i.e., the second noise grid point) and is eliminated from the set of candidate grid points. The remaining grid points in the set of candidate grid points are used as target grid points to fit the wall contour in the grid map.

[0094] Furthermore, after identifying the target grid points from the wall grid points, the target grid points are fitted in the grid map to form a wall Obtain the body contour.

[0095] In one specific example of this embodiment, the target grid points can be fitted based on the least squares method to obtain the following wall contour equation: y=ax+b (24) Specifically, for each target grid point pi = (xi, yi), the square of the error in the distance to the wall contour is (yi - (axi + b)) 2 It can be expressed as JPEG2026503250000033.jpg89157

[0096] Further rewriting, JPEG2026503250000034.jpg26157

number

[0097] In order that those skilled in the art may better understand the present invention, reference is now made to FIG.

[0098] Referring to FIG. 12, an illustration 700 of fitting wall grid points corresponding to the walls in the grid map is shown in accordance with an embodiment of the present invention.

[0099] As shown in FIG. 12, after filtering noise grid points 701 from the wall grid points of the grid map, target grid points 702 are obtained, and a wall contour line 703 is obtained by fitting the target grid points.

[0100] Still referring to FIG. 7, in step 770, the distance between the robot and the wall is calculated based on the wall contour.

[0101] In the present invention, after the wall contour line y=ax+b is obtained by fitting the target grid points, the distance between the robot and the wall can be calculated based on the wall contour line. That is, the distance information from the robot center point pr=(xr, yr) to the wall contour line y=ax+b is JPEG2026503250000037.jpg29117This distance information can be output to the robot wall following planning and control module and used for accurate control of the robot as it follows a wall.

[0102] As can be seen from the method shown in Figure 7, the present invention acquires 3D point cloud data of a wall in a coordinate system established with the robot at its current position as the origin, merges the 3D point cloud data into a robot-centered grid map to identify wall grid points in the grid map, and then fits the wall grid points corresponding to the wall in the grid map to obtain a wall contour. The distance between the robot and the wall can be calculated based on the wall contour. Because the 3D point cloud data of the wall in the robot coordinate system can accurately reflect the relative position between the wall and the robot, merging the 3D point cloud data into a robot-centered grid map ensures that the wall grid points in the grid map accurately reflect the relative position between the wall and the robot. Therefore, the distance between the robot and the wall can be accurately calculated based on the wall contour obtained by fitting the wall grid points.

[0103] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

[0104] In order to help those skilled in the art to have a better overall understanding of the present invention, the application process of the solution of the present invention will be briefly described below in one specific embodiment with reference to FIG.

[0105] 13, there is shown a flowchart of a control robot moving along a wall in an embodiment of the present invention, which specifically includes the following steps 801 to 809. In step 801, the robot moves from one location to another. In step 802, a wall image is acquired with the line laser on. In step 803, a wall image is acquired with the line laser off. In step 804, robot pose data is obtained. In step 805, three-dimensional point cloud data of the wall is identified based on the wall image with the line laser on and with the line laser off, and based on the robot pose data image. In step 806, a two-dimensional grid map is generated based on the three-dimensional point cloud data and the robot pose data. In step 807, noise grid points in the two-dimensional grid map are filtered. In step 808, the wall contour is fitted in the two-dimensional grid map, and the distance from the robot to the wall is calculated based on the wall contour. In step 809, the robot is controlled to move along the wall based on the distance from the robot to the wall.

[0106] In the present invention, when a robot moves from one position to another (i.e., its current position), 3D point cloud data of a wall in the robot coordinate system at the current position is determined in real time, the 3D point cloud data is integrated into a robot-centered grid map to identify wall grid points in the grid map, and then the wall grid points corresponding to the wall are fitted to the grid map to obtain a wall contour. The distance between the robot and the wall can be calculated based on the wall contour. The 3D point cloud data of the wall in the robot coordinate system can accurately reflect the relative position between the wall and the robot. By incorporating the 3D point cloud data into the robot-centered grid map, the wall grid points in the grid map can be ensured to accurately reflect the relative position between the wall and the robot. Therefore, the distance between the robot and the wall can be accurately calculated based on the wall contour obtained by fitting the wall grid points, further improving the control accuracy of the robot's movement along the wall.

[0107] Referring to FIG. 14, FIG. 14 shows a schematic diagram of a flow chart of an obstacle detection method according to an embodiment of the present invention, and this embodiment includes steps 901 to 909. In step 901, a laser image of the target area is acquired. In step 902, candidate laser light bars in the laser image are extracted. In step 903, the theoretical position of the reference laser line in the laser image is obtained. In step 904, valid laser light bars are selected from the candidate laser light bars based on the theoretical positions. In step 905 , the position of the wall is obtained based on the position of the effective laser light bar in the world coordinate system, and the obtained position of the wall is set as the initial value of the wall in step 906 . In step 906, three-dimensional point cloud data of the wall in a first coordinate system is acquired, the first coordinate system being a coordinate system established by the robot at its current position with the robot as the origin. In step 907, wall grid points are identified in a robot-centered grid map by fusing the 3D point cloud data into the grid map. In step 908, wall grid points corresponding to the wall in the grid map are fitted to obtain a wall contour. In step 909, the distance between the robot and the wall is calculated based on the wall contour.

[0108] Hereinafter, an embodiment of the apparatus of the present invention will be described, which can be used to perform the method in the above embodiment of the present invention. For details not disclosed in the embodiment of the apparatus of the present invention, please refer to the above embodiment of the present invention.

[0109] 15 is a schematic diagram of an obstacle recognition device according to an embodiment of the present invention. Based on the same concept as shown in FIG. 14, an exemplary embodiment of the present invention further provides an obstacle recognition device, which comprises: an acquisition module 1 for acquiring a laser image of a target area; an extraction module 2 for extracting candidate laser light bars in the laser image; an acquisition module 3 for acquiring the theoretical position of the reference laser line in the laser image; a sorting module 4 for sorting valid laser light bars from the candidate laser light bars based on their theoretical positions; and a transformation module 5 for obtaining the location of the obstacle based on the location of the effective laser light bar in the world coordinate system.

[0110] 16, there is shown a block diagram of an apparatus 900 for detecting a distance between a robot and a wall according to an embodiment of the present invention. As shown in FIG. 15, the apparatus 900 for detecting a distance between a robot and a wall according to an embodiment of the present invention includes an acquisition unit 901, a fusion unit 902, a fitting unit 903 and a calculation unit 904.

[0111] Here, the acquisition unit 901 is used to acquire three-dimensional point cloud data of the wall in a first coordinate system, which is a coordinate system established by the robot at its current position with the robot as the origin. A fusion unit 902 is used to fuse the 3D point cloud data into a robot-centric grid map and identify wall grid points in the grid map. A fitting unit 903 is used to fit the wall grid points corresponding to the wall in the grid map to obtain a wall contour line. A calculation unit 904 is used to calculate the distance between the robot and the wall based on the wall contour.

[0112] An exemplary embodiment of the present invention further provides an electronic device, the electronic device including at least one processor and a memory communicatively coupled to the at least one processor, the memory storing a computer program executable by the at least one processor, the computer program being used, when executed by the at least one processor, to cause the electronic device to perform a method according to an embodiment of the present invention.

[0113] An exemplary embodiment of the present invention further provides a non-transitory computer-readable storage medium having stored thereon a computer program, the computer program being used to cause the computer to perform a method according to an embodiment of the present invention when executed by a processor of the computer.

[0114] It should be understood that the above-described specific embodiments of the present invention are merely for illustrative purposes of explaining or interpreting the principles of the present invention, and are not intended to limit the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included within the scope of protection of the present invention. Furthermore, the claims appended to the present invention are intended to encompass all changes and modifications that fall within the scope and boundaries of the appended claims, or equivalents of such scope and boundaries.

Claims

1. An obstacle recognition method, comprising: collecting a laser image of the target area; extracting candidate laser light bars in the laser image; obtaining a theoretical position of a reference laser line in the laser image; sorting valid laser light bars from the candidate laser light bars based on the theoretical positions; and obtaining a position of an obstacle based on the position of the effective laser light bar in a world coordinate system.

2. Obtaining a theoretical position of a reference laser line in the laser image includes: Obtaining a light plane equation of the laser in a camera coordinate system; Obtaining a reference plane equation in the camera coordinate system; The obstacle recognition method according to claim 1 , further comprising: obtaining an expression of an intersection line between the light plane equation and the reference plane equation to obtain the theoretical position.

3. Obtaining the reference plane equation in the camera coordinate system is A transformation matrix T from the camera coordinate system to the world coordinate system ｗｃ Transforming coordinates of a plurality of points on the reference plane in the world coordinate system into coordinates in the camera coordinate system based on the and establishing a reference plane equation in a camera coordinate system based on the transformed coordinates of the plurality of points. 【Request Item 4】 【Number】 [Equation 1] The obstacle recognition method according to claim 3 . 【Request Item 5】 【Number】 [Equation 2] The obstacle recognition method according to claim 2 .

6. Collecting a laser image of a target area includes: Emitting a horizontal line laser beam to a target area, the included angle with respect to a reference plane being greater than 0° and smaller than 90°; and acquiring a laser image of the target area.

7. further comprising acquiring a background image of the target area; After acquiring the laser image of the target area, further The method of claim 6 , further comprising performing background subtraction on the laser image based on the background image.

8. The obstacle recognition method is used in a mobile robot, before performing background subtraction on the laser image based on the background image, Obtaining motion information of a mobile robot; The obstacle recognition method according to claim 7, further comprising: performing motion compensation on the background image based on the motion information.

9. Extracting candidate laser light bars in the laser image includes:

2. The obstacle recognition method according to claim 1, further comprising extracting, from the laser image, pixel regions having pixel gradation values ​​greater than a preset gradation value as the candidate laser light bars.

10. Sorting valid laser light bars from the candidate laser light bars based on the theoretical positions includes: obtaining a distance between each of the candidate laser light bars and the theoretical position; 2. The obstacle recognition method according to claim 1, further comprising determining the candidate laser light bar with the smallest distance as the valid laser light bar.

11. Obtaining a position of an obstacle based on a position of the effective laser light bar in a world coordinate system includes: The method for recognizing an obstacle according to claim 1 , further comprising obtaining a position of a center pixel point of the effective laser light bar in a world coordinate system to obtain the position of the obstacle.

12. the obstacle is a wall, The obstacle recognition method further comprises: Acquiring three-dimensional point cloud data of the wall body in a first coordinate system, which is a coordinate system established by the robot at its current position with the robot as the origin; fusing the 3D point cloud data into a robot-centered grid map to identify wall grid points in the grid map; fitting wall grid points corresponding to the wall in the grid map to obtain a wall contour; 2. The obstacle recognition method according to claim 1, further comprising: calculating a distance between the robot and the wall based on the wall contour.

13. Acquiring three-dimensional point cloud data of the wall body in a first coordinate system includes: Acquiring light bar information projected onto the wall via a line laser, which is collected at the current position by a robot; The obstacle recognition method according to claim 12, further comprising: identifying three-dimensional point cloud data of the wall in a first coordinate system based on the light bar information.

14. Acquiring light bar information projected onto the wall via a line laser, the light bar information being collected at a current position by a robot, Acquiring a wall image collected by the robot at a current position as a first image while the line laser is turned on; Acquiring, as a second image, a wall image collected by the robot at the current position while the line laser is in an off state; 14. The obstacle recognition method according to claim 13, further comprising: performing a differential process on the first image and the second image to obtain light bar information projected onto the wall via a line laser.

15. Identifying three-dimensional point cloud data of the wall body in a first coordinate system based on the light bar information includes: Identifying wall point cloud coordinates in the camera coordinate system of the light bar pixel center in the light bar information based on a mapping relationship from the camera coordinate system to the pixel coordinate system; and identifying three-dimensional point cloud data of the wall in the first coordinate system based on the wall point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system.

16. The acquisition of the three-dimensional point cloud data of the wall body in the first coordinate system further includes: Obtaining position change information indicating that the robot has moved from a previous position to a current position; Acquiring three-dimensional point cloud data of the wall body in a second coordinate system, which is a coordinate system established by the robot at the previous position with the robot as the origin; 14. The obstacle recognition method according to claim 13, further comprising: converting three-dimensional point cloud data of the wall in the second coordinate system into three-dimensional point cloud data in the first coordinate system based on the position change information.

17. fitting wall grid points corresponding to the wall in the grid map to obtain a wall contour line, Identifying noise grid points from the wall grid points of the grid map, and identifying wall grid points other than the noise grid points as target grid points; 13. The method of claim 12, further comprising fitting the target grid points in the grid map to obtain a wall contour.

18. Identifying noise grid points from wall grid points of the grid map includes: traversing each wall grid point in the grid map and identifying a probability value for the wall grid point that indicates a confidence that the wall grid point can be used to reflect a wall; if the probability value is equal to or less than a probability threshold, identifying the wall grid point as a first noise grid point, and identifying wall grid points other than the first noise grid point as candidate grid points; traversing each candidate grid point in the grid map and determining the number of candidate grid points in a predetermined grid region in which the candidate grid point is located; identifying the candidate grid points as second noise grid points if the number of the candidate grid points is equal to or less than a number threshold; and identifying the first noise grid point and the second noise grid point as the noise grid points.

19. An obstacle recognition device, an acquisition module for acquiring a laser image of the target area; an extraction module for extracting candidate laser light bars in the laser image; an acquisition module for acquiring a theoretical position of a reference laser line in the laser image; a sorting module for sorting valid laser light bars from the candidate laser light bars based on the theoretical positions; a transformation module for obtaining a position of an obstacle based on the position of the effective laser light bar in a world coordinate system.

20. A mobile robot, a robot body, a horizontal laser module, an inertial measurement unit, and a control module; The horizontal laser module is installed on the robot body, and the horizontal laser module includes an infrared laser emitter and a camera, and the horizontal laser module collects a laser image of a target area and transmits it to the control module; the inertial measurement unit is provided in the robot body, acquires the posture of the mobile robot in a world coordinate system, and transmits the acquired posture to the control module; A mobile robot, wherein the control module executes the obstacle recognition method according to any one of claims 1 to 18.

21. An electronic device, a processor; a memory in which a program is stored, The electronic device, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the obstacle recognition method according to any one of claims 1 to 18.

22. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: A non-transitory computer-readable storage medium in which the computer instructions are used to cause the computer to perform the obstacle recognition method according to any one of claims 1 to 18.

Images