A wall line detection method, device, electronic equipment, storage medium and robot

Abstract

A wall line detection method and device, electronic equipment, storage medium and robot are disclosed. The method comprises: extracting contour points of a wall line based on multiple frames of observation data collected by a first sensor installed on the robot; extracting static contour points based on the contour points; performing line segment segmentation fitting based on the static contour points to obtain a wall line contour approximation curve; performing wall line tracking matching based on the wall line contour approximation curve to obtain a first wall line; and determining a final wall line according to the first wall line and a second wall line detected by a second sensor installed on the robot for different problem scenarios of a wall surface. The above method can obtain a more accurate wall line and save labor operation amount by real-time wall line detection through multiple sensors installed on the robot; and zero edge sticking can also be achieved for various complex wall surfaces.

Description

Technical Field

[0001] This invention relates to the field of robot detection technology, and more particularly to a wall line detection method, device, electronic device, storage medium, and robot. Background Technology

[0002] During cleaning operations, robots can autonomously perform functions such as mapping and localization, obstacle detection, and path planning, achieving automated operation. However, in normal operation, robots cannot walk close to walls, leaving cleaning blind spots, which necessitates manual re-cleaning after the robot's work.

[0003] To address the issue of manual rework, a teaching path can be pre-deployed on the wall surface. During robot operation, the robot can achieve zero-edge contact based on the teaching path. However, deploying the teaching path increases manual deployment work, and the accuracy of the teaching path drawing can also affect the edge-contact performance. Summary of the Invention

[0004] This invention provides a wall line detection method, device, electronic device, storage medium, and robot to solve the problem that in the prior art, achieving zero-edge wall line detection based on a teaching path requires a lot of manpower and has poor edge-fitting effect.

[0005] According to one aspect of the present invention, a wall line detection method is provided, comprising:

[0006] The contour points of the wall line are extracted based on multi-frame observation data collected by the first sensor installed on the robot.

[0007] Extract static contour points based on the aforementioned contour points;

[0008] Based on the static contour points, a line segmentation fitting is performed to obtain an approximate curve of the wall line contour.

[0009] The first wall line is obtained by wall line tracking and matching based on the approximate curve of the wall line contour.

[0010] For different problem scenarios on the wall, the final wall line is determined based on the first wall line and the second wall line detected by the second sensor installed on the robot;

[0011] The first sensor and the second sensor are sensors installed at different locations on the robot, and the detection process of the second wall line is the same as that of the first wall line.

[0012] Optionally, the extraction of the wall line contour points based on multi-frame observation data collected by the first sensor installed on the robot includes:

[0013] The multi-frame observation data collected by the first sensor installed on the robot is stored as historical multi-frame observation data;

[0014] The historical multi-frame observation data is accumulated to obtain multi-frame accumulated point cloud data;

[0015] The strategy of extracting the nearest point using equal angular resolution is used to extract the contour points of the wall line from the accumulated point cloud data of the multiple frames.

[0016] To address the issues of point cloud disappearance and detection range, this embodiment can obtain more complete wall contour data by accumulating historical observation data from multiple frames.

[0017] Optionally, the step of extracting static contour points based on the contour points includes:

[0018] Map the contour points onto a static map;

[0019] For each of the contour points, if a contour point is mapped to a grid in the static map where the corresponding grid is occupied, then the contour point is determined to be a static contour point.

[0020] To address the issue that moving objects in the detection scenario can affect the extraction of wall lines, this embodiment removes dynamic contour points from the contour points by extracting static contour points.

[0021] Optionally, for temporary static objects not included in the static image, each contour point of the temporary static object is observed and counted using historical multi-frame observation data to determine whether the contour point is a static contour point.

[0022] This embodiment targets temporary static objects in the scene. By observing and counting, the static outline points of the temporary static objects can be determined, and then the static outline points determined by loading a static image can be supplemented.

[0023] Optionally, the step of observing and counting each contour point of the temporary static object using historical multi-frame observation data to determine whether the contour point is a static contour point includes:

[0024] For each contour point of the temporary static object, obtain the current frame observation data of one contour point;

[0025] The observation count of a contour point is determined based on the current frame observation data and historical multi-frame observation data.

[0026] Whether a contour point is a static contour point is determined based on the observed count and the count threshold.

[0027] This embodiment can effectively determine whether a contour point is a static contour point by observing and counting each contour point using historical multi-frame observation data.

[0028] Optionally, the step of obtaining an approximate curve of the wall line contour by segmenting and fitting the line segments based on the static contour points includes: recursively segmenting the static contour points into multiple line segments using a straight line feature extraction algorithm.

[0029] Connecting the dividing points of each line segment in sequence forms a broken line;

[0030] The broken line is used as an approximate curve for the wall profile.

[0031] To address the issue of sensor ranging fluctuations, this embodiment uses a line segmentation fitting method to replace the original ranging points with the approximate curve of the wall profile obtained from the fitting method, which can effectively eliminate the influence of ranging fluctuations.

[0032] Optionally, the approximate curve of the wall line profile is the wall line detection result at the current observation time. Correspondingly, the step of obtaining the first wall line by wall line tracking and matching based on the approximate curve of the wall line profile includes:

[0033] For each straight line segment in the approximate curve of the wall line profile, a straight line segment is matched with each tracking unit in the tracking unit set; wherein, the tracking unit set includes wall line detection results at different observation times;

[0034] If a line segment satisfies the matching condition, the line segment is stored in the line segment set of the target tracking unit, and the target tracking unit is the tracking unit that matches the line segment.

[0035] If a line segment does not meet the matching condition, then the line segment is constructed as a new tracking unit and added to the tracking unit set;

[0036] The first wall line is obtained by filtering the wall line detection results at the current observation time included in the updated tracking unit set.

[0037] To address the potential issues of incomplete wall recognition and wall position detection errors in single-frame wall lines at the current observation time, this embodiment employs wall line tracking and matching to obtain more stable detection results for each wall line. Optionally, for different wall surface problem scenarios, the final wall line is determined based on the first wall line and the second wall line detected by the second sensor mounted on the robot, including:

[0038] For scenarios where there is a cutout at the bottom of the wall, the first wall line is used as the final wall line;

[0039] For scenarios where there is a protrusion at the bottom of the wall, the second wall line detected by the second sensor installed on the robot is taken as the final wall line.

[0040] To address irregular wall surfaces such as protrusions or cutouts, this embodiment uses the fusion of a first sensor and a second sensor to extract the optimal wall line.

[0041] According to another aspect of the present invention, a wall line detection device is provided, comprising:

[0042] The first extraction module is used to extract the contour points of the wall line based on multi-frame observation data collected by the first sensor installed on the robot.

[0043] The second extraction module is used to extract static contour points based on the contour points;

[0044] The segmentation and fitting module is used to perform line segmentation and fitting based on the static contour points to obtain an approximate curve of the wall line contour.

[0045] The wall line tracking and matching module is used to perform wall line tracking and matching based on the approximate curve of the wall line contour to obtain the first wall line;

[0046] The determination module is used to determine the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot, for different problem scenarios of the wall.

[0047] The first sensor and the second sensor are sensors installed at different locations on the robot, and the detection process of the second wall line is the same as that of the first wall line.

[0048] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and

[0049] A memory communicatively connected to the at least one processor; wherein,

[0050] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the wall detection method according to any embodiment of the present invention.

[0051] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the wall wire detection method according to any embodiment of the present invention.

[0052] The technical solution of this invention extracts the contour points of a wall line based on multi-frame observation data collected by a first sensor installed on a robot; extracts static contour points based on the contour points; performs line segmentation fitting based on the static contour points to obtain an approximate curve of the wall line contour; performs wall line tracking and matching based on the approximate curve of the wall line contour to obtain a first wall line; and determines the final wall line based on the first wall line and the second wall line detected by a second sensor installed on the robot for different problem scenarios of the wall surface. The first and second sensors are sensors installed at different positions on the robot, and the detection process of the second wall line is the same as that of the first wall line. This solves the problem that existing robots cannot achieve precise zero-edge contact when operating along wall lines, and achieves the beneficial effect of zero-edge contact for robots on various complex walls.

[0053] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 These are schematic diagrams illustrating different wall edge scenarios provided in embodiments of the present invention;

[0056] Figure 2 This is a schematic flowchart of a wall line detection method provided in an embodiment of the present invention;

[0057] Figure 3 A flowchart of another wall detection method provided in an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of multi-frame observation data storage in another wall line detection method provided in an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of contour point extraction in another wall line detection method provided in an embodiment of the present invention;

[0060] Figure 6 A flowchart of another wall detection method provided in an embodiment of the present invention;

[0061] Figure 7This invention provides a schematic diagram of static contour point extraction in another wall line detection method.

[0062] Figure 8 A schematic diagram of line segmentation fitting in another wall line detection method provided in an embodiment of the present invention;

[0063] Figure 9 A flowchart of another wall inspection method provided in an embodiment of the present invention;

[0064] Figure 10 This is a schematic diagram of wall line tracking and matching in another wall line detection method provided in an embodiment of the present invention;

[0065] Figure 11 This is a schematic diagram of wall line fusion in another wall line detection method provided in an embodiment of the present invention;

[0066] Figure 12 This is a schematic diagram of the structure of a wall line detection device provided in an embodiment of the present invention;

[0067] Figure 13 This is a schematic diagram of the electronic device used in the wall wire detection method according to an embodiment of the present invention;

[0068] Figure 14 This is a schematic diagram of the structure of a robot provided in an embodiment of the present invention. Detailed Implementation

[0069] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention. It should be understood that the various steps described in the method embodiments of the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0070] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0071] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0072] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0073] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0074] Figure 1 This is a schematic diagram illustrating different wall edge scenarios provided in embodiments of the present invention, such as... Figure 1 As shown, 1 represents a schematic diagram of a robot operating along the wall line with zero edge contact in a straight wall scenario; 2 represents a schematic diagram of a robot operating along the wall line with zero edge contact in a straight wall scenario with a raised bottom; 3 represents a schematic diagram of a robot operating along the wall line with zero edge contact in a straight wall scenario with a hollow bottom; 4 represents a schematic diagram of a robot operating along the wall line with zero edge contact in a positive corner scenario; and 5 represents a schematic diagram of a robot operating along the wall line with zero edge contact in a negative corner scenario.

[0075] When the robot is operating with zero-edge contact, it needs to detect wall lines very accurately, including detecting the presence of multiple wall lines in the current scene and whether the position of each wall line is accurate. However, problems with the data observed by the sensors mounted on the robot can significantly affect wall line detection, as shown in Table 1, which is a table of sensor data problems.

[0076] Problem Categories Problems Distance fluctuation Distance measurement deviation or fluctuation, such as a fluctuation of approximately 0.02m. Point clouds disappear When applying the material to the edge, due to the angle of incidence or material properties, there may be no return point when fixing the angle. Detection range There is a return point only within the sensor's detection range. Sensor differences For the same wall, the distance and angle of the point cloud returned by each sensor may vary. Installation location The sensor above can only detect wall lines of the same height, and cannot detect protrusions at the bottom.

[0077] Table 1

[0078] The wall line detection method provided in this embodiment of the invention can effectively solve the problems existing in Table 1.

[0079] Figure 2This is a flowchart illustrating a wall line detection method provided in an embodiment of the present invention. This method is applicable to situations where a robot achieves zero contact with wall lines during cleaning operations. The method can be executed by a wall line detection device, which can be implemented by software and / or hardware and is generally integrated into an electronic device. In this embodiment, the electronic device includes, but is not limited to, the robot's controller.

[0080] like Figure 2 As shown, a wall line detection method may include the following steps:

[0081] S110. Extract the contour points of the wall line based on the multi-frame observation data collected by the first sensor installed on the robot.

[0082] The robot can be any intelligent robot with any function, such as a cleaning robot. The first sensor can be a sensor installed on the upper part of the robot's body. There are no restrictions on the number and type of sensors. The first sensor can detect wall lines of the same height, but cannot detect protrusions at the bottom of the wall.

[0083] In this context, multi-frame observation data can be understood as multiple frames of data collected by the first sensor within an observation period. These multi-frame observation data need to be stored to form the longest possible raw point cloud data set, from which complete wall information can be extracted.

[0084] In this embodiment, contour points, i.e., the contour point cloud of the wall line, are extracted from the accumulated multi-frame observation data. The method of extracting contour points is not limited here.

[0085] S120. Extract static contour points based on the contour points.

[0086] It is understandable that moving objects in the robot's working environment can affect wall line detection. Therefore, when performing wall line detection, the point clouds of dynamic objects can be removed, and wall line detection can be performed only on the point clouds of static objects.

[0087] Among them, static contour points can be understood as the contour points of static objects, which can include walls and immovable objects.

[0088] In this embodiment, the point clouds of dynamic objects in the point cloud data collected by the first sensor are removed, and wall line detection is performed only on the point clouds of static objects. The contour points of static objects, i.e., static contour points, can be extracted through static maps and static counting.

[0089] Specifically, by loading a static map, prior information about static objects can be obtained. This can be understood as follows: when loading a static map, static objects in the current scene are scanned into the static map, while moving objects are not. After the static map is established, if there are temporarily placed long, straight objects in the current scene, these temporarily existing long, straight objects are not scanned into the static map. In this case, new static contour points can be determined through static counting. The static counting method involves observing and counting a point in the current frame using stored multi-frame observation data. When the count value meets certain conditions, that point can be set as a static contour point.

[0090] S130. Based on the static contour points, perform line segmentation fitting to obtain an approximate curve of the wall contour.

[0091] In this embodiment, the point cloud data obtained by the first sensor scanning on the real wall can be fitted into a long straight line segment, while the point cloud data scanned by the first sensor on other static objects exhibits irregular shapes. The straightness can distinguish the real wall from other static objects. Therefore, it is necessary to perform straight line fitting on the static contour points, recursively dividing the static contour points into multiple curves; then, using a single curve as an evaluation element, different processing is performed according to the geometric characteristics of the curve; after all curves have been processed, the segmentation points are connected sequentially to form a broken line, which can serve as an approximate curve for the wall contour.

[0092] S140. Based on the approximate curve of the wall line contour, wall line tracking and matching are performed to obtain the first wall line.

[0093] Among them, the wall line contour curve can be understood as the single-frame wall line detection result at the current observation time, and the first wall line can be understood as the wall line detected by the first sensor.

[0094] Understandably, on the one hand, because the robot moves continuously during operation, the position scanned by the first sensor for each wall will be different at different observation times; on the other hand, the first sensor experiences ranging fluctuations, so the point scanned at the same location will also differ at different observation times. Based on these reasons, the wall contour curve obtained through S130 may have issues with incomplete wall recognition and wall position detection errors.

[0095] In this embodiment, the detection results of each wall line at different observation times are matched and the detection results are filtered to obtain more stable detection results for each wall line.

[0096] Specifically, taking a wall line as an example, it is necessary to match the detection results of the wall line at different observation times—that is, different straight line segments in the wall line contour curve—with the straight line segments in the set one by one. The set includes the wall line detection results at different observation times. Taking the detection result of the wall line at the current observation time as an example, a straight line segment in the wall line contour curve is matched with the straight line segments in the set one by one, and the set is updated according to the matching results. Based on the updated set, the wall line detected at the current observation time is determined, which is the first wall line.

[0097] S150. For different problem scenarios on the wall, determine the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot.

[0098] The second sensor can be understood as a sensor installed on the lower part of the robot's body; the type and number of the second sensor are not specifically limited here. The detection process for the second wall line is the same as that for the first wall line, and will not be described in detail here.

[0099] In this embodiment, in order to solve the problem of irregular wall surfaces such as protrusions or hollows, a multi-sensor fusion algorithm can be used to extract the optimal wall line as the final wall line.

[0100] Specifically, when there are protrusions or openings at the bottom of the wall, the first sensor can detect the wall surface, and the second sensor can detect the protrusions or openings.

[0101] The wall line detection method provided in the above embodiments of the present invention first extracts the contour points of the wall line based on multi-frame observation data collected by a first sensor installed on a robot; secondly, it extracts static contour points based on the contour points; then, it performs line segmentation fitting based on the static contour points to obtain an approximate curve of the wall line contour; subsequently, it performs wall line tracking and matching based on the approximate curve of the wall line contour to obtain a first wall line; finally, for different problem scenarios of the wall surface, it determines the final wall line based on the first wall line and the second wall line detected by a second sensor installed on the robot; wherein, the first sensor and the second sensor are sensors installed at different positions on the robot, and the detection process of the second wall line is the same as the detection process of the first wall line. The above method performs wall line detection in real time by multiple sensors installed on the robot, which can obtain more accurate wall lines, enabling the robot to achieve zero-edge contact with the wall line. This method saves manual labor and can effectively solve the problem of poor zero-edge contact effect caused by the robot using a manually deployed teaching path for wall line contact in the prior art; with this method, the robot can also achieve zero-edge contact with various complex walls.

[0102] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0103] Figure 3 This is a flowchart of another wall detection method provided by an embodiment of the present invention. This embodiment is an optimization based on the above embodiment. (Refer to...) Figure 3 The wall inspection method may include the following steps:

[0104] S210. Store the multi-frame observation data collected by the first sensor installed on the robot as historical multi-frame observation data.

[0105] To address the issues of point cloud disappearance and detection range, storing multiple frames of observation data collected at different times can yield more complete wall contour data. Point cloud disappearance can be understood as the situation where, when adhering to an edge, a fixed angle may not have a return point due to the incident angle or wall material; the detection range issue can be understood as the return point only existing within the sensor's detection range.

[0106] In this embodiment, the observation data collected by the first sensor is stored and accumulated. The observation data can be stored in a queue container of fixed size. If the storage capacity of the queue container reaches the upper limit, the first stored observation data can be discarded and new data can be stored. Figure 4 This is a schematic diagram of multi-frame observation data storage in another wall line detection method provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the queue container can store a maximum of 6 data items. When storing new data 7 into the queue container, data 1 needs to be discarded before data 7 can be stored into the queue. At this time, the queue container includes 6 data items from data 2 to data 7.

[0107] S220. Accumulate the historical multi-frame observation data to obtain multi-frame accumulated point cloud data; extract the outline points of the wall line from the multi-frame accumulated point cloud data using a strategy of extracting the nearest point with equal angular resolution.

[0108] In this embodiment, accumulating the multi-frame observation data stored in the queue container yields multi-frame accumulated point cloud data. Extracting contour points from this multi-frame accumulated point cloud data yields the contour point cloud of the wall line. During contour point extraction, the nearest point data is extracted at equal angular resolution as the contour points of the wall line. Figure 5 This is a schematic diagram of contour point extraction in another wall line detection method provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the nearest point is extracted as the contour point from the accumulated point cloud data of multiple frames.

[0109] S230. Extract static contour points based on the contour points.

[0110] S240. Based on the static contour points, perform line segmentation fitting to obtain an approximate curve of the wall contour.

[0111] S250. Based on the approximate curve of the wall line profile, wall line tracking and matching are performed to obtain the first wall line.

[0112] S260. For different problem scenarios on the wall, determine the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot.

[0113] The detection process for the second wall line is the same as that for the first wall line, and will not be described in detail here.

[0114] Another wall detection method provided in the above embodiments addresses the issues of point cloud disappearance and detection range by accumulating historical multi-frame observation data to obtain more complete wall contour data.

[0115] Figure 6 This is a flowchart of another wall detection method provided by an embodiment of the present invention. This embodiment is an optimization based on the above embodiment. (Refer to...) Figure 6 The wall inspection method may include the following steps:

[0116] S310. Extract the contour points of the wall line based on the multi-frame observation data collected by the first sensor installed on the robot.

[0117] S320. Map the contour points to a static map; for each contour point, if the corresponding grid in the static map is occupied, then the contour point is determined to be a static contour point.

[0118] In this embodiment, the method for determining whether each contour point is a static contour point is the same; here, we take one contour point as an example for explanation. The contour point is mapped to the static map. If the grid cell containing the contour point in the static map is occupied, then the contour point can be set as a static contour point.

[0119] Figure 7 This invention provides a schematic diagram of static contour point extraction in another wall line detection method, as shown in the embodiment of the invention. Figure 7 As shown, after mapping the contour points onto the static map, the contour points in the occupied grid are static contour points, and the contour points in the free grid are dynamic contour points.

[0120] S330. For temporary static objects not included in the static image, each contour point of the temporary static object is observed and counted using historical multi-frame observation data to determine whether the contour point is a static contour point.

[0121] Among them, the historical multi-frame observation data is the historical multi-frame observation data in S210, and each contour point of the temporary static object can be collected by the first sensor.

[0122] In this embodiment, the temporary static object may include multiple contour points. The method for determining whether each contour point is a static contour point is the same; here, we will use a single contour point as an example. The steps include:

[0123] S3101. Obtain the current frame observation data of a contour point.

[0124] Specifically, the current frame observation data of a contour point can be obtained from the observation data of all contour points of a temporary static object collected by the first sensor.

[0125] S3102. Determine the observation count of a contour point based on the current frame observation data and historical multi-frame observation data.

[0126] Specifically, it can be determined whether the current frame observation data has appeared in the historical multi-frame observation data, and the observation count is determined based on the number of times it appears. Each time it appears, the count is incremented by 1. For example, if the current frame observation data has appeared 3 times in the historical multi-frame observation data, then the observation count of the contour point is 3.

[0127] S3103. Determine whether a contour point is a static contour point based on the observed count and the count threshold.

[0128] The counting threshold can be a pre-set value.

[0129] Specifically, if the observation count of a contour point reaches the counting threshold, the contour point can be determined as a static contour point; if the observation count of a contour point does not reach the counting threshold, the contour point can be determined as a dynamic contour point.

[0130] S340. Based on the static contour points, perform line segmentation fitting to obtain an approximate curve of the wall contour.

[0131] Specifically, a feasible way to determine the approximate curve of the wall outline is as follows: recursively divide the static outline points into multiple line segments using a straight line feature extraction algorithm; connect the dividing points of each line segment in sequence to form a polyline; and use the polyline as the approximate curve of the wall outline.

[0132] For example, Figure 8 This is a schematic diagram of line segmentation and fitting in another wall line detection method provided by an embodiment of the present invention, as shown below. Figure 8As shown, the contour curve can be obtained by connecting the static contour points in sequence. The following steps are performed: Step a: Connect the first two points A and B of the contour curve to form a straight line AB; Step b: Calculate the point C on the contour curve that is the furthest from the straight line AB, and calculate the distance d from point C to the straight line AB; Step c: Compare the distance d with a pre-given threshold. If the distance d is less than the threshold, the straight line AB is used as the approximation of the contour curve, i.e., the approximate curve of the wall contour, and the processing is completed; Step d: If the distance d is greater than the threshold, the contour curve is divided into curve AC and curve BC by point C, and steps a to c are performed on curve AC and curve BC respectively; After all curves have been processed, the broken lines formed by connecting each dividing point in sequence are used to obtain the approximate curve of the wall contour.

[0133] S350. Based on the approximate curve of the wall line profile, wall line tracking and matching are performed to obtain the first wall line.

[0134] S360. For different problem scenarios on the wall, determine the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot.

[0135] The detection process for the second wall line is the same as that for the first wall line, and will not be described in detail here.

[0136] The above embodiments provide another wall line detection method that addresses the problem that moving objects in the detection scene can affect wall line extraction. By extracting static contour points to remove dynamic contour points, and by observing and counting, the static contour points of temporary static objects can be determined, which can then supplement the static contour points determined by loading static images. To address the problem of sensor ranging fluctuations, a line segmentation fitting method is used to replace the original ranging points with the obtained approximate curve of the wall line contour, which can effectively eliminate the influence of ranging fluctuations.

[0137] Figure 9 This is a flowchart of another wall detection method provided in an embodiment of the present invention. This embodiment is an optimization based on the above embodiment. (Refer to...) Figure 9 The wall inspection method may include the following steps:

[0138] S410: Extract the contour points of the wall line based on multi-frame observation data collected by the first sensor installed on the robot.

[0139] S420. Extract static contour points based on the contour points.

[0140] S430. Based on the static contour points, perform line segmentation fitting to obtain an approximate curve of the wall contour.

[0141] S440. Based on the approximate curve of the wall line contour, wall line tracking and matching are performed to obtain the first wall line.

[0142] Among them, the approximate curve of the wall line profile can be used as the wall line detection result at the current observation time.

[0143] In this embodiment, the approximate curve of the wall outline includes multiple straight line segments. Taking the tracking and matching process of one straight line segment as an example, the process includes the following steps:

[0144] S4401. Match a line segment with each tracking unit in the tracking unit set.

[0145] The tracking unit set includes wall line detection results at different observation times.

[0146] S4402. If a line segment satisfies the matching condition, the line segment is stored in the line segment set of the target tracking unit, where the target tracking unit is the tracking unit that matches the line segment.

[0147] S4403. If a line segment does not meet the matching condition, then the line segment is constructed as a new tracking unit and added to the tracking unit set.

[0148] S4404. Filter the wall line detection results at the current observation time included in the updated tracking unit set to obtain the first wall line.

[0149] For example, each straight line segment Li (i = 1 to m) in the approximate curve of the wall line contour is matched with each tracking unit Tj (j = 1 to n) in the tracking unit set. If a straight line segment matches a tracking unit Tk, the straight line segment can be stored in the straight line segment set of tracking unit Tk and the set can be updated. If the straight line segment does not have a matching tracking unit, a new tracking unit Tn+1 can be constructed using the straight line segment.

[0150] Figure 10 This is a schematic diagram of wall line tracking and matching in another wall line detection method provided in an embodiment of the present invention, as shown below. Figure 10 As shown, the specific matching method can be as follows: if two straight line segments are collinear and have a low degree of overlap, then the two straight line segments are determined to be mismatched, i.e., different wall lines; if two straight line segments are collinear and have a high degree of overlap, then the two straight line segments are determined to be matched, i.e., the same wall line.

[0151] S450. For scenarios where there is a hollowed-out area at the bottom of the wall, the first wall line is taken as the final wall line; for scenarios where there is a protrusion at the bottom of the wall, the second wall line detected by the second sensor installed on the robot is taken as the final wall line.

[0152] Figure 11 This is a schematic diagram of wall line fusion in another wall line detection method provided in an embodiment of the present invention, as shown below. Figure 11 As shown, in a scenario with a protruding wall, if the robot operates along the first wall line detected by the first sensor without touching the edge, it will scrape the bottom protrusion of the wall. In this case, the second wall line detected by the second sensor needs to be used as the final wall line, and the robot can operate along the second wall line without touching the edge. In a scenario with a hollow bottom, the first wall line detected by the first sensor can be used as the final wall line, and the robot can operate along the first wall line without touching the edge.

[0153] The above embodiment provides another wall line detection method. In view of the problem that the wall line in a single frame at the current observation time may have incomplete wall recognition and wall position detection error, the wall line tracking and matching can obtain a more stable detection result for each wall line. In view of the problem of irregular wall surfaces such as protrusions or hollows, the optimal wall line can be extracted by fusing the first sensor and the second sensor.

[0154] Figure 12 This is a schematic diagram of a wall line detection device provided in an embodiment of the present invention. This device is applicable to situations where a robot achieves zero-edge contact with wall lines during cleaning operations. This device can perform the wall line detection method described in any of the above embodiments. (Refer to...) Figure 12 The device includes: a first extraction module 110, a second extraction module 120, a wall line tracking and matching module 130, and a determination module 140.

[0155] The first extraction module 110 is used to extract the contour points of the wall line based on multi-frame observation data collected by the first sensor installed on the robot.

[0156] The second extraction module 120 is used to extract static contour points based on the contour points;

[0157] The segmentation and fitting module 130 is used to perform line segmentation and fitting based on the static contour points to obtain an approximate curve of the wall line contour.

[0158] The wall line tracking and matching module 140 is used to perform wall line tracking and matching based on the approximate curve of the wall line contour to obtain a first wall line;

[0159] The determination module 150 is used to determine the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot, for different problem scenarios of the wall.

[0160] The first sensor and the second sensor are sensors installed at different locations on the robot, and the detection process of the second wall line is the same as that of the first wall line.

[0161] In this embodiment, the device first extracts the contour points of the wall line based on multi-frame observation data collected by the first sensor installed on the robot through the first extraction module 110; then, it extracts static contour points based on the contour points through the second extraction module 120; then, it performs line segmentation fitting based on the static contour points through the segmentation fitting module 130 to obtain an approximate curve of the wall line contour; then, it performs wall line tracking and matching based on the approximate curve of the wall line contour to obtain the first wall line; finally, it determines the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot for different problem scenarios of the wall surface through the determination module 150; wherein, the first sensor and the second sensor are sensors installed at different positions on the robot, and the detection process of the second wall line is the same as the detection process of the first wall line.

[0162] This embodiment provides a wall line detection device that can detect wall lines in real time through sensors, obtain more accurate wall lines, and save manual labor; it can also achieve zero-edge contact for various complex wall surfaces.

[0163] Furthermore, the first extraction module 110 includes:

[0164] The storage unit is used to store the multi-frame observation data collected by the first sensor installed on the robot as historical multi-frame observation data.

[0165] The accumulation unit is used to accumulate the historical multi-frame observation data to obtain multi-frame accumulated point cloud data;

[0166] An extraction unit is used to extract the contour points of the wall line from the multi-frame accumulated point cloud data using a strategy of extracting the nearest point at equal angle resolution.

[0167] Furthermore, the second extraction module 120 includes:

[0168] The first determining unit is used to map the contour points to a static map; for each contour point, if the corresponding grid in the static map is occupied, then the contour point is determined to be a static contour point.

[0169] The second determining unit is used to determine whether a contour point is a static contour point by observing and counting each contour point of a temporary static object not included in the static image through historical multi-frame observation data.

[0170] Based on the above optimization, the second determining unit is specifically used to: for each contour point of the temporary static object, acquire the current frame observation data of a contour point; determine the observation count of the contour point based on the current frame observation data and historical multi-frame observation data; and determine whether the contour point is a static contour point based on the observation count and the counting threshold.

[0171] Furthermore, the segmentation fitting module 130 is specifically used to: recursively segment the static contour points into multiple line segments using a straight line feature extraction algorithm; sequentially connect the segmentation points of each line segment to form a polyline; and use the polyline as an approximate curve of the wall contour.

[0172] Furthermore, the approximate curve of the wall line contour is the wall line detection result at the current observation time. Accordingly, the wall line tracking and matching module 140 is specifically used to: match a straight line segment with each tracking unit in the tracking unit set for each straight line segment in the approximate curve of the wall line contour; wherein, the tracking unit set includes wall line detection results at different observation times.

[0173] If a line segment meets the matching condition, the line segment is stored in the line segment set of the target tracking unit, and the target tracking unit is the tracking unit that matches the line segment; if the line segment does not meet the matching condition, the line segment is constructed as a new tracking unit and added to the tracking unit set; the wall line detection results at the current observation time included in the updated tracking unit set are filtered to obtain the first wall line.

[0174] Furthermore, the determining module 150 is specifically used for: in the case of a problem scenario where there is a hollow at the bottom of the wall, taking the first wall line as the final wall line; and in the case of a problem scenario where there is a protrusion at the bottom of the wall, taking the second wall line detected by the second sensor installed on the robot as the final wall line.

[0175] The above-described wall line detection device can execute the wall line detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0176] Figure 13 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of control devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0177] like Figure 13As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0178] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0179] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as wall wire detection methods.

[0180] In some embodiments, the wall wire detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the wall wire detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the wall wire detection method by any other suitable means (e.g., by means of firmware).

[0181] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0182] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0183] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0184] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0185] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0186] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0187] Figure 14 This is a schematic diagram of the structure of a robot provided in an embodiment of the present invention, such as... Figure 14 As shown, the robot is equipped with a first sensor and a second sensor; the robot also includes the electronic equipment described in the embodiments of the present invention. The robot can execute the wall line detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0188] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0189] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting wall lines, characterized in that, The method includes: The contour points of the wall line are extracted based on multi-frame observation data collected by the first sensor installed on the robot. Extract static contour points based on the aforementioned contour points; Based on the static contour points, a line segmentation fitting is performed to obtain an approximate curve of the wall line contour. The first wall line is obtained by wall line tracking and matching based on the approximate curve of the wall line contour. For different problem scenarios on the wall, the final wall line is determined based on the first wall line and the second wall line detected by the second sensor installed on the robot; The first sensor and the second sensor are sensors installed at different locations on the robot. The first sensor is installed on the upper half of the robot body, and the second sensor is installed on the lower half of the robot body. The first sensor is used to detect the wall surface, and the second sensor is used to detect protrusions or cutouts. The detection process of the second wall line is the same as the detection process of the first wall line. The step of extracting static contour points based on the contour points includes: mapping the contour points to a static map; for each contour point, if the corresponding grid in the static map is occupied, then the contour point is determined to be a static contour point. The method for determining the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot for different problem scenarios of the wall includes: for the problem scenario where there is a hole at the bottom of the wall, the first wall line is used as the final wall line; for the problem scenario where there is a protrusion at the bottom of the wall, the second wall line detected by the second sensor installed on the robot is used as the final wall line.

2. The method according to claim 1, characterized in that, The extraction of wall line contour points based on multi-frame observation data collected by the first sensor mounted on the robot includes: The multi-frame observation data collected by the first sensor installed on the robot is stored as historical multi-frame observation data; The historical multi-frame observation data is accumulated to obtain multi-frame accumulated point cloud data; The strategy of extracting the nearest point using equal angular resolution is used to extract the contour points of the wall line from the accumulated point cloud data of the multiple frames.

3. The method according to claim 1, characterized in that, Also includes: For temporary static objects not included in the static map, each contour point of the temporary static object is observed and counted using historical multi-frame observation data to determine whether the contour point is a static contour point.

4. The method according to claim 3, characterized in that, The step of observing and counting each contour point of the temporary static object using historical multi-frame observation data to determine whether the contour point is a static contour point includes: For each contour point of the temporary static object, obtain the current frame observation data of one contour point; The observation count of a contour point is determined based on the current frame observation data and historical multi-frame observation data. Whether a contour point is a static contour point is determined based on the observed count and the count threshold.

5. The method according to claim 1, characterized in that, The process of obtaining an approximate curve of the wall profile by segmenting and fitting line segments based on the static contour points includes: The static contour points are recursively divided into multiple line segments using a straight line feature extraction algorithm; Connecting the dividing points of each line segment in sequence forms a broken line; The broken line is used as an approximate curve for the wall line profile.

6. The method according to claim 1, characterized in that, The approximate wall profile curve is the wall line detection result at the current observation time. Correspondingly, the step of obtaining the first wall line by wall line tracking and matching based on the approximate wall profile curve includes: For each straight line segment in the approximate curve of the wall line profile, a straight line segment is matched with each tracking unit in the tracking unit set; wherein, the tracking unit set includes wall line detection results at different observation times; If a line segment satisfies the matching condition, the line segment is stored in the line segment set of the target tracking unit, and the target tracking unit is the tracking unit that matches the line segment. If a line segment does not meet the matching condition, then the line segment is constructed as a new tracking unit and added to the tracking unit set; The first wall line is obtained by filtering the wall line detection results at the current observation time included in the updated tracking unit set.

7. A wall line detection device, characterized in that, The device includes: The first extraction module is used to extract the contour points of the wall line based on multi-frame observation data collected by the first sensor installed on the robot. The second extraction module is used to extract static contour points based on the contour points; The segmentation and fitting module is used to perform line segmentation and fitting based on the static contour points to obtain an approximate curve of the wall line contour. The wall line tracking and matching module is used to perform wall line tracking and matching based on the approximate curve of the wall line contour to obtain the first wall line; The determination module is used to determine the final wall line based on the first wall line and the second wall line detected by the second sensor installed on the robot, for different problem scenarios of the wall. The first sensor and the second sensor are sensors installed at different locations on the robot. The first sensor is installed on the upper half of the robot body, and the second sensor is installed on the lower half of the robot body. The first sensor is used to detect the wall surface, and the second sensor is used to detect protrusions or cutouts. The detection process of the second wall line is the same as the detection process of the first wall line. The second extraction module includes a first determining unit, used to map the contour points to a static map; for each contour point, if the corresponding grid in the static map is occupied, then the contour point is determined to be a static contour point. The determining module is specifically used for: in the case of a problem where there is a hollow at the bottom of the wall, taking the first wall line as the final wall line; and in the case of a problem where there is a protrusion at the bottom of the wall, taking the second wall line detected by the second sensor installed on the robot as the final wall line.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the wall line detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the wall wire detection method according to any one of claims 1-6.

10. A robot, characterized in that, The robot is equipped with a first sensor and a second sensor; the robot includes the electronic device as described in claim 8.

Images