Obstacle footprint estimation method and device, robot, and storage medium

Abstract

The application discloses an obstacle footprint contour estimation method and device, a robot and a storage medium. The method comprises the following steps: collecting obstacle images through a camera at multiple different positions during the movement of the robot around the obstacle; performing target detection on each obstacle image to obtain an obstacle detection frame; estimating a ground projection area formed by the projection of the obstacle detection frame to the ground according to the internal and external parameter matrices of the camera and the obstacle detection frame; determining an overlapping area of the ground projection areas obtained by the projection of the obstacle at multiple different positions, and determining the footprint contour of the obstacle according to the overlapping area. The application solves the problem of inaccurate or incomplete sensing of the laser sensor in detecting the footprint contour of the obstacle, realizes comprehensive and accurate sensing of the footprint contour of the obstacle, avoids the running jam phenomenon caused by the discontinuity of the trajectory when the robot moves around the obstacle, and improves the intelligence, smoothness and cleaning efficiency of the robot.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, robot, and storage medium for estimating the footprint contour of an obstacle. Background Technology

[0002] With the rapid development of automation and artificial intelligence, robots have been applied in various scenarios. In fields such as floor cleaning, cleaning robots can complete simple and repetitive cleaning tasks through autonomous driving technology, greatly reducing labor costs and automating cleaning work.

[0003] In scenarios where cleaning robots perform floor cleaning tasks, they often need to navigate around obstacles. However, due to the nature of the cleaning robot's work, while avoiding obstacles, it is also necessary to get as close to the obstacles as possible to clean more of the floor. This requires accurately determining the footprint and outline of the obstacles.

[0004] Currently, when using laser sensors on robots to detect the footprint of obstacles, the robots may not be able to perceive the entire footprint of the obstacles due to reasons such as the low reflectivity of the obstacles or the existence of space between the obstacles, such as vehicles and shelves, and the ground. This can lead to discontinuous trajectories when navigating around obstacles, potentially causing operational stuttering and reducing the robot's intelligence and cleaning efficiency. Summary of the Invention

[0005] This invention provides a method, apparatus, robot, and storage medium for estimating the footprint contour of obstacles, in order to solve the problem of inaccurate or incomplete perception of the footprint contour of obstacles by laser sensors. It enables comprehensive and accurate perception of the footprint contour of obstacles, thereby avoiding discontinuous trajectory and running stutters when navigating around obstacles, and improving the intelligence, smoothness, and cleaning efficiency of the robot.

[0006] According to one aspect of the present invention, a method for estimating the footprint contour of an obstacle is provided, comprising:

[0007] As the robot moves around the obstacle, it captures images of the obstacle at multiple different locations using cameras;

[0008] Object detection is performed on each obstacle image to obtain the obstacle detection box;

[0009] Based on the camera's intrinsic and extrinsic parameter matrix and the obstacle detection frame, the ground projection area formed by the obstacle detection frame projected onto the ground is determined; the ground projection area completely includes the actual occupied area of ​​the obstacle.

[0010] Determine the overlapping area of ​​the ground projection regions obtained by projecting the obstacle at multiple different locations, and estimate the footprint outline of the obstacle based on the overlapping area.

[0011] According to another aspect of the present invention, an apparatus for estimating the footprint contour of an obstacle is provided, comprising:

[0012] The image acquisition module is used to acquire images of the obstacle at multiple different positions using cameras as the robot moves around the obstacle;

[0013] The object detection module is used to perform object detection on each obstacle image to obtain the obstacle detection box;

[0014] The projection area determination module is used to determine the ground projection area formed by the obstacle detection frame projected onto the ground based on the intrinsic and extrinsic parameter matrix of the camera and the obstacle detection frame; the ground projection area completely includes the actual occupied area of ​​the obstacle;

[0015] The land area contour estimation module is used to determine the overlapping area of ​​the ground projection area obtained by projecting the obstacle at multiple different locations, and to estimate the land area contour of the obstacle based on the overlapping area.

[0016] According to another aspect of the present invention, a robot is provided, the robot comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the obstacle footprint contour estimation method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the obstacle footprint contour estimation method according to any embodiment of the present invention.

[0021] The technical solution of this invention involves acquiring obstacle images at multiple different positions using cameras while the robot moves around an obstacle; performing target detection on each obstacle image to obtain an obstacle detection box; estimating the ground projection area formed by the obstacle detection box projected onto the ground based on the camera's intrinsic and extrinsic parameter matrices and the obstacle detection box; ensuring that the ground projection area completely encompasses the actual area occupied by the obstacle; determining the overlapping area of ​​the ground projection areas obtained from the obstacle's projection at multiple different positions; and determining the obstacle's footprint contour based on the overlapping area. Determining the obstacle's footprint contour based on the projection of the obstacle images acquired by the cameras onto the ground solves the problem of inaccurate or incomplete perception of obstacle footprint contours by laser sensors, achieving comprehensive and accurate perception of the obstacle's footprint contour. This avoids discontinuous trajectory and operational stuttering when navigating around obstacles, improving the robot's intelligence, smoothness, and cleaning efficiency.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a method for estimating the footprint of an obstacle according to Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart of a method for estimating the footprint of an obstacle according to Embodiment 2 of the present invention;

[0026] Figure 3 This is a schematic diagram illustrating the determination of the ground projection area of ​​an obstacle in an obstacle footprint contour estimation method provided in Embodiment 2 of the present invention.

[0027] Figure 4 This is a schematic diagram of the overlapping area of ​​the ground projection region of an obstacle in the method for estimating the outline of an obstacle's footprint provided in Embodiment 2 of the present invention;

[0028] Figure 5 This is a schematic diagram of the obstacle footprint contour obtained by fitting laser point data in an obstacle footprint contour estimation method provided in Embodiment 2 of the present invention.

[0029] Figure 6This is a schematic diagram of the structure of an obstacle footprint contour estimation device according to Embodiment 3 of the present invention;

[0030] Figure 7 This is a schematic diagram of the structure of a robot that implements the obstacle footprint contour estimation method of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a method for estimating the footprint contour of an obstacle according to Embodiment 1 of the present invention. This embodiment is applicable to situations where a robot estimates the footprint contour of surrounding obstacles during a cleaning task. This method can be executed by an obstacle footprint contour estimation device, which can be implemented in hardware and / or software and can be configured within the robot. Figure 1 As shown, the method includes:

[0035] S110. During the robot's movement around the obstacle, images of the obstacle are captured by cameras at multiple different locations.

[0036] In this embodiment, the robot may include robots that perform ground tasks, such as sweeping robots, floor cleaning robots, and crystallizing robots. The scenarios in which the robot performs ground tasks may be shopping malls, above-ground / underground parking lots, or indoor spaces. Obstacles can be understood as objects that create obstacles during the robot's task execution. During the robot's task execution, obstacles are generally stationary relative to the ground, such as parked vehicles or shelves placed on the ground.

[0037] The robot is equipped with sensors to sense and collect information about obstacles in its surroundings. For example, cameras can capture images of obstacles, typically in RGB format; laser sensors can also be used to capture the outer contours of obstacles.

[0038] Specifically, during the robot's task execution, if it senses obstacles in the surrounding environment, it moves around the obstacles at a preset distance, collecting information about the obstacles at multiple different locations using sensors mounted on the robot. The acquired information includes obstacle images captured by a camera.

[0039] For example, as the robot moves around the obstacle at a preset distance, images of the obstacle are captured by a camera on the robot at preset intervals. The preset distance and preset interval can be determined based on the size and shape of the obstacle; the larger the obstacle, the larger the preset distance; the more complex the shape of the obstacle, the smaller the preset interval. This embodiment of the invention does not impose any limitations on this.

[0040] S120. Perform target detection on each obstacle image to obtain the obstacle detection box.

[0041] Among them, the obstacle detection box is a detection box used to describe the location of the obstacle, and the detection box is generally rectangular.

[0042] Specifically, deep learning methods are used to detect obstacle images acquired at different locations, resulting in obstacle detection boxes at different angles. These boxes include the position coordinates of the obstacle detection boxes (such as the coordinates of the top-left or bottom-right vertex of the detection box in the obstacle image).

[0043] S130. Based on the camera's intrinsic and extrinsic parameter matrix and the obstacle detection frame, determine the ground projection area formed by the obstacle detection frame projected onto the ground. The ground projection area completely includes the actual area occupied by the obstacle.

[0044] The camera's intrinsic and extrinsic parameter matrices include the camera's intrinsic parameter matrix and extrinsic parameter matrix. The ground projection area of ​​an obstacle can be understood as the projection area formed on the ground after the obstacle detection box is projected onto the ground. For the estimated obstacle footprint contour to include the complete obstacle, the ground projection area formed by the obstacle detection box projection must satisfy the condition that the ground projection area completely encompasses the actual footprint of the obstacle.

[0045] For example, one way to ensure that the ground projection area completely includes the actual area occupied by the obstacle can be to estimate the type and size of the obstacle based on the obstacle information collected by the sensors on the robot, determine the maximum projection distance of the obstacle, and project the obstacle detection box according to the maximum projection distance to determine the ground projection area; or one way to estimate the type and size of the obstacle based on the obstacle information collected by the sensors on the robot, determine the maximum projection ratio of the obstacle, and project the obstacle detection box according to the maximum projection ratio.

[0046] Specifically, each obstacle detection box is projected onto the ground to form a ground projection area. Since the obstacle detection box can contain the complete side view of the obstacle captured by the camera, the coordinates of the ground projection area can be determined based on the camera's intrinsic and extrinsic parameter matrices and the coordinates of the obstacle detection box. It can be understood that for obstacle detection boxes corresponding to obstacle images acquired at different locations, a ground projection area for each obstacle can be determined.

[0047] S140. Determine the overlapping area of ​​the ground projection area obtained by projecting the obstacle at multiple different positions, and determine the footprint outline of the obstacle based on the overlapping area.

[0048] The footprint outline of an obstacle can be understood as the outline of the actual area occupied by the obstacle on the ground. In this application, the footprint outline of the obstacle is represented by the outline of the obstacle's projection on the ground.

[0049] Specifically, projecting the obstacle detection frame onto multiple different locations of the same obstacle yields a ground projection area. These ground projection areas completely encompass the actual area occupied by the obstacle. However, each ground projection area also contains a significant portion that does not belong to the actual obstacle area. Therefore, the overlapping area of ​​the ground projection areas obtained from multiple different locations significantly reduces the overall ground projection area. It is understood that the overlapping area still ensures that the actual obstacle area is completely included. Therefore, the obstacle's footprint outline can be determined based on the overlapping area. The obstacle's footprint outline can be used for route planning and obstacle avoidance, etc., and this embodiment of the invention does not impose any limitations on this.

[0050] For example, the way to determine the footprint outline of an obstacle based on the overlapping area is as follows: when the number of ground projection areas is insufficient or the shape of the obstacle is complex, the overlapping area may not accurately represent the actual footprint of the obstacle. In this case, the overlapping area can be combined with information collected by other sensors installed on the robot, such as radar sensors and laser sensors, to determine the footprint outline of the obstacle.

[0051] Optionally, estimating the footprint contour of the obstacle based on the overlapping region includes:

[0052] The outer contour of the overlapping area is defined as the footprint contour of the obstacle.

[0053] Specifically, another way to determine the footprint outline of an obstacle based on the overlapping area is: when the number of ground projection areas is large enough or the shape of the obstacle is relatively simple, the overlapping area formed will infinitely approach the actual footprint area of ​​the obstacle. Therefore, the outline of the overlapping area can be determined as the footprint outline of the obstacle.

[0054] The technical solution of this invention involves acquiring obstacle images at multiple different positions using cameras while the robot moves around an obstacle; performing target detection on each obstacle image to obtain an obstacle detection box; estimating the ground projection area formed by the obstacle detection box projected onto the ground based on the camera's intrinsic and extrinsic parameter matrices and the obstacle detection box; ensuring that the ground projection area completely encompasses the actual area occupied by the obstacle; determining the overlapping area of ​​the ground projection areas obtained from the obstacle's projection at multiple different positions; and determining the obstacle's footprint contour based on the overlapping area. This solves the problem of inaccurate or incomplete perception of the obstacle's footprint contour by laser sensors, achieving comprehensive and accurate perception of the obstacle's footprint contour. This avoids discontinuous trajectory and operational stuttering when navigating around obstacles, improving the robot's intelligence, smoothness, and cleaning efficiency.

[0055] Optionally, after estimating the footprint contour of the obstacle based on the overlapping region, the method further includes:

[0056] Get the robot's current position;

[0057] Based on the footprint and outline of the obstacle and the robot's current position, the robot plans its task execution path to avoid the obstacle.

[0058] Specifically, after determining the outline of the obstacle, the robot can plan its task execution path based on its current position and the outline of the obstacle. While avoiding the obstacle, it can perform the cleaning task along the outline as much as possible to expand the cleaning area.

[0059] Example 2

[0060] Figure 2 This is a flowchart of a method for estimating the footprint of an obstacle according to Embodiment 2 of the present invention. This embodiment further refines the above embodiment. Figure 2 As shown, the method includes:

[0061] S210. During the robot's movement around the obstacle, images of the obstacle are captured by cameras at multiple different locations.

[0062] Optionally, the location of the obstacle image captured by the camera on the robot is such that the obstacle is completely covered within the camera's field of view.

[0063] Specifically, as the robot moves around the obstacle, images of the obstacle are captured from a position where the camera's view can completely cover the obstacle, so that the captured obstacle images can completely contain the obstacle.

[0064] S220. Perform target detection on each obstacle image to obtain the obstacle detection box.

[0065] S230. Based on the camera's intrinsic and extrinsic parameter matrix, determine the projection line segment of the bottom of the obstacle detection box onto the ground.

[0066] Specifically, all pixels of the obstacle detection box are projected onto the ground to form a ground projection area. The coordinates of the ground projection area can be determined based on the camera's intrinsic and extrinsic parameter matrices and the coordinates of the obstacle detection box. However, the pixel positions projected onto the ground on the line segment at the bottom of the obstacle detection box, close to the ground plane, are completely accurate. Therefore, the projection line segment of the bottom of the obstacle detection box onto the ground is determined, and the ground projection area of ​​the obstacle is further expanded based on this projection line segment.

[0067] S240, Obtain the maximum projected distance of the obstacle.

[0068] The maximum projection distance can be understood as the maximum length of the obstacle's projection on the ground.

[0069] Specifically, the maximum projection distance of the obstacle is obtained so that the ground projection area extended based on the maximum projection distance can completely encompass the actual area occupied by the obstacle.

[0070] For example, the maximum projected distance of an obstacle can be obtained by estimating the obstacle's length, width, and height based on information collected by sensors on the robot, or by determining the obstacle type based on the information collected by sensors on the robot and obtaining a pre-configured maximum projected distance based on the obstacle type. For instance, if the obstacle type is identified as a small vehicle, the maximum projected distance corresponding to the vehicle is obtained as 5 meters.

[0071] S250. Determine the ground projection area formed by the projection of the obstacle detection frame onto the ground based on the projection line segment and the maximum projection distance.

[0072] Specifically, the bottom projection line segment of the obstacle detection box on the ground is projected according to the maximum projection distance, so that the projection line segment is expanded into a ground projection area.

[0073] For example, Figure 3This is a schematic diagram illustrating the determination of the ground projection area of ​​an obstacle in a method for estimating the outline of an obstacle's footprint, provided in Embodiment 2 of the present invention. According to steps S210 to S250, the following can be obtained: Figure 3 The ground projection area shown is obtained by performing target detection on obstacle images captured within the camera's field of view (the range covered by the camera lens) to obtain obstacle detection boxes. The obstacle detection boxes are then projected onto the ground along their maximum projection length to obtain the ground projection area of ​​the obstacle. This ground projection area can completely encompass the actual area occupied by the obstacle.

[0074] S260. Determine the overlapping area of ​​the ground projection area obtained by projecting the obstacle at multiple different positions.

[0075] Specifically, while a single ground projection area can contain the actual outline of an obstacle, the proportion of irrelevant regions (regions that do not contain the actual outline) within that area can be very large, leading to highly inaccurate estimations of the obstacle's outline. To obtain a more accurate region containing the obstacle's outline, this invention fuses ground projection areas obtained from camera projections at different locations to gradually reduce irrelevant regions. The specific method is as follows:

[0076] Given that each frame contains a complete outline of the obstacle's footprint, let V be the complete obstacle footprint set, and let P be the ground projection area of ​​any frame. i (i = 0, 1, 2, 3... ∈ N, representing the ground projection area of ​​the same obstacle obtained from different positions), then we have:

[0077] V∈P i i = 0, 1, 2…N;

[0078] The relationship between the intersection of two ground projection regions P0 and P1 and the footprint outline of the obstacle is as follows:

[0079] V∈(P0∩P1);

[0080] The intersection of the two ground projection areas not only includes the footprint outline of the obstacle, but is also smaller than the area of ​​each ground projection area, that is, it reduces the non-footprint outline portion of the projection area.

[0081] Then, we can take the overlapping region (i.e., the intersection) of any number of ground projection regions, and we have:

[0082] V∈(P0∩P1∩…∩P n ).

[0083] Therefore, by superimposing the ground projection areas obtained by the robot at multiple different locations when it moves near the vehicle and taking the intersection, a more accurate obstacle footprint contour can be obtained; and the more ground projection areas there are, the more accurate the estimate of the obstacle footprint contour determined by the obtained intersection area will be.

[0084] For example, Figure 4 This is a schematic diagram illustrating the overlapping region of the ground projection area of ​​an obstacle in an obstacle footprint contour estimation method provided in Embodiment 2 of the present invention. Figure 4 The diagram shows the ground projection areas (first ground projection area and second ground projection area) corresponding to obstacle images captured by the camera at two different locations, as well as the overlapping area formed by the two ground projection areas. It can be seen that the overlapping area completely includes the actual area occupied by the obstacle and significantly reduces the non-obstacle outline area in the ground projection area. It is understandable that the more obstacle images captured by the camera at different locations, the closer the overlapping area formed by the multiple ground projection areas will be to the actual area occupied by the obstacle.

[0085] S270. Acquire laser point data of obstacles. The laser point data is collected by laser sensors during the robot's movement around the obstacles.

[0086] Laser point data refers to the location of points on the surface of an obstacle where the laser emitted by the laser sensor reflects off. This data can be used to reflect the outer contour of the obstacle. For example, for vehicle-type obstacles, the laser beam is affected by parameters such as vehicle size, color, and the robot's distance from the vehicle. It may detect the vehicle's outer contour, chassis, and wheel positions, but it will not hit anything outside the vehicle's outline. A 2D laser sensor is sufficient.

[0087] Specifically, as the robot moves around the obstacle, it collects laser point data of the obstacle using laser sensors installed on the robot.

[0088] S280. Fit the laser point data falling into the overlapping area to obtain the footprint contour of the obstacle.

[0089] Specifically, the laser point data falling into the overlapping area can be considered as data representing the actual area occupied by the obstacle, that is, data that contributes to the obstacle's footprint contour. Therefore, fitting the laser point data falling into the overlapping area can estimate the obstacle's footprint area, and determining the outline of the obstacle's footprint area as the obstacle's footprint contour, thus realizing the estimation of the complete footprint contour of the obstacle using the obstacle detection box and 2D lasers. This invention does not limit the fitting method; it can be determined based on the shape of the obstacle, etc.

[0090] Optionally, fitting the target laser point data to obtain the footprint contour of the obstacle includes:

[0091] If the obstacle is a stationary vehicle, the minimum bounding rectangle of the laser points is obtained by fitting the target laser point data;

[0092] The outline of the minimum bounding rectangle of the laser points is determined as the footprint outline of the vehicle.

[0093] Among them, the minimum bounding rectangle of the laser points is the minimum bounding rectangle formed by the laser points.

[0094] For example, Figure 5 This is a schematic diagram illustrating the method for estimating the footprint of an obstacle using laser point data, as provided in Embodiment 2 of the present invention. Figure 5 In this example, the obstacle is a stationary vehicle. Figure 5 As shown, the minimum bounding rectangle of the laser points emitted by the laser sensor on the robot onto the vehicle surface is obtained by L-Shape fitting. The outline of the minimum bounding rectangle of the laser points is then used to determine the vehicle's footprint outline. For the laser point rectangle fitting of the vehicle, a complete vehicle outline can be fitted when only one short side and one long side are observed. Moreover, the vehicle's footprint outline obtained by the laser point fitting basically coincides with the actual vehicle footprint outline, thus achieving accurate estimation of the vehicle's footprint outline.

[0095] In this embodiment of the invention, obstacle images are captured by cameras at multiple different positions as the robot moves around an obstacle. Target detection is performed on each obstacle image to obtain an obstacle detection box. Based on the camera's intrinsic and extrinsic parameter matrices and the obstacle detection box, the ground projection area formed by the obstacle detection box projected onto the ground is estimated. The ground projection area completely includes the actual area occupied by the obstacle. The overlapping area of ​​the ground projection areas obtained from the obstacle's projection at multiple different positions is determined. Laser point data of the obstacle is acquired. The laser point data falling into the overlapping area is fitted to obtain the obstacle's footprint contour. By using obstacle detection boxes and 2D laser points to estimate the complete footprint contour of the obstacle, the problem of inaccurate or incomplete perception of the obstacle footprint contour by laser sensors is solved, further improving the accuracy of obstacle footprint contour perception. This avoids discontinuous trajectory and running stutters when navigating around obstacles, improving the robot's intelligence, smoothness, and cleaning efficiency.

[0096] Example 3

[0097] Figure 6 This is a schematic diagram of the structure of an obstacle footprint estimation device provided in Embodiment 3 of the present invention. Figure 6 As shown, the device includes:

[0098] The image acquisition module 310 is used to acquire images of the obstacle at multiple different positions using a camera while the robot is moving around the obstacle;

[0099] The target detection module 320 is used to perform target detection on each obstacle image to obtain an obstacle detection box;

[0100] The projection area determination module 330 is used to determine the ground projection area formed by the obstacle detection frame projected onto the ground based on the intrinsic and extrinsic parameter matrix of the camera and the obstacle detection frame; the ground projection area completely includes the actual occupied area of ​​the obstacle.

[0101] The land area contour estimation module 340 is used to determine the overlapping area of ​​the ground projection area obtained by projecting the obstacle at multiple different positions, and to estimate the land area contour of the obstacle based on the overlapping area.

[0102] Optionally, the projection area determination module 330 is specifically used for:

[0103] Based on the intrinsic and extrinsic parameter matrix of the camera, determine the projection line segment of the bottom of the obstacle detection frame onto the ground;

[0104] Obtain the maximum projected distance of the obstacle;

[0105] The ground projection area formed by the obstacle detection frame projected onto the ground is determined based on the projection line segment and the maximum projection distance.

[0106] Optionally, the site contour estimation module 340 includes:

[0107] The first land area contour determination unit is used to determine the outer contour of the overlapping area as the land area contour of the obstacle.

[0108] Optionally, the land area contour estimation module includes:

[0109] The first laser data acquisition unit is used to acquire laser point data of the obstacle. The laser point data is collected by the laser sensor during the robot's movement around the obstacle.

[0110] The second laser data acquisition unit is used to acquire target laser point data of the obstacle outline falling into the overlapping area;

[0111] The fitting unit is used to fit the target laser point data to obtain the footprint contour of the obstacle.

[0112] Optionally, the fitting unit is specifically used for:

[0113] If the obstacle is a stationary vehicle, the minimum bounding rectangle of the laser points is obtained by fitting the target laser point data;

[0114] The outline of the minimum bounding rectangle of the laser points is determined as the footprint outline of the vehicle.

[0115] Optionally, the location of the obstacle image captured by the camera on the robot is such that the obstacle is completely covered within the camera's field of view.

[0116] Optional, also includes:

[0117] The position acquisition module is used to acquire the robot's current position after estimating the footprint contour of the obstacle based on the overlapping area;

[0118] The path planning module is used to plan the robot's task execution path based on the footprint of the obstacle and the robot's current position, and to avoid the obstacle.

[0119] The obstacle footprint estimation device provided in the embodiments of the present invention can execute the obstacle footprint estimation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0120] Example 4

[0121] Figure 7 A schematic diagram of a robot 10, which can be used to implement embodiments of the present invention, is shown. The robot is intended to represent various forms of intelligent machines, such as sweeping robots, crystallizing robots, and floor-washing robots. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0122] like Figure 7 As shown, robot 10 includes at least one processor 11 and a memory, such as read-only memory (ROM) 12, random access memory (RAM) 13, etc., communicatively connected to at least one processor 11. The memory stores computer programs executable by at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer program stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of robot 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. Input / output (I / O) interface 15 is also connected to bus 14.

[0123] Multiple components in robot 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows robot 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for estimating the obstacle footprint contour.

[0125] In some embodiments, the obstacle footprint estimation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on robot 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the obstacle footprint estimation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the obstacle footprint estimation method by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a robot having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the robot. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0131] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0132] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for estimating the footprint contour of an obstacle, applied to a robot performing ground tasks, characterized in that, The method includes: As the robot moves around the obstacle, it captures images of the obstacle at multiple different locations using cameras; Object detection is performed on each obstacle image to obtain the obstacle detection box; Based on the camera's intrinsic and extrinsic parameter matrix and the obstacle detection frame, the ground projection area formed by the obstacle detection frame projected onto the ground is determined; the ground projection area completely includes the actual occupied area of ​​the obstacle. Determine the overlapping area of ​​the ground projection regions obtained by projecting the obstacle at multiple different positions, and estimate the footprint outline of the obstacle based on the overlapping area; The step of determining the ground projection area formed by the obstacle detection frame projected onto the ground based on the camera's intrinsic and extrinsic parameter matrix and the obstacle detection frame includes: Based on the intrinsic and extrinsic parameter matrix of the camera, determine the projection line segment of the bottom of the obstacle detection frame onto the ground; The maximum projection distance of the obstacle is obtained; wherein, the maximum projection distance of the obstacle is obtained by determining the maximum projection distance of the obstacle based on the obstacle size information estimated by the obstacle information collected by the sensors on the robot, or by determining the obstacle type based on the obstacle information collected by the sensors on the robot, and obtaining the pre-configured maximum projection distance based on the obstacle type. The bottom projection line segment of the obstacle detection frame on the ground is projected according to the maximum projection distance, so that the projection line segment is expanded into a ground projection area.

2. The method according to claim 1, characterized in that, The step of estimating the footprint contour of the obstacle based on the overlapping region includes: The outer contour of the overlapping area is defined as the footprint contour of the obstacle.

3. The method according to claim 1, characterized in that, Estimating the footprint profile of the obstacle based on the overlapping region includes: The robot acquires laser point data of obstacles, which is collected by laser sensors during the robot's movement around the obstacles. Acquire target laser point data of the obstacle contour falling into the overlapping region; The footprint contour of the obstacle is obtained by fitting the target laser point data.

4. The method according to claim 3, characterized in that, The process of fitting the target laser point data to obtain the footprint contour of the obstacle includes: If the obstacle is a stationary vehicle, the minimum bounding rectangle of the laser points is obtained by fitting the target laser point data; The outline of the minimum bounding rectangle of the laser points is determined as the footprint outline of the vehicle.

5. The method according to any one of claims 1-4, characterized in that, The camera on the robot captures the location of the obstacle image, ensuring that the obstacle is completely within the camera's field of view.

6. The method according to any one of claims 1-4, characterized in that, After estimating the footprint profile of the obstacle based on the overlapping region, the method further includes: Get the robot's current position; Based on the footprint and outline of the obstacle and the robot's current position, the robot plans its task execution path to avoid the obstacle.

7. An obstacle footprint estimation device, applied to a robot performing ground tasks, characterized in that, The device includes: The image acquisition module is used to acquire images of the obstacle at multiple different positions using cameras as the robot moves around the obstacle; The object detection module is used to perform object detection on each obstacle image to obtain the obstacle detection box; The projection area determination module is used to determine the ground projection area formed by the obstacle detection frame projected onto the ground based on the intrinsic and extrinsic parameter matrix of the camera and the obstacle detection frame; the ground projection area completely includes the actual occupied area of ​​the obstacle; The land area contour estimation module is used to determine the overlapping area of ​​the ground projection area obtained by projecting the obstacle at multiple different locations, and to estimate the land area contour of the obstacle based on the overlapping area. The projection area determination module is specifically used for: Based on the intrinsic and extrinsic parameter matrix of the camera, determine the projection line segment of the bottom of the obstacle detection frame onto the ground; The maximum projection distance of the obstacle is obtained; wherein, the maximum projection distance of the obstacle is obtained by determining the maximum projection distance of the obstacle based on the obstacle size information estimated by the obstacle information collected by the sensors on the robot, or by determining the obstacle type based on the obstacle information collected by the sensors on the robot, and obtaining the pre-configured maximum projection distance based on the obstacle type. The bottom projection line segment of the obstacle detection frame on the ground is projected according to the maximum projection distance, so that the projection line segment is expanded into a ground projection area.

8. A robot, characterized in that, The robot includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the method for estimating the obstacle footprint profile according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for estimating the obstacle footprint contour as described in any one of claims 1-6.

Images