Mobile robot obstacle avoidance method, device, computer equipment and readable storage medium

Abstract

The present invention relates to the field of robotics, and discloses a method, device, computer equipment, and readable storage medium for obstacle avoidance of a mobile robot. The method comprises: acquiring sensor data, locating obstacle objects according to the sensor data, and generating a global cost map according to the sensor data and the obstacle objects; calibrating a monocular camera on the obstacle objects to obtain the planar physical coordinates of the obstacle objects in the robot coordinate system; extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data; if so, extracting the effective contour of the target object, and generating a virtual point cloud according to the planar physical coordinates and effective contour corresponding to the target object; updating the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map. The present application uses visual calibration to perceive specific objects and construct object contours, so that the robot can navigate and avoid obstacles safely and smoothly.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a mobile robot obstacle avoidance method, device, computer equipment, and readable storage medium. Background Art

[0002] Robot navigation and obstacle avoidance are crucial for ensuring safety, improving task completion rates, enhancing efficiency and autonomy, and enabling robots to adapt to diverse environments and mission requirements. These capabilities are crucial for the successful deployment of robots in a variety of applications, including industrial automation, service robotics, and autonomous vehicles.

[0003] However, traditional sensors like 2D LiDAR, RGBD cameras (which rely on infrared), and ultrasonic sensors have limitations when it comes to accurately identifying and avoiding black or reflective objects. These technologies are limited by their data acquisition capabilities or angles, making them less effective at identifying black obstacles.

[0004] Currently, the main solution to this problem is to set up virtual walls to protect the corresponding objects or use virtual tracks to make the robot follow a specific route. However, this will greatly limit the flexibility of the robot's movement and cannot effectively solve the situation where specific obstacles move, such as the black car in the 4S store that needs to be temporarily moved to a new location for display. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and to provide a mobile robot obstacle avoidance method, device, computer equipment and readable storage medium.

[0006] The present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present disclosure provides a method for avoiding obstacles in a mobile robot, the method comprising:

[0008] Acquire sensor data, locate obstacles based on the sensor data, and generate a global cost map based on the sensor data and the obstacles;

[0009] Performing monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system;

[0010] extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data, wherein the obstacle object includes the target object;

[0011] If the target object is detected in the color data, extracting the effective contour of the target object, and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour;

[0012] The global cost map is updated according to the virtual point cloud to obtain a local cost map, and navigation and obstacle avoidance are performed according to the local cost map.

[0013] Optionally, the sensor data includes radar data, ultrasonic data, and depth data, and locating obstacle objects according to the sensor data and generating a global cost map according to the sensor data and the obstacle objects includes:

[0014] extracting point cloud features from the radar data, extracting measurement values ​​from the ultrasonic data, performing image processing on the depth data to obtain an obstacle area, and locating the obstacle object based on the point cloud features, the measurement values, and the obstacle area;

[0015] Converting the radar data, the ultrasonic data, and the depth data into the robot coordinate system and performing feature fusion to obtain fused data;

[0016] An initial cost map is constructed according to a preset navigation map, and the positions of the obstacle objects and their expansion areas are marked in the initial cost map according to the fusion data to obtain the global cost map.

[0017] Optionally, the performing a monocular camera calibration on the obstacle to obtain the planar physical coordinates of the obstacle includes:

[0018] Obtaining a checkerboard calibration plate, recording the distance between the checkerboard calibration plate and the robot at different heights, and calculating the vertex pixel coordinates of the checkerboard calibration plate at different heights based on each of the distances;

[0019] Determine the pixel coordinates of four checkerboard vertices that are closest to any pixel point on the obstacle and are located at the same height as any pixel point on the obstacle, and determine the physical coordinates of the pixel coordinates of the four checkerboard vertices in the robot coordinate system;

[0020] The physical coordinates of the four checkerboard vertices are perspective transformed to obtain the physical coordinates of the corresponding arbitrary pixel points in the robot coordinate system, and the physical coordinates of all arbitrary pixel points on the obstacle object are summarized to obtain the plane physical coordinates of the obstacle object.

[0021] Optionally, after extracting the color data from the sensor data, the method further includes:

[0022] Using a labeling tool to label the target object in the color data, and setting a model configuration file of the target object according to task requirements, wherein the model configuration file includes a training category;

[0023] The target object detection model is trained using the labeled color data and the model configuration file.

[0024] Optionally, performing visual perception detection on the color data to determine whether a target object is detected in the color data includes:

[0025] Preprocessing the color data, and inputting the preprocessed color data into the detection model for target detection, and outputting bounding box coordinates, confidence levels, and category labels of detected feature objects;

[0026] It is determined whether the category label of the feature object matches the training category of the target object. If so, the feature object is determined to be the target object.

[0027] Optionally, extracting the effective contour of the target object and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour includes:

[0028] Extracting the overall contour of the target object using an edge detection algorithm, and performing data analysis on the shape and position of the overall contour to determine a valid contour located at the lower edge of the target object;

[0029] The effective contour is converted into the robot coordinate system using the plane physical coordinates corresponding to the target object, and the virtual point cloud is created according to the converted effective contour.

[0030] Optionally, updating the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map includes:

[0031] updating the position of the target object and its expanded area in the global cost map according to the position and shape of the virtual point cloud to obtain an updated cost map;

[0032] Setting contour information of the robot according to the inscribed circle and the circumscribed circle of the robot, integrating the contour information of the robot into the updated cost map, and generating the local cost map;

[0033] A path planning algorithm is used to plan a feasible path for the robot according to the local cost map, and navigation and obstacle avoidance are performed according to the feasible path.

[0034] In a second aspect, an embodiment of the present disclosure provides an obstacle avoidance device for a mobile robot, the device comprising:

[0035] an acquisition module, configured to acquire sensor data, locate obstacles based on the sensor data, and generate a global cost map based on the sensor data and the obstacles;

[0036] A calibration module is used to calibrate the monocular camera on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system;

[0037] a detection module, configured to extract color data from the sensor data, perform visual perception detection on the color data, and determine whether a target object is detected in the color data, wherein the obstacle object includes the target object;

[0038] an extraction module, configured to extract a valid outline of the target object if the target object is detected in the color data, and generate a virtual point cloud according to the plane physical coordinates corresponding to the target object and the valid outline;

[0039] The obstacle avoidance module is used to update the global cost map according to the virtual point cloud, obtain a local cost map, and perform navigation and obstacle avoidance according to the local cost map.

[0040] In a third aspect, a computer device is provided in an embodiment of the present disclosure, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the mobile robot obstacle avoidance method described in the first aspect are implemented.

[0041] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the mobile robot obstacle avoidance method described in the first aspect are implemented.

[0042] Beneficial effects of this application:

[0043] The mobile robot obstacle avoidance method provided by the embodiment of the present application obtains sensor data, locates obstacle objects based on the sensor data, and generates a global cost map based on the sensor data and the obstacle objects; performs monocular camera calibration on the obstacle objects to obtain the planar physical coordinates of the obstacle objects in the robot coordinate system; extracts color data from the sensor data, performs visual perception detection on the color data, and determines whether the target object is detected in the color data, wherein the obstacle objects include the target object; if the target object is detected in the color data, extracts the effective contour of the target object, and generates a virtual point cloud based on the planar physical coordinates and effective contour corresponding to the target object; updates the global cost map based on the virtual point cloud to obtain a local cost map, and performs navigation and obstacle avoidance based on the local cost map. The present application uses visual calibration to perceive specific objects, thereby constructing the object contour, so that the robot can navigate and avoid obstacles safely and smoothly.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be considered as limiting the scope. A person of ordinary skill in the art can also derive other relevant drawings based on these drawings without inventive effort. Similar components are numbered similarly in the various drawings.

[0046] Figure 1 A flowchart of a mobile robot obstacle avoidance method provided by an embodiment of the present application is shown;

[0047] Figure 2 A schematic diagram showing the placement of a checkerboard calibration plate provided in an embodiment of the present application is shown;

[0048] Figure 3 A schematic structural diagram of a mobile robot obstacle avoidance device provided in an embodiment of the present application is shown;

[0049] Figure 4 A schematic structural diagram of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0051] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. Conversely, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0052] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0053] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the template description herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0055] Example 1

[0056] like Figure 1 FIG. 1 is a flow chart of a mobile robot obstacle avoidance method according to an embodiment of the present application. The mobile robot obstacle avoidance method provided by the embodiment of the present application includes the following steps:

[0057] Step S110 : acquiring sensor data, locating obstacle objects according to the sensor data, and generating a global cost map according to the sensor data and the obstacle objects.

[0058] In this embodiment, the robot uses sensors including a lidar (LiDAR), an ultrasonic sensor, and an RGBD (RGB+Depth) monocular camera. Sensor data collected by each sensor is acquired. The lidar data contains multiple points, which are grouped into clusters. Each cluster represents a possible obstacle. Point cloud features, such as shape, size, and position, are extracted from each cluster. Obstacles can be identified based on the extracted point cloud features. The presence of obstacles can be directly determined based on the measured values ​​of the ultrasonic data collected by the ultrasonic sensor. The depth data collected by the RGBD monocular camera is processed using image processing techniques such as edge detection, image segmentation, and morphological processing to extract obstacle regions and thus locate the obstacle.

[0059] Furthermore, the coordinate systems of radar data, ultrasonic data, and depth data are converted into the robot coordinate system, and the three are feature fused to obtain fused data to integrate the advantages of different sensors.

[0060] Next, a preset navigation map is obtained, typically containing static information about the environment. Based on the size and resolution of the preset navigation map, a blank initial costmap is created corresponding to it. The location of each obstacle is then marked in this initial costmap based on the fused data. Furthermore, for each obstacle, an expansion radius is added outward from the initial costmap, centered around it. This expansion radius is a parameter used to ensure the robot maintains a safe distance from obstacles. It can be determined based on factors such as the robot's size, speed, and dynamic performance, and is not limited in this embodiment.

[0061] The initial cost map, the positions of obstacles and their dilation areas are combined to form the final global cost map.

[0062] By fusing data from multiple sensors, this method can more comprehensively capture obstacle information, thereby improving obstacle detection accuracy. The global cost map generated from sensor data and obstacles intuitively represents the traffic conditions in different areas of the environment, providing an important reference for subsequent path planning algorithms.

[0063] Step S120 , performing monocular camera calibration on the obstacle object to obtain the planar physical coordinates of the obstacle object in the robot coordinate system.

[0064] Specifically, first, a checkerboard calibration plate consisting of black and white square grids is made (the side length of each checkerboard grid is equal, both are l). The checkerboard calibration plate is placed as follows: Figure 2 Place the checkerboard calibration board at different heights h (such as the ground, 0.2m, 0.3m, etc.) and record the distance X from the robot. c , according to X c The vertex pixel coordinates of the checkerboard calibration plate at different heights can be calculated respectively.

[0065] Then determine the pixel coordinates of the four checkerboard vertices that are closest to any pixel point on the obstacle and are at the same height. For example, if the height of an arbitrary pixel point P(x, y) from the ground is determined to be 0, then the pixel coordinates of the four checkerboard vertices closest to it and at the same height are (x1, y1), (x2, y2), (x3, y3), and (x4, y4). The physical coordinates of the pixel coordinates of these four checkerboard vertices in the robot coordinate system are (X1, y1), (X2, y2), (X3, y3), and (X4, y4). c +2*l,0)、(X c +2*l,-l)、(X c +1,0),(X c +l, -l).

[0066] Given the pixel coordinates and corresponding physical coordinates of any pixel point P and its four surrounding checkerboard vertices (on the same plane), we can establish a perspective transformation matrix through these points. Then, we can perform perspective transformation on the physical coordinates of the four checkerboard vertices to obtain the physical coordinates of the corresponding arbitrary pixel point P in the robot coordinate system. By summing up the physical coordinates of all arbitrary pixel points on the obstacle object in the robot coordinate system, we can obtain the planar physical coordinates of the obstacle object in the robot coordinate system.

[0067] The above method, through monocular camera calibration, converts pixel coordinates in an image into physical coordinates in the robot's coordinate system. This significantly improves the accuracy of obstacle location, providing a more reliable basis for subsequent path planning and obstacle avoidance strategies. Furthermore, monocular camera calibration does not rely on fixed external devices or landmarks, making it highly adaptable to various environments. Whether indoors or outdoors, and regardless of varying lighting conditions, as long as the camera can capture an image of an obstacle, its precise location can be determined through calibration.

[0068] Step S130: extract color data from the sensor data, perform visual perception detection on the color data, and determine whether a target object is detected in the color data, wherein the obstacle object includes the target object.

[0069] It should be noted that obstacle objects refer to all objects that need to be avoided in the sensor data, including static objects and dynamic objects. Target objects refer to dynamic and movable objects among obstacle objects, such as a black car that needs to be temporarily moved in a 4S store for display needs.

[0070] It can be understood that after obtaining the color data collected by the RGBD monocular camera, the target objects in the color data are labeled using annotation tools (such as LabelImg, VGG Image Annotator, etc.), and the model configuration file of the target object is set according to the task requirements, wherein the model configuration file includes the training category, input size, training parameters, etc.; the labeled color data and the set model configuration file are used to train the detection model of the target object (such as training the detection model of tires, cars and SUVs).

[0071] Furthermore, the color data is preprocessed as necessary to meet the input requirements of the detection model, usually including image resizing and pixel normalization. The preprocessed color data is input into each detection model for forward propagation to perform object detection (mainstream detection methods such as YOLOv5 can be used). Each detection model will output the bounding box coordinates, confidence score, and category label of each detected feature object.

[0072] If the model output contains the same category label as during training, the target object can be considered detected. In an optional embodiment, in order to reduce false detections, a confidence threshold can be set. Only when the confidence level of the detected feature object is higher than this threshold, it is considered as the target object.

[0073] It should be noted that if the target object is not detected in the model output, the next frame of color data is obtained to restart the visual perception detection.

[0074] Through visual perception and detection, these methods efficiently process sensor data and provide a real-time, intuitive understanding of the surrounding environment, including the presence of obstacles and the target objects within them. Compared to other non-visual methods, they typically offer higher accuracy and faster response times. In complex and changing scenarios, such as those subject to lighting variations, occlusions, and background clutter, visual perception and detection can effectively identify target objects by analyzing various image features, such as color, texture, and shape. This capability is crucial for improving the adaptability and robustness of robots in complex environments.

[0075] Step S140 : If the target object is detected in the color data, extract the effective contour of the target object, and generate a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour.

[0076] When a target object is detected in the color data, an edge detection algorithm (such as the Canny edge detector) is used to extract edges from the target object region. These edges generally correspond to the overall outline of the target object.

[0077] For each detected overall contour, analyze its shape and position to determine which parts correspond to the bottom edge of the target object, i.e., the valid contour of the target object (e.g., the contour of a car chassis). This may involve calculating the bounding box or convex hull of the overall contour, or analyzing the y-coordinates of the contour points (assuming that the y-axis in the image coordinate system is downwardly positive) to determine which points are at the lowest point. This application does not elaborate on this.

[0078] Using the planar physical coordinates of the target object calibrated in step S120, the extracted valid contour points are converted from the image coordinate system to the robot coordinate system, and a virtual point cloud is created based on the converted valid contour. A point cloud is a collection of points in three-dimensional space that represent the surface shape of an object. When generating the point cloud, the density and distribution of the points can be adjusted as needed to ensure that the point cloud accurately reflects the shape and characteristics of the contour.

[0079] The above method can generate a virtual point cloud of the target object by combining the effective outline of the target object in the color data and its corresponding plane physical coordinates. As a visual representation of three-dimensional data, the virtual point cloud can provide more intuitive and richer visual perception, and more accurately reflect the position and posture of the target object in three-dimensional space, thereby improving the accuracy and reliability of data processing.

[0080] Step S150 , updating the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map.

[0081] According to the position and shape of the virtual point cloud, the position of the target object and its expansion area is updated in the global cost map, which can include adding new obstacles, adjusting the size or position of existing obstacles, etc., to obtain an updated cost map.

[0082] Furthermore, the robot's inscribed and circumscribed circles are determined. The inscribed circle is typically used to determine the minimum space the robot can pass through, while the circumscribed circle is used to ensure sufficient safety distance when obstacles expand. These two circles are used to approximate the robot's outline information, which can determine whether the robot can pass through an obstacle (such as a door). This outline information is also integrated into the updated costmap to generate a local costmap.

[0083] Once the local cost map is updated and contains all the necessary information (including static obstacles, dynamic obstacles, and the robot's outline information), it can be published to the path planning algorithm (such as A*, Dijkstra, RRT, etc.). The path planning algorithm receives the local cost map as input and plans a feasible path for the robot based on the information in the local cost map, so that the robot can navigate and avoid obstacles according to the feasible path.

[0084] The local costmap generated by this method based on a virtual point cloud can more accurately reflect the distribution and shape characteristics of obstacles in the environment, providing more precise environmental information for navigation and obstacle avoidance. By fusing the global and local costmaps, the robot can comprehensively consider overall path planning and local obstacle avoidance requirements, making more accurate obstacle avoidance decisions and improving the success rate and safety of obstacle avoidance. In specific implementations, the method of this application can be combined with virtual walls and virtual tracks to make mobile robots move more smoothly and safely.

[0085] The mobile robot obstacle avoidance method provided by the embodiment of the present application obtains sensor data, locates obstacle objects based on the sensor data, and generates a global cost map based on the sensor data and the obstacle objects; performs monocular camera calibration on the obstacle objects to obtain the planar physical coordinates of the obstacle objects in the robot coordinate system; extracts color data from the sensor data, performs visual perception detection on the color data, and determines whether the target object is detected in the color data, wherein the obstacle objects include the target object; if the target object is detected in the color data, extracts the effective contour of the target object, and generates a virtual point cloud based on the planar physical coordinates and effective contour corresponding to the target object; updates the global cost map based on the virtual point cloud to obtain a local cost map, and performs navigation and obstacle avoidance based on the local cost map. The present application uses visual calibration to perceive specific objects, thereby constructing the object contour, so that the robot can navigate and avoid obstacles safely and smoothly.

[0086] Example 2

[0087] like Figure 3 FIG. 1 is a schematic structural diagram of a mobile robot obstacle avoidance device 300 according to an embodiment of the present application, wherein the device includes:

[0088] an acquisition module 310 for acquiring sensor data, locating obstacles based on the sensor data, and generating a global costmap based on the sensor data and the obstacles;

[0089] The calibration module 320 is used to perform a monocular camera calibration on the obstacle object to obtain the planar physical coordinates of the obstacle object in the robot coordinate system;

[0090] a detection module 330 for extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data, wherein the obstacle object includes the target object;

[0091] an extraction module 340 for extracting a valid outline of the target object if the target object is detected in the color data, and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the valid outline;

[0092] The obstacle avoidance module 350 is used to update the global cost map according to the virtual point cloud, obtain a local cost map, and perform navigation and obstacle avoidance according to the local cost map.

[0093] The mobile robot obstacle avoidance device provided in the embodiment of the present application uses visual calibration to perceive specific objects, thereby constructing the object outline, allowing the robot to navigate and avoid obstacles safely and smoothly.

[0094] Example 3

[0095] The present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0096] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0097] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0098] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D slot compatibility test memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for the slot compatibility test method. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0099] In some embodiments, the processor 42 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other mobile robot obstacle avoidance chip. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for executing the slot compatibility testing method.

[0100] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0101] The computer device provided in this embodiment can execute the above-mentioned mobile robot obstacle avoidance method. Here, the mobile robot obstacle avoidance method can be the mobile robot obstacle avoidance method of each of the above-mentioned embodiments.

[0102] Example 4

[0103] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the mobile robot obstacle avoidance method in the embodiment are implemented.

[0104] In this embodiment, the computer-readable storage medium includes flash memory, hard disks, multimedia cards, card-type memories (e.g., SD or DX memories), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk equipped with the computer device, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Of course, the computer-readable storage medium may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are about to be output.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0106] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0107] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium can be a non-volatile storage medium or a volatile storage medium. For example, the storage medium can be: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and other media that can store program codes.

[0108] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A mobile robot obstacle avoidance method, characterized in that: The method comprises: Acquire sensor data, locate obstacles based on the sensor data, and generate a global cost map based on the sensor data and the obstacles; Performing monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system; extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data, wherein the obstacle object includes the target object; If the target object is detected in the color data, extracting the effective contour of the target object, and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour; The global cost map is updated according to the virtual point cloud to obtain a local cost map, and navigation and obstacle avoidance are performed according to the local cost map.

2. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The sensor data includes radar data, ultrasonic data, and depth data. Positioning obstacle objects according to the sensor data and generating a global cost map according to the sensor data and the obstacle objects include: extracting point cloud features from the radar data, extracting measurement values ​​from the ultrasonic data, performing image processing on the depth data to obtain an obstacle area, and locating the obstacle object based on the point cloud features, the measurement values, and the obstacle area; Converting the radar data, the ultrasonic data, and the depth data into the robot coordinate system and performing feature fusion to obtain fused data; An initial cost map is constructed according to a preset navigation map, and the positions of the obstacle objects and their expansion areas are marked in the initial cost map according to the fusion data to obtain the global cost map.

3. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The monocular camera calibration of the obstacle object to obtain the planar physical coordinates of the obstacle object includes: Obtaining a checkerboard calibration plate, recording the distance between the checkerboard calibration plate and the robot at different heights, and calculating the vertex pixel coordinates of the checkerboard calibration plate at different heights based on each of the distances; Determine the pixel coordinates of four checkerboard vertices that are closest to any pixel point on the obstacle and are located at the same height as any pixel point on the obstacle, and determine the physical coordinates of the pixel coordinates of the four checkerboard vertices in the robot coordinate system; The physical coordinates of the four checkerboard vertices are perspective transformed to obtain the physical coordinates of the corresponding arbitrary pixel points in the robot coordinate system, and the physical coordinates of all arbitrary pixel points on the obstacle object are summarized to obtain the plane physical coordinates of the obstacle object.

4. The mobile robot obstacle avoidance method according to claim 1, characterized in that: After extracting the color data from the sensor data, the method further includes: Using a labeling tool to label the target object in the color data, and setting a model configuration file of the target object according to task requirements, wherein the model configuration file includes a training category; The target object detection model is trained using the labeled color data and the model configuration file.

5. The mobile robot obstacle avoidance method according to claim 4, characterized in that: The performing visual perception detection on the color data to determine whether a target object is detected in the color data includes: Preprocessing the color data, and inputting the preprocessed color data into the detection model for target detection, and outputting bounding box coordinates, confidence levels, and category labels of detected feature objects; It is determined whether the category label of the feature object matches the training category of the target object. If so, the feature object is determined to be the target object.

6. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The extracting the effective contour of the target object and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour includes: Extracting the overall contour of the target object using an edge detection algorithm, and performing data analysis on the shape and position of the overall contour to determine a valid contour located at the lower edge of the target object; The effective contour is converted into the robot coordinate system using the plane physical coordinates corresponding to the target object, and the virtual point cloud is created according to the converted effective contour.

7. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The updating of the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map, includes: updating the position of the target object and its expanded area in the global cost map according to the position and shape of the virtual point cloud to obtain an updated cost map; Setting contour information of the robot according to the inscribed circle and the circumscribed circle of the robot, integrating the contour information of the robot into the updated cost map, and generating the local cost map; A path planning algorithm is used to plan a feasible path for the robot according to the local cost map, and navigation and obstacle avoidance are performed according to the feasible path.

8. A mobile robot obstacle avoidance device, characterized in that: The device comprises: an acquisition module, configured to acquire sensor data, locate obstacles based on the sensor data, and generate a global cost map based on the sensor data and the obstacles; A calibration module is used to perform a monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system; a detection module, configured to extract color data from the sensor data, perform visual perception detection on the color data, and determine whether a target object is detected in the color data, wherein the obstacle object includes the target object; an extraction module, configured to extract a valid outline of the target object if the target object is detected in the color data, and generate a virtual point cloud according to the plane physical coordinates corresponding to the target object and the valid outline; The obstacle avoidance module is used to update the global cost map according to the virtual point cloud, obtain a local cost map, and perform navigation and obstacle avoidance according to the local cost map.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the mobile robot obstacle avoidance method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the mobile robot obstacle avoidance method according to any one of claims 1 to 7 are implemented.

Images