Obstacle avoidance method and device for mobile robot, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of robots, and discloses a mobile robot obstacle avoidance method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring sensor data, positioning obstacles according to the sensor data, and generating a global cost map according to the sensor data and the obstacles; monocular camera calibration is carried out on the obstacle object to obtain plane physical coordinates of the obstacle object in a robot coordinate system; extracting color data in the sensor data, performing visual perception detection on the color data, and judging whether a target object is detected in the color data; if yes, extracting an effective contour of the target object, and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour; and updating the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation obstacle avoidance according to the local cost map. Visual calibration is used for sensing a specific object, the object contour is constructed, and the robot can safely and smoothly conduct navigation obstacle avoidance.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a mobile robot obstacle avoidance method, device, computer equipment and readable storage medium. Background Art

[0002] The importance of robot navigation and obstacle avoidance lies in ensuring safety, improving task completion rate, enhancing efficiency and autonomy, and enabling robots to adapt to different environments and task requirements. These capabilities are crucial for the successful use of robots in various application fields, including industrial automation, service robots, unmanned vehicles, etc.

[0003] However, when facing black or reflective objects, traditional technologies such as 2D LiDAR, RGBD cameras (relying on infrared) and ultrasonic sensors have limitations and are difficult to accurately identify and avoid. These technologies are limited by data acquisition capabilities or angle restrictions, and are not effective in identifying black obstacles.

[0004] Currently, the main solution to this problem is to set up virtual walls to protect the corresponding objects or use virtual tracks to make the robot travel along a specific route. However, this will greatly limit the flexibility of the robot's movement and cannot solve the situation of specific obstacle movement well, such as the black car in the 4S store is temporarily moved to a new location for display due to display needs. Summary of the invention

[0005] In view of this, an object of the present invention is to overcome the deficiencies in the prior art and to provide a mobile robot obstacle avoidance method, device, computer equipment and readable storage medium.

[0006] The present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present disclosure provides a method for avoiding obstacles in a mobile robot, the method comprising:

[0008] Acquire sensor data, locate obstacle objects according to the sensor data, and generate a global cost map according to the sensor data and the obstacle objects;

[0009] Performing monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system;

[0010] Extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data, wherein the obstacle object includes the target object;

[0011] If the target object is detected in the color data, extracting the effective contour of the target object, and generating a virtual point cloud according to the plane physical coordinates and the effective contour corresponding to the target object;

[0012] The global cost map is updated according to the virtual point cloud to obtain a local cost map, and navigation and obstacle avoidance are performed according to the local cost map.

[0013] Optionally, the sensor data includes radar data, ultrasonic data, and depth data, and locating obstacle objects according to the sensor data, and generating a global cost map according to the sensor data and the obstacle objects, includes:

[0014] extracting point cloud features from the radar data, extracting measurement values ​​from the ultrasonic data, performing image processing on the depth data to obtain an obstacle area, and locating the obstacle object according to the point cloud features, the measurement values, and the obstacle area;

[0015] Converting the radar data, the ultrasonic data and the depth data into the robot coordinate system, and performing feature fusion to obtain fused data;

[0016] An initial cost map is constructed according to a preset navigation map, and the positions of the obstacle objects and their expansion areas are marked in the initial cost map according to the fusion data to obtain the global cost map.

[0017] Optionally, the performing monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object includes:

[0018] Obtain a checkerboard calibration plate, record the distance between the checkerboard calibration plate and the robot at different heights, and calculate the vertex pixel coordinates of the checkerboard calibration plate at different heights according to each of the distances;

[0019] Determine the pixel coordinates of four checkerboard vertices that are closest to any pixel point on the obstacle object and are located at the same height as any pixel point on the obstacle object, and determine the physical coordinates of the pixel coordinates of the four checkerboard vertices in the robot coordinate system;

[0020] The physical coordinates of the four chessboard vertices are perspective transformed to obtain the physical coordinates of the corresponding arbitrary pixel points in the robot coordinate system, and the physical coordinates of all arbitrary pixel points on the obstacle object are summarized to obtain the plane physical coordinates of the obstacle object.

[0021] Optionally, after extracting the color data from the sensor data, the method further includes:

[0022] Using a labeling tool to label the target object in the color data, and setting a model configuration file of the target object according to task requirements, wherein the model configuration file includes a training category;

[0023] The detection model of the target object is trained using the labeled color data and the model configuration file.

[0024] Optionally, performing visual perception detection on the color data to determine whether a target object is detected in the color data includes:

[0025] Preprocessing the color data, and inputting the preprocessed color data into the detection model for target detection, and outputting bounding box coordinates, confidence levels, and category labels of the detected feature objects;

[0026] It is determined whether the category label of the feature object matches the training category of the target object. If so, the feature object is determined to be the target object.

[0027] Optionally, extracting the effective contour of the target object and generating a virtual point cloud according to the plane physical coordinates and the effective contour corresponding to the target object includes:

[0028] Extracting the overall contour of the target object using an edge detection algorithm, and performing data analysis on the shape and position of the overall contour to determine a valid contour located at the lower edge of the target object;

[0029] The effective contour is converted into the robot coordinate system by using the plane physical coordinates corresponding to the target object, and the virtual point cloud is created according to the converted effective contour.

[0030] Optionally, updating the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map includes:

[0031] According to the position and shape of the virtual point cloud, the position of the target object and its dilated area are updated in the global cost map to obtain an updated cost map;

[0032] Setting contour information of the robot according to the inscribed circle and the circumscribed circle of the robot, integrating the contour information of the robot into the updated cost map, and generating the local cost map;

[0033] A path planning algorithm is used to plan a feasible path for the robot according to the local cost map, and navigation and obstacle avoidance are performed according to the feasible path.

[0034] In a second aspect, an embodiment of the present disclosure provides a mobile robot obstacle avoidance device, the device comprising:

[0035] An acquisition module, used to acquire sensor data, locate obstacle objects according to the sensor data, and generate a global cost map according to the sensor data and the obstacle objects;

[0036] A calibration module, used to perform monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system;

[0037] a detection module, configured to extract color data from the sensor data, perform visual perception detection on the color data, and determine whether a target object is detected in the color data, wherein the obstacle object includes the target object;

[0038] an extraction module, configured to extract a valid contour of the target object if the target object is detected in the color data, and generate a virtual point cloud according to the plane physical coordinates and the valid contour corresponding to the target object;

[0039] The obstacle avoidance module is used to update the global cost map according to the virtual point cloud, obtain a local cost map, and perform navigation and obstacle avoidance according to the local cost map.

[0040] In a third aspect, a computer device is provided in an embodiment of the present disclosure, wherein the computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the mobile robot obstacle avoidance method described in the first aspect are implemented.

[0041] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present disclosure, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the mobile robot obstacle avoidance method described in the first aspect are implemented.

[0042] Beneficial effects of this application:

[0043] The mobile robot obstacle avoidance method provided in the embodiment of the present application obtains sensor data, locates obstacle objects according to the sensor data, and generates a global cost map according to the sensor data and the obstacle objects; performs monocular camera calibration on the obstacle objects to obtain the plane physical coordinates of the obstacle objects in the robot coordinate system; extracts color data from the sensor data, performs visual perception detection on the color data, and determines whether the target object is detected in the color data, wherein the obstacle object includes the target object; if the target object is detected in the color data, extracts the effective contour of the target object, and generates a virtual point cloud according to the plane physical coordinates and effective contour corresponding to the target object; updates the global cost map according to the virtual point cloud to obtain a local cost map, and performs navigation and obstacle avoidance according to the local cost map. The present application uses visual calibration to perceive specific objects, thereby constructing the contours of the objects, so that the robot can navigate and avoid obstacles safely and smoothly.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work. In each of the drawings, similar components are numbered similarly.

[0046] Figure 1 A flow chart of a mobile robot obstacle avoidance method provided by an embodiment of the present application is shown;

[0047] Figure 2 A schematic diagram showing the placement of a chessboard calibration plate provided in an embodiment of the present application is shown;

[0048] Figure 3 A schematic structural diagram of a mobile robot obstacle avoidance device provided in an embodiment of the present application is shown;

[0049] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0051] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0052] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0053] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0055] Example 1

[0056] like Figure 1 , which is a flow chart of a mobile robot obstacle avoidance method in an embodiment of the present application. The mobile robot obstacle avoidance method provided in an embodiment of the present application includes the following steps:

[0057] Step S110 , acquiring sensor data, locating obstacle objects according to the sensor data, and generating a global cost map according to the sensor data and the obstacle objects.

[0058] In this embodiment, the sensors used by the robot include laser radar, ultrasonic sensor, and RGBD (RGB+Depth) monocular camera. The sensor data collected by each sensor is obtained, wherein there are multiple points in the radar data collected by the laser radar, and the points in the radar data are grouped into different clusters, each cluster representing a possible obstacle, and point cloud features such as shape, size, position, etc. are extracted from each cluster. The obstacle object can be identified based on the extracted point cloud features; the existence of the obstacle object can be directly judged based on the measurement value of the ultrasonic data collected by the ultrasonic sensor; the depth data collected by the RGBD monocular camera is subjected to image processing such as edge detection, image segmentation, and morphological processing to extract the obstacle area, and then locate the obstacle object.

[0059] Furthermore, the coordinate systems of radar data, ultrasonic data, and depth data are converted into the robot coordinate system, and the three are feature fused to obtain fused data to integrate the advantages of different sensors.

[0060] Next, a preset navigation map is obtained, which usually contains static information of the environment. A blank initial cost map corresponding to the preset navigation map is created according to the size and resolution of the preset navigation map, and the position of each obstacle object is marked in the initial cost map according to the fusion data. In addition, for each obstacle object, an area with an expansion radius is expanded outward in the initial cost map with it as the center. The expansion radius is a parameter used to ensure that the robot maintains a safe distance from the obstacle object. It can be determined according to factors such as the size, speed, and dynamic performance of the robot, and this embodiment does not limit it.

[0061] The initial cost map, the positions of obstacle objects and their dilation areas are integrated together to form the final global cost map.

[0062] The above method can obtain more comprehensive information about obstacles through data fusion of multiple sensors, thereby improving the accuracy of obstacle detection. The global cost map generated based on sensor data and obstacle objects can intuitively represent the traffic conditions in different areas of the environment, providing an important reference for subsequent path planning algorithms.

[0063] Step S120, performing monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system.

[0064] Specifically, first, a checkerboard calibration board consisting of black and white square grids needs to be made (the side length of each checkerboard grid is equal, both are l). The checkerboard calibration board is placed as follows: Figure 2 Place the checkerboard calibration board at different heights h (such as the ground, 0.2m, 0.3m, etc.) and record the distance X from the robot. c , according to X c Then the vertex pixel coordinates of the chessboard calibration plate at different heights can be calculated respectively.

[0065] Then determine the pixel coordinates of the four chessboard vertices that are closest to any pixel point on the obstacle and are at the same height. For example, if the height of an arbitrary pixel point P(x, y) from the ground is determined to be 0, then the pixel coordinates of the four chessboard vertices that are closest to it and at the same height are (x1, y1), (x2, y2), (x3, y3), and (x4, y4). The physical coordinates of the pixel coordinates of these four chessboard vertices in the robot coordinate system are (x1, y1), (x2, y2), (x3, y3), and (x4, y4). c +2*l,0)、(X c +2*l,-l)、(X c +1,0),(X c +l, -l).

[0066] Given the pixel coordinates and corresponding physical coordinates of any pixel point P and its four surrounding checkerboard vertices (on the same plane), a perspective transformation matrix can be established through these points. Then, the physical coordinates of the four checkerboard vertices can be transformed in perspective to obtain the physical coordinates of the corresponding arbitrary pixel point P in the robot coordinate system. By summarizing the physical coordinates of all arbitrary pixel points on the obstacle object in the robot coordinate system, the planar physical coordinates of the obstacle object in the robot coordinate system can be obtained.

[0067] The above method can convert the pixel coordinates in the image into physical coordinates in the robot coordinate system through monocular camera calibration. This greatly improves the accuracy of obstacle positioning and provides a more reliable basis for subsequent path planning and obstacle avoidance strategies. In addition, monocular camera calibration does not rely on fixed external equipment or markers, so it has strong environmental adaptability. Whether indoors or outdoors, no matter how the ambient lighting conditions change, as long as the camera can capture the image of the obstacle, its accurate position can be obtained through calibration.

[0068] Step S130, extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data, wherein the obstacle object includes the target object.

[0069] It should be noted that obstacle objects refer to all objects that need to be avoided in the sensor data, including static objects and dynamic objects, and target objects refer to dynamic and movable objects among obstacle objects, such as a black car that needs to be temporarily moved in a 4S store due to display needs.

[0070] Understandably, after acquiring the color data collected by the RGBD monocular camera, use a labeling tool (such as LabelImg, VGG Image Annotator, etc.) to label the target object in the color data, and set the model configuration file of the target object according to the task requirements, wherein the model configuration file includes training categories, input sizes, training parameters, etc.; use the labeled color data and the set model configuration file to train the detection model of the target object (such as training the detection model of tires, cars and SUVs).

[0071] Furthermore, the color data is preprocessed as necessary to meet the input requirements of the detection model, usually including adjusting the image size, normalizing pixel values, etc. The preprocessed color data is input into each detection model for forward propagation to perform target detection (mainstream detection methods such as yolov5 can be used), and each detection model will output the bounding box coordinates, confidence score, and category label of each detected feature object.

[0072] If the model output contains the same category label as that during training, it can be considered that the target object is detected. In an optional implementation, in order to reduce false detections, a confidence threshold can also be set, and only when the confidence of the detected feature object is higher than this threshold, it is considered as a target object.

[0073] It should be noted that if the target object is not detected in the model output, the next frame of color data is obtained to restart the visual perception detection.

[0074] The above method can efficiently process sensor data through visual perception detection, and can understand its surrounding environment in real time and intuitively, including the presence of obstacles and target objects in it. Compared with other non-visual methods, it usually has higher accuracy and faster response speed. In complex and changing scenes, such as lighting changes, occlusion, and background clutter, visual perception detection can effectively identify target objects by analyzing multiple features in the image, such as color, texture, shape, etc. This ability is of great significance to improving the adaptability and robustness of robots in complex environments.

[0075] Step S140 , if the target object is detected in the color data, extracting the effective contour of the target object, and generating a virtual point cloud according to the plane physical coordinates corresponding to the target object and the effective contour.

[0076] When a target object is detected in the color data, an edge detection algorithm (such as the Canny edge detector) is used to extract edges from the target object region, which edges generally correspond to the overall outline of the target object.

[0077] For each detected overall contour, its shape and position are analyzed to determine which parts correspond to the lower edge of the target object, that is, the effective contour of the target object (such as the contour of the chassis of a car). This may involve calculating the bounding box or convex hull of the overall contour or determining which points are at the lowest point by analyzing the y coordinates of the contour points (assuming that the y axis in the image coordinate system is downwardly positive), which will not be described in detail in this application.

[0078] Using the plane physical coordinates of the target object calibrated in step S120, the extracted valid contour points are converted from the image coordinate system to the robot coordinate system, and a virtual point cloud is created based on the converted valid contour. A point cloud is a collection of a large number of points in a three-dimensional space, which can represent the surface shape of an object. When generating a point cloud, the density and distribution of the points can be adjusted as needed to ensure that the point cloud can accurately reflect the shape and characteristics of the contour.

[0079] The above method can generate a virtual point cloud of the target object by combining the effective outline of the target object in the color data and its corresponding plane physical coordinates. The virtual point cloud, as a visual representation of three-dimensional data, can provide more intuitive and richer visual perception, and more accurately reflect the position and posture of the target object in three-dimensional space, thereby improving the accuracy and reliability of data processing.

[0080] Step S150, updating the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map.

[0081] According to the position and shape of the virtual point cloud, the position of the target object and its dilated area is updated in the global cost map, which may include adding new obstacles, adjusting the size or position of existing obstacles, etc., to obtain an updated cost map.

[0082] Furthermore, the inscribed circle and circumscribed circle of the robot are determined. The inscribed circle is usually used to determine the minimum space that the robot can pass through, while the circumscribed circle is used to ensure sufficient safety distance when the obstacle expands. These two circles are used to approximate the contour information of the robot, which can determine whether the robot can pass through an obstacle (such as a door). The contour information is integrated into the update cost map to generate a local cost map.

[0083] Once the local cost map is updated and contains all the necessary information (including static obstacles, dynamic obstacles, and robot outline information, etc.), it can be published to the path planning algorithm (such as A*, Dijkstra, RRT, etc.). The path planning algorithm receives the local cost map as input and plans a feasible path for the robot based on the information in the local cost map, so that the robot can navigate and avoid obstacles according to the feasible path.

[0084] The local cost map generated by the above method based on the virtual point cloud can more accurately reflect the distribution and shape characteristics of obstacles in the environment, and provide more accurate environmental information for navigation and obstacle avoidance. By integrating the global cost map and the local cost map, the robot can comprehensively consider the overall path planning and local obstacle avoidance requirements, make more accurate obstacle avoidance decisions, and improve the success rate and safety of obstacle avoidance. In specific implementation, the method of this application can be combined with virtual walls and virtual tracks to make the movement of mobile robots smoother and safer.

[0085] The mobile robot obstacle avoidance method provided in the embodiment of the present application obtains sensor data, locates obstacle objects according to the sensor data, and generates a global cost map according to the sensor data and the obstacle objects; performs monocular camera calibration on the obstacle objects to obtain the plane physical coordinates of the obstacle objects in the robot coordinate system; extracts color data from the sensor data, performs visual perception detection on the color data, and determines whether the target object is detected in the color data, wherein the obstacle object includes the target object; if the target object is detected in the color data, extracts the effective contour of the target object, and generates a virtual point cloud according to the plane physical coordinates and effective contour corresponding to the target object; updates the global cost map according to the virtual point cloud to obtain a local cost map, and performs navigation and obstacle avoidance according to the local cost map. The present application uses visual calibration to perceive specific objects, thereby constructing the contours of the objects, so that the robot can navigate and avoid obstacles safely and smoothly.

[0086] Example 2

[0087] like Figure 3 FIG. 1 is a schematic diagram of a mobile robot obstacle avoidance device 300 in an embodiment of the present application, and the device includes:

[0088] An acquisition module 310, configured to acquire sensor data, locate obstacle objects according to the sensor data, and generate a global cost map according to the sensor data and the obstacle objects;

[0089] The calibration module 320 is used to perform a monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system;

[0090] a detection module 330, configured to extract color data from the sensor data, perform visual perception detection on the color data, and determine whether a target object is detected in the color data, wherein the obstacle object includes the target object;

[0091] An extraction module 340 is used to extract the effective contour of the target object if the target object is detected in the color data, and generate a virtual point cloud according to the plane physical coordinates and the effective contour corresponding to the target object;

[0092] The obstacle avoidance module 350 is used to update the global cost map according to the virtual point cloud, obtain a local cost map, and perform navigation and obstacle avoidance according to the local cost map.

[0093] The mobile robot obstacle avoidance device provided in the embodiment of the present application uses visual calibration to perceive specific objects, thereby constructing the outline of the object, so that the robot can navigate and avoid obstacles safely and smoothly.

[0094] Example 3

[0095] The present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0096] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0097] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0098] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D slot compatibility test memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the slot compatibility test method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0099] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other mobile robot obstacle avoidance chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the slot compatibility test method.

[0100] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0101] The computer device provided in this embodiment can execute the above-mentioned mobile robot obstacle avoidance method. Here, the mobile robot obstacle avoidance method can be the mobile robot obstacle avoidance method of each of the above-mentioned embodiments.

[0102] Example 4

[0103] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the mobile robot obstacle avoidance method in the embodiment are implemented.

[0104] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, referred to as SMC), a secure digital (Secure Digital, referred to as SD) card, a flash card, etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0106] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0107] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium can be a non-volatile storage medium or a volatile storage medium. For example, the storage medium can be: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0108] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A mobile robot obstacle avoidance method, characterized in that: The method comprises: Acquire sensor data, locate obstacle objects according to the sensor data, and generate a global cost map according to the sensor data and the obstacle objects; Performing monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system; Extracting color data from the sensor data, performing visual perception detection on the color data, and determining whether a target object is detected in the color data, wherein the obstacle object includes the target object; If the target object is detected in the color data, extracting the effective contour of the target object, and generating a virtual point cloud according to the plane physical coordinates and the effective contour corresponding to the target object; The global cost map is updated according to the virtual point cloud to obtain a local cost map, and navigation and obstacle avoidance are performed according to the local cost map.

2. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The sensor data includes radar data, ultrasonic data and depth data, and locating obstacle objects according to the sensor data, and generating a global cost map according to the sensor data and the obstacle objects, including: extracting point cloud features from the radar data, extracting measurement values ​​from the ultrasonic data, performing image processing on the depth data to obtain an obstacle area, and locating the obstacle object according to the point cloud features, the measurement values, and the obstacle area; Converting the radar data, the ultrasonic data and the depth data into the robot coordinate system, and performing feature fusion to obtain fused data; An initial cost map is constructed according to a preset navigation map, and the positions of the obstacle objects and their expansion areas are marked in the initial cost map according to the fusion data to obtain the global cost map.

3. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The step of calibrating the obstacle object with a monocular camera to obtain the planar physical coordinates of the obstacle object includes: Obtain a checkerboard calibration plate, record the distance between the checkerboard calibration plate and the robot at different heights, and calculate the vertex pixel coordinates of the checkerboard calibration plate at different heights according to each of the distances; Determine the pixel coordinates of four checkerboard vertices that are closest to any pixel point on the obstacle object and are located at the same height as any pixel point on the obstacle object, and determine the physical coordinates of the pixel coordinates of the four checkerboard vertices in the robot coordinate system; The physical coordinates of the four chessboard vertices are perspective transformed to obtain the physical coordinates of the corresponding arbitrary pixel points in the robot coordinate system, and the physical coordinates of all arbitrary pixel points on the obstacle object are summarized to obtain the plane physical coordinates of the obstacle object.

4. The mobile robot obstacle avoidance method according to claim 1, characterized in that: After extracting the color data from the sensor data, the method further includes: Using a labeling tool to label the target object in the color data, and setting a model configuration file of the target object according to task requirements, wherein the model configuration file includes a training category; The detection model of the target object is trained using the labeled color data and the model configuration file.

5. The mobile robot obstacle avoidance method according to claim 4, characterized in that: The performing visual perception detection on the color data to determine whether a target object is detected in the color data includes: Preprocessing the color data, and inputting the preprocessed color data into the detection model for target detection, and outputting bounding box coordinates, confidence levels, and category labels of the detected feature objects; It is determined whether the category label of the feature object matches the training category of the target object. If so, the feature object is determined to be the target object.

6. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The step of extracting the effective contour of the target object and generating a virtual point cloud according to the plane physical coordinates and the effective contour corresponding to the target object includes: Extracting the overall contour of the target object using an edge detection algorithm, and performing data analysis on the shape and position of the overall contour to determine a valid contour located at the lower edge of the target object; The effective contour is converted into the robot coordinate system by using the plane physical coordinates corresponding to the target object, and the virtual point cloud is created according to the converted effective contour.

7. The mobile robot obstacle avoidance method according to claim 1, characterized in that: The updating of the global cost map according to the virtual point cloud to obtain a local cost map, and performing navigation and obstacle avoidance according to the local cost map, includes: According to the position and shape of the virtual point cloud, the position of the target object and its dilated area are updated in the global cost map to obtain an updated cost map; Setting contour information of the robot according to the inscribed circle and the circumscribed circle of the robot, integrating the contour information of the robot into the updated cost map, and generating the local cost map; A path planning algorithm is used to plan a feasible path for the robot according to the local cost map, and navigation and obstacle avoidance are performed according to the feasible path.

8. A mobile robot obstacle avoidance device, characterized in that: The device comprises: An acquisition module, used to acquire sensor data, locate obstacle objects according to the sensor data, and generate a global cost map according to the sensor data and the obstacle objects; A calibration module, used to perform monocular camera calibration on the obstacle object to obtain the plane physical coordinates of the obstacle object in the robot coordinate system; a detection module, configured to extract color data from the sensor data, perform visual perception detection on the color data, and determine whether a target object is detected in the color data, wherein the obstacle object includes the target object; an extraction module, configured to extract a valid contour of the target object if the target object is detected in the color data, and generate a virtual point cloud according to the plane physical coordinates and the valid contour corresponding to the target object; The obstacle avoidance module is used to update the global cost map according to the virtual point cloud, obtain a local cost map, and perform navigation and obstacle avoidance according to the local cost map.

9. A computer device, characterized in that: It comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the mobile robot obstacle avoidance method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the mobile robot obstacle avoidance method described in any one of claims 1 to 7 are implemented.

Images