An autonomous navigation and obstacle avoidance method for robots in dynamic scenes

Abstract

The present application provides a method for autonomous navigation and obstacle avoidance in a dynamic scene of a robot. An initial navigation path is obtained, and an environmental image of a moving target point is obtained through the robot's sensor equipment; the environmental image is input into a pre-trained image recognition model to obtain obstacle information and road information; the obstacle information includes the obstacle category corresponding to the obstacle, and the obstacle category includes static obstacles and dynamic obstacles; the obstacle avoidance algorithm corresponding to the obstacle is determined according to the obstacle category; during the navigation process, the initial navigation path is adjusted based on the obstacle information, road information and obstacle avoidance algorithm to obtain the target navigation path. Through the above method, the obstacle categories are divided into dynamic obstacles and static obstacles, and the corresponding obstacle avoidance algorithms are set respectively, which improves the adaptability and efficiency of the navigation system when facing different types of obstacles, and realizes the autonomous navigation and obstacle avoidance of the robot in a dynamic environment.

Description

Technical Field

[0001] The present application relates to the field of robot navigation technology, and in particular to an autonomous navigation and obstacle avoidance method for a robot in a dynamic scene. Background Art

[0002] A construction site is a complex and ever-changing environment, often with dense human activity, a large amount of construction materials and mechanical equipment, and potential dangerous factors. Currently, there are handling robots used to carry heavy objects on construction sites.

[0003] However, current traditional navigation and positioning technologies, such as the Global Positioning System (GPS), have positioning error problems in frequently changing or complex environments, which makes it impossible for robots to achieve accurate location information. In addition, existing navigation methods are usually based on static assumptions, that is, during the entire Simultaneous Localization and Mapping (SLAM) process, objects in the default environment are in a static state. However, most actual application environments are dynamic, with objects moving. Existing methods are difficult to meet the needs of autonomous navigation and obstacle avoidance of robots in dynamic environments, and often require operators to remotely control the movement of the handling robot. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method for autonomous navigation and obstacle avoidance of a robot in a dynamic scene, so as to realize autonomous navigation and obstacle avoidance of the robot in a dynamic environment.

[0005] To achieve the above objectives, the present application discloses the following technical solutions:

[0006] In a first aspect, the present application provides a method for autonomous navigation and obstacle avoidance in a dynamic scene of a robot, comprising:

[0007] Acquire an initial navigation path, wherein the initial navigation path includes a plurality of moving target points;

[0008] Acquiring an environmental image of the moving target point through a sensor device of the robot;

[0009] Inputting the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model; the obstacle information includes obstacle categories corresponding to the obstacles, and the obstacle categories include static obstacles and dynamic obstacles;

[0010] Determining an obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category;

[0011] During the navigation process, the initial navigation path is adjusted based on the obstacle information, the road information and the obstacle avoidance algorithm to obtain a target navigation path.

[0012] Optionally, if the obstacle category is a static obstacle, the obstacle avoidance algorithm corresponding to the obstacle includes: a two-way search A* algorithm; if the obstacle category of the obstacle is a dynamic obstacle, the obstacle avoidance algorithm corresponding to the obstacle includes: an artificial potential field algorithm.

[0013] Optionally, the obstacle information further includes: obstacle coordinates; if the obstacle category is a dynamic obstacle, adjusting the initial navigation path based on the obstacle, the road information and the obstacle avoidance strategy to obtain a target navigation path includes:

[0014] Establish an elliptical field and determine the semi-axis length of the elliptical field;

[0015] Determining whether the robot has a collision risk according to the semi-axis length and the obstacle coordinates;

[0016] If it is determined that there is a collision risk, the initial navigation path is adjusted based on the road information and the obstacle avoidance algorithm to obtain a target navigation path.

[0017] Optionally, the image recognition model includes a semantic segmentation sub-model and a target detection sub-model, and the inputting of the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model includes:

[0018] Inputting the environment image into the semantic segmentation sub-model, so that the semantic segmentation sub-model determines a first category of an object of a preset category in the environment image, and determines road information according to a pixel area whose first category is a road, to obtain the road information output by the semantic segmentation sub-model;

[0019] The environment image is input into the target detection sub-model to obtain obstacle information output by the target detection sub-model.

[0020] Optionally, the environment image includes a depth image and a red, green, and blue (RGB) image, and the method further includes:

[0021] Using a bilateral filtering algorithm and combining the gradient information of the RGB image, the hole part in the depth image is repaired to obtain a repaired depth image;

[0022] Then, inputting the environment image into a pre-trained image recognition model to obtain obstacle information output by the image recognition model includes: inputting the repaired depth image and RGB image into a pre-trained image recognition model to obtain the obstacle category output by the image recognition model.

[0023] Optionally, the adjusting the initial navigation path based on the obstacle information, the road information and the obstacle avoidance algorithm includes:

[0024] The obstacle is converted into a bounding volume by an axial bounding box algorithm;

[0025] Based on the bounding volume, the road information and the obstacle avoidance algorithm, the initial navigation path is adjusted in real time to obtain the target navigation path.

[0026] Optionally, before acquiring the environment image of the moving target point by the sensor device of the robot, the method further includes:

[0027] The sensor device is calibrated by a calibration algorithm.

[0028] Optionally, the semantic segmentation sub-model includes a SegNet model.

[0029] Optionally, the target detection submodel includes a YOLOV3 network model.

[0030] Optionally, obtaining the initial navigation path includes:

[0031] Based on the building information and navigation requirements provided by the BIM model, global path planning is carried out to obtain the initial navigation path.

[0032] In a second aspect, the present application provides an autonomous navigation and obstacle avoidance device, comprising:

[0033] An acquisition unit, configured to acquire an initial navigation path, wherein the initial navigation path includes a plurality of moving target points;

[0034] The acquisition unit is further used to acquire the environment image of the moving target point through the sensor device of the robot;

[0035] A model output unit, used to input the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model; the obstacle information includes obstacle categories corresponding to the obstacles, and the obstacle categories include static obstacles and dynamic obstacles;

[0036] A determination unit, configured to determine an obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category;

[0037] The adjustment unit is used to adjust the initial navigation path based on the obstacle information, the road information and the obstacle avoidance algorithm during the navigation process to obtain a target navigation path.

[0038] In a third aspect, the present application provides an autonomous navigation and obstacle avoidance device, comprising:

[0039] Memory for storing computer programs;

[0040] A processor is used to execute the computer program stored in the memory to implement the steps of the autonomous navigation and obstacle avoidance method of the robot in a dynamic scene as provided in the first aspect.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the autonomous navigation and obstacle avoidance method of the robot in a dynamic scene as provided in the first aspect.

[0042] The present application provides a method for autonomous navigation and obstacle avoidance in a dynamic scene of a robot, which obtains an initial navigation path, obtains an environmental image of a moving target point through the robot's sensor equipment; inputs the environmental image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model; the obstacle information includes the obstacle category corresponding to the obstacle, and the obstacle category includes static obstacles and dynamic obstacles; the obstacle avoidance algorithm corresponding to the obstacle is determined according to the obstacle category; during the navigation process, the initial navigation path is adjusted based on the obstacle information, road information and obstacle avoidance algorithm to obtain the target navigation path. Through the above method, the obstacle categories are divided into dynamic obstacles and static obstacles, and the corresponding obstacle avoidance algorithms are set respectively, which improves the adaptability and efficiency of the navigation system when facing different types of obstacles, and realizes the autonomous navigation and obstacle avoidance of the robot in a dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of an autonomous navigation and obstacle avoidance method for a robot in a dynamic scene provided by an embodiment of the present application;

[0044] Figure 2 It is a structural diagram of a robot perception technology framework provided by an embodiment of the present application;

[0045] Figure 3 is a schematic diagram of an elliptical field provided in an embodiment of the present application;

[0046] Figure 4 This is an example diagram of an obstacle avoidance process based on a bidirectional A* algorithm provided in an embodiment of the present application;

[0047] Figure 5 It is a schematic diagram of a Gmapping algorithm flow provided in an embodiment of the present application;

[0048] Figure 6 It is a structural schematic diagram of an autonomous navigation and obstacle avoidance device for a robot in a dynamic scene provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The following is a preliminary introduction to the terms involved in this application.

[0051] Obstacle avoidance refers to the process of ensuring that the device reaches the target location safely by identifying, avoiding or bypassing obstacles in the environment during movement. Obstacle avoidance is an important aspect of path planning, which aims to prevent collisions, reduce potential damage, and ensure the safe navigation of mobile objects in complex environments. To facilitate the understanding of the technical solution provided by the present application, the following will describe the autonomous navigation and obstacle avoidance method for a robot in a dynamic scene provided by the present application in conjunction with the accompanying drawings. It should be noted that the autonomous navigation and obstacle avoidance method for a navigation and obstacle avoidance robot in a dynamic scene provided by the present application can be used in the construction field or other fields. For example, it can be used in scenes in the construction field where autonomous navigation and obstacle avoidance of the robot is required in construction scenes. Other fields are any fields other than the construction field, for example, other application scenarios where autonomous navigation and obstacle avoidance of the robot is required. The above is only an example and does not limit the application field of the autonomous navigation and obstacle avoidance method for a navigation and obstacle avoidance robot in a dynamic scene provided by the present application.

[0052] See also Figure 1 , this figure is a flow chart of an autonomous navigation and obstacle avoidance method for a robot in a dynamic scene provided in an embodiment of the present application, and the method includes S101-S105.

[0053] S101: Acquire an initial navigation path, where the initial navigation path includes a plurality of moving target points.

[0054] In the embodiment of the present application, an initial navigation path is obtained, and the initial navigation path includes a plurality of moving target points. It can be understood that the moving target points are positions that the robot needs to pass through in the initial navigation path.

[0055] The present application does not limit the specific method of obtaining the initial navigation path. As an example, the present application can perform global path planning based on the building information provided by the Building Information Modeling (BIM) and navigation requirements to obtain the initial navigation path.

[0056] It is understandable that Building Information Modeling (BIM) can realize the integration of building information and integrate building information into a three-dimensional model information database. Building information can include the geometric information of components (such as walls, columns, beams, etc.) and the relative positions of each component in the entire building. Navigation requirements can include the location of the robot and the target location. Therefore, the robot can perform global path planning based on building information and navigation requirements to obtain an initial navigation path.

[0057] S102: Acquire the environmental image of the moving target point through the robot's sensor equipment.

[0058] After global path planning, the robot will use the moving target point as a scanning point for real-time positioning and map construction. At the moving target point, the robot's sensor device can be used to obtain the environmental image of the moving target point to realize the subsequent SLAM mapping process. The SLAM mapping process also includes the navigation process. SLAM is a technology that simultaneously performs positioning and map construction in an unknown environment. SLAM senses information about the surrounding environment by installing sensors on the device. The sensor data is then combined with the previous map information to continuously update the position of the device in the environment and update the environmental map at the same time, enabling the device to perform real-time positioning and navigation in an unknown or dynamic environment.

[0059] The present application does not limit the specific content of the sensing device. As an example, the sensing device may include a camera, a lidar, an infrared sensing device, an inertial measurement unit (IMU), and the like.

[0060] S103: Inputting the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model, wherein the obstacle information includes obstacle categories corresponding to the obstacles, and the obstacle categories include static obstacles and dynamic obstacles.

[0061] In an embodiment of the present application, after acquiring an environmental image, the environmental image can be input into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model. The present application does not limit the specific content of the obstacle information. As an example, the obstacle information includes the obstacle category corresponding to the obstacle, and the obstacle category includes static obstacles and dynamic obstacles. The present application does not limit the specific content of the road information. As an example, the road information can include road boundary lines, and the route can be accurately planned through the road information.

[0062] This application does not limit the specific content of the image recognition model, which is explained in detail below.

[0063] S104: Determine the obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category.

[0064] The present application does not limit the specific method of determining the obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category. As an example, the present application can pre-set the preset correspondence between the obstacle category and the obstacle avoidance algorithm. When the obstacle category is determined, the obstacle avoidance algorithm can be directly and quickly determined based on the obstacle category and the preset correspondence.

[0065] The present application does not limit the obstacle avoidance algorithm corresponding to the obstacle category. As a possible implementation method, if the obstacle category is determined to be a static obstacle in S103, the obstacle avoidance algorithm corresponding to the obstacle includes: a two-way search A* algorithm; if the obstacle category of the obstacle is determined to be a dynamic obstacle in S103, the obstacle avoidance algorithm corresponding to the obstacle includes: an artificial potential field algorithm. The specific contents of the two-way search A* algorithm and the artificial potential field algorithm are described below.

[0066] S105: During the navigation process, the initial navigation path is adjusted based on obstacle information, road information and obstacle avoidance algorithm to obtain the target navigation path.

[0067] In the embodiment of the present application, different obstacle avoidance algorithms are used to adjust the initial navigation path based on obstacles of different obstacle categories, so as to flexibly avoid obstacles of different types.

[0068] Through the above method, the obstacle categories are divided into dynamic obstacles and static obstacles, and corresponding obstacle avoidance algorithms are set respectively, which can improve the adaptability and efficiency of the navigation system when facing different types of obstacles, and realize the autonomous navigation and obstacle avoidance of the robot in a dynamic environment.

[0069] The present application does not limit the specific content of the image recognition model. As an example, the image recognition model includes a semantic segmentation sub-model and a target detection sub-model. Then, S103 inputs the environment image into the pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model, including:

[0070] A1: Input the environment image into the semantic segmentation sub-model so that the semantic segmentation sub-model determines the first category of the object of the preset category in the environment image, and determines the road information according to the pixel area whose first category is the road, and obtains the road information output by the semantic segmentation sub-model.

[0071] A2: Input the environment image into the target detection sub-model to obtain the obstacle information output by the target detection sub-model.

[0072] In an embodiment of the present application, the image recognition model may include two parts, namely a semantic segmentation submodel and a target detection submodel. The semantic segmentation submodel is used to perform semantic segmentation on the environmental image. As a possible implementation method, the semantic segmentation submodel may be a SegNet model. Semantic segmentation is the process of classifying pixels with similar categories or labels into one category. All objects of the same category are assigned the same label. The semantic segmentation submodel can be used to determine the first category of objects of a preset category in the environmental image. As an example, the present application can use the semantic segmentation submodel to segment the input environmental image into three categories: sky, building, and road. The semantic segmentation submodel can further determine the road information based on the pixel area whose first category is road.

[0073] This application uses the SegNet model for semantic segmentation. It is understandable that the SegNet model is trained using the SUN RGB-D dataset, which has achieved advanced performance in all major scene understanding tasks. The SegNet model can run efficiently with relatively low memory consumption on devices with GPUs, and SegNet exhibits clearer and more accurate characteristics in segmentation tasks. This application uses the SegNet model for semantic segmentation, which can more accurately identify the environment, greatly improving the robot's ability to solve dynamic obstacle perception problems during autonomous navigation, thereby effectively coping with complex scenes in navigation and achieving obstacle avoidance.

[0074] As a possible implementation method, the target detection submodel may be a YOLO V3 network model, which can simultaneously detect multiple targets in an image and directly output their location and category information.

[0075] The present application detects road information and obstacle information respectively through a semantic segmentation sub-model and a target detection sub-model, which can more effectively provide more accurate obstacle information and road information for obstacle avoidance, thereby improving the accuracy of autonomous navigation and obstacle avoidance.

[0076] As a possible implementation method, the image recognition model is trained in the following way. The training steps include:

[0077] Step 1: Obtain training data, which includes training environment images and labels corresponding to the training environment.

[0078] The embodiment of the present application first obtains training data to obtain the training environment image and the label corresponding to the training environment image included in the training data. It can be understood that the label is used to identify the obstacle information and road information in the training environment image.

[0079] Step 2: Input the training environment image into the model to be trained, and obtain the training obstacle information and training road information output by the model to be trained.

[0080] In the embodiment of the present application, the training environment image is input into the model to be trained, and the training obstacle information and training road information output by the model to be trained can be obtained. It can be understood that when the image recognition model includes a semantic segmentation sub-model and a target detection sub-model, the present application also divides the model to be trained into a semantic segmentation sub-model to be trained and a target detection sub-model to be trained. Thus, the training environment image is respectively input into the semantic segmentation sub-model to be trained and the target detection sub-model to be trained for training.

[0081] Step 3: Use the training obstacle information, training road information and labels to train the model to be trained until the loss value is less than the threshold to obtain the image recognition model. The loss value is used to measure the gap between the training obstacle information, training road information and the labels.

[0082] It is understandable that the network structure and parameters of the model will be continuously adjusted during the model training process. When the difference between the training obstacle information and training road information output by the model and the label is less than the threshold, the training ends and the network structure and parameters of the model are fixed. Based on the fixed network structure and parameters of the model, a trained image recognition model can be obtained.

[0083] As another possible implementation method, in an embodiment of the present application, the training process of the YOLOV3 network is divided into two steps. First, the pre-trained weights are imported and the last three convolutional layers are frozen for 200 epochs. Epoch is a unit, and an epoch represents the number of updates when all training data in learning have been used once. The initial learning rate is 10-3, and the batch size is 32. Next, the model is fine-tuned, the last three convolutional layers are unfrozen, and 200 epochs are performed again, with an initial learning rate of 10-4 and a batch size of 8. During the training process, the ReduceLROnPlateau callback function is used, and the learning rate is multiplied by 0.5 when the training loss no longer decreases in 10 consecutive iterations. At the same time, the Tensorboard callback function is applied to dynamically observe and save the model parameters.

[0084] like Figure 2 As shown, Figure 2 A schematic diagram of the structure of a robot perception technology framework provided by an embodiment of the present application. The present application can input a test image (the test image can be the environment image recorded above) into the semantic segmentation sub-model and the target detection sub-model respectively. Figure 2The dashed box on the left in the figure represents the semantic segmentation sub-model, which includes an encoder layer, a decoder layer, and a fully connected layer. The semantic segmentation sub-model is trained based on the training image data, so that the semantic segmentation sub-model can realize image semantic segmentation and thus extract road boundary lines. Figure 2 The dotted box on the right side of the figure represents the target detection sub-model, which includes a convolutional layer, a feature layer, a pooling layer, and a fully connected layer. Obstacle recognition can be achieved through this target detection sub-model. Combining the semantic segmentation sub-model and the target detection sub-model can obtain a visual perception result, which includes obstacle information and road information.

[0085] As a possible implementation, the obstacle information determined in S103 further includes: obstacle coordinates; if the obstacle category determined in S103 is a dynamic obstacle, adjusting the initial navigation path based on the obstacle information, road information, and obstacle avoidance algorithm to obtain the target navigation path includes the following steps B1-B3:

[0086] B1: Establish an elliptical shape field and determine the semi-axis length of the elliptical shape field.

[0087] In the embodiment of the present application, when the obstacle category is determined to be a dynamic obstacle, an artificial potential field algorithm is used as an obstacle avoidance algorithm. The artificial potential field established by the present application is elliptical. The present application can establish different elliptical potential fields according to the needs of different obstacle avoidance accuracies. Different elliptical potential fields can have different semi-axis lengths. It can be understood that the semi-axis length can include the length of the major semi-axis and the minor semi-axis of the ellipse. Figure 3 As shown, Figure 3 A schematic diagram of an elliptical field provided in this application. R0 represents the position of the robot, a is the major semi-axis, b is the minor semi-axis, and the robot's movement direction and magnitude are represented by vector v0. O1 and O2 represent different obstacles, and their movement directions and speed magnitudes are represented by vectors v1 and v2, respectively.

[0088] B2: Determine whether the robot has a collision risk based on the semi-axis length and the obstacle coordinates.

[0089] The present application does not limit the specific method of determining whether the robot has a collision risk based on the semi-axis length and the obstacle coordinates. As an example, the present application can calculate the distance between the obstacle and the robot according to the following formula.

[0090] ;

[0091] ;

[0092] ;

[0093] Where d represents the distance between the obstacle and the robot, ( , )and( , ) are the coordinates of the foci C1 and C2 of the ellipse, ( , ) is the robot coordinate, θ is the robot's moving direction, (x, y) is the obstacle coordinate, is the focal length of the ellipse.

[0094] It can be understood that if the distance d between the obstacle and the robot is less than or equal to twice the major semi-axis of the elliptical field, for example, the major semi-axis of the elliptical field is a, if it is determined that the distance d between the obstacle and the robot is less than or equal to 2a, it indicates that the obstacle has the possibility of colliding with the robot, and an obstacle avoidance strategy needs to be implemented. Otherwise, it means that the probability of collision between the obstacle and the robot is low, and no obstacle avoidance measures are needed.

[0095] Furthermore, the present application can also calculate the repulsion between the robot and the obstacle according to the following repulsion function, so as to more accurately determine whether there is a collision risk.

[0096] ;

[0097] in, represents the repulsive force between the robot and the obstacle, is the repulsive field function, is the proportional gain coefficient of the repulsive field, It means that the repulsive force acts within the ellipse. It means that the repulsive force acts outside the ellipse, a refers to the long semi-axis of the ellipse field, is the distance from the robot's horizontal coordinate to the obstacle's horizontal coordinate, Represents the sum of the distances from the horizontal coordinate of the obstacle to the horizontal coordinates of the two foci of the ellipse.

[0098] B3: If it is determined that there is a collision risk, the initial navigation path is adjusted based on the road information and the obstacle avoidance algorithm to obtain the target navigation path.

[0099] After determining that there is a collision risk, the application can adjust the initial navigation path based on the road information. As a possible implementation method, the application can randomly simulate changing the navigation path, and then cyclically execute the steps of determining whether the robot has a collision risk based on the semi-axis length and the obstacle coordinates, until it is determined that the robot does not have a collision risk, and obtain the target navigation path.

[0100] Through the above method, the initial navigation path can be adjusted through the improved artificial potential field algorithm. The present application improves the repulsion function in the artificial potential field algorithm to obtain a local optimal path and avoid dynamic obstacles more accurately.

[0101] As a possible implementation, Figure 4 As shown, Figure 4 An example diagram of an obstacle avoidance process based on a two-way search A* algorithm provided in an embodiment of the present application; if the obstacle category determined in S103 is a static obstacle, the present application determines the target navigation path through a two-way search A* algorithm, which may specifically include the following steps C1-C4:

[0102] C1: Create forward lists Open1 and Close1, and reverse lists Open2 and Close2 respectively, add the start node S to forward Open1, and the target node K to reverse Open2, and start searching from both ends at the same time.

[0103] It is understandable that the forward list and the reverse list are used to manage the data structure of the state node in the search process. These lists help the bidirectional search A* algorithm maintain and select the appropriate path during the search process.

[0104] C2: Select the node with the smallest evaluation value from the positive and negative Open1 and Open2 as their respective current nodes, and add them to their respective Close1 and Close2 lists.

[0105] It is understood that the evaluation value can be calculated based on the following formula:

[0106] ;

[0107] Among them, n is the current node, F(n) is the estimated value from the starting node to the target node, which is also the evaluation value recorded above, G(n) is the actual value from the starting node to the current node, and H(n) is the estimated value from the current node to the target node, also known as the heuristic function. The heuristic function in this paper uses the Euclidean distance, which is:

[0108] ; (Xn, Yn) represents the position coordinates of the current node, and (Xg, Yg) represents the coordinates of the target node.

[0109] C3: Traverse the adjacent nodes of the current node. If they are not in the corresponding Open list, add them to the Open list and set the current node as its parent node. If they are already in the Open list, compare the current F value with the F value in the Open list. If it is less than the F value in the Open list, replace it with the current value and change its parent node to the current node. If the adjacent node is an obstacle or in the Close table, skip the node. The F value is the evaluation value mentioned above.

[0110] C4: Loop through C2-C3 until the bidirectional search finds an intersection point, the target navigation path is determined, and the algorithm ends.

[0111] It can be seen that the present application can achieve static obstacle avoidance through the above-mentioned two-way search A* algorithm, and through two-way search, the target navigation path can be determined more quickly and effectively.

[0112] As a possible implementation method, the environmental image includes a depth image and a red, green, and blue (RGB) image. After obtaining the environmental image, the autonomous navigation and obstacle avoidance method for a robot in a dynamic scene provided in the present application also includes: using a bilateral filtering algorithm and combining the gradient information of the RGB image to repair the hole parts in the depth image to obtain a repaired depth image.

[0113] It is understandable that the environment image includes depth image and RGB image. In order to achieve accurate positioning of the mobile robot when it reaches the target position, high-quality depth image is crucial. However, when collecting depth images, due to problems such as regional occlusion and mirror reflection, a large number of holes and noise will appear in the image. Holes refer to grayscale values ​​of certain areas in the depth image being 0. In order to obtain reliable depth data, the collected depth image needs to be repaired to ensure that the depth value matches the actual situation.

[0114] The present application may use an improved bilateral filtering algorithm to repair the hole portion in the depth image in combination with the gradient information of the RGB image to obtain a repaired depth image. As an example, the present application may filter the depth image based on the following formula.

[0115] ;

[0116] ;

[0117] In the formula, and are the pixel values ​​of pixel points p and q respectively, D q is the depth of pixel q. p represents the filtered depth map, G e Represents the gradient calculation formula. is the area covered by the convolution kernel, represents the distance weight, represents the gray value weight, represents the normalized weight coefficient, is the gradient derivative function, This means that image gradient information is incorporated into the traditional bilateral filtering algorithm. Then, in S103, the environment image is input into a pre-trained image recognition model to obtain obstacle information output by the image recognition model, including: inputting the restored depth image and RGB image into a pre-trained image recognition model to obtain obstacle categories output by the image recognition model.

[0118] This application is to input the repaired depth image into the image recognition model, fill the lost information of the depth image with the gradient information in the RGB image, protect the image edge details, and effectively repair the small holes on the edge of the target obstacle. The local features of the image are introduced, and the gradient similarity is used instead of the grayscale similarity, and the gradient bilateral filter is formed with the spatial distance to average the neighborhood pixels of the image. This algorithm can reduce the influence of the positioning of the target recognition box during the recognition process, and provide a more informative and accurate input image for the subsequent positioning of obstacles.

[0119] As a possible implementation manner, the initial navigation path is adjusted based on the obstacle information, the road information and the obstacle avoidance algorithm to obtain a target navigation path, including D1-D2:

[0120] D1: The obstacle is converted into a bounding volume through an axial bounding box algorithm.

[0121] D2: Based on the bounding volume, the road information and the obstacle avoidance algorithm, the initial navigation path is adjusted in real time to obtain the target navigation path.

[0122] The implementation principle of the axial bounding box algorithm is shown in the following formula: ;

[0123] Among them, C represents the center point of the OBB bounding box, v3, v4, and v5 represent three mutually perpendicular directions, r3, r4, and r5 represent the side lengths in the three corresponding directions, R represents the position information of the bounding box, and a, b, and c are calculation coefficients.

[0124] Through the axial bounding box algorithm, obstacles can be converted into simple bounding volumes, which simplifies the difficulty of collision detection during autonomous navigation and obstacle avoidance, makes the algorithm more real-time, and reduces the probability of collision.

[0125] As a possible implementation method, this application can also use the Gmapping algorithm for autonomous navigation during the SLAM mapping process, such as Figure 5 As shown, Figure 5 A flowchart of a Gmapping algorithm provided for an embodiment of the present application. The basic idea of ​​the Gmapping algorithm is to simulate robot mapping as a Bayesian filtering problem, and update the robot's position and map through a recursive algorithm to achieve synchronous positioning and map construction. First, the robot is positioned through particle filtering; then the scanned particles are matched with the generated map; finally, new scanned particles are added based on the corrected odometer error to create or update the map. The specific steps are as follows:

[0126] Step 1: Data input. This application will receive the laser radar data (that is, Figure 5 The point cloud data in the image is aligned with the odometer data by time stamp, and packaged into a frame of laser odometer data. The weight of each particle is initialized according to the following formula as a preparation for subsequent mapping. This application uses the particle filter-based positioning mapping method (RBPF, Rao-Blackwellized Particle Filters) for mapping. t-1 The particle position and t Time odometer data, prediction t The particle's position at the moment, as a preparation for subsequent scan matching.

[0127] ;

[0128] in, is the resampling importance coefficient of the i-th particle in a frame of laser odometer data, i can be any positive integer from 1 to N; N is the total number of particles.

[0129] Step 2: Scan matching. Based on the predicted pose of the particle, scan matching is performed. For each particle, the current lidar and odometer data are used to search in six directions (up, down, left, right, upper left, and upper right) through the hill climbing algorithm to find the optimal pose. The matching degree of the laser data in the six directions is compared with the current map. If a better pose is found to make the map more matched, the pose of the current particle is updated. According to the updated pose, the laser beam is traversed under the current map, the likelihood value is calculated, and accumulated as the new weight of the particle.

[0130] Step 3: Weight normalization and resampling: After scanning and matching, the weights of all particles have been updated. Then, the weights of all particles are normalized and the discreteness of the particle set is calculated. N eff .

[0131] ;in,

[0132] N effIndicates the degree of discreteness of the particle set.

[0133] like N eff If the value is less than a certain threshold, it indicates that the particles are very different and need to be resampled; otherwise, no resampling is performed. This process is repeated until resampling is no longer needed.

[0134] Step 4: Map construction: Select the particle with the highest weight from all particles, and then use the historical posture data and laser data stored by the particle to update the map. The map update is Figure 5 The map update step in . At this point, a Gmapping operation is completed. This application can quickly and accurately implement SLAM mapping through the Gmapping algorithm.

[0135] As a possible implementation method, before acquiring the environmental image of the moving target point through the robot's sensor device, the autonomous navigation and obstacle avoidance method of a robot in a dynamic scene provided by the present application also includes: calibrating the sensor device through a calibration algorithm.

[0136] The present application does not limit the specific content of the calibration algorithm. As an example, when the sensing device includes a camera, the present application can use Zhang Youzheng's calibration method to quickly obtain the camera's intrinsic parameter matrix and extrinsic parameter matrix, and combine the Levenberg-Marquardt (LM) algorithm to iteratively optimize the intrinsic parameter matrix and the extrinsic parameter matrix to complete the camera calibration.

[0137] As another example, when the sensing device includes a camera and a radar, the present application may use a specific calibration plate to jointly calibrate the data collected by both, so as to integrate the data from different sensors into a consistent coordinate system.

[0138] It can be understood that the present application helps to improve the accuracy of the sensing device in measurement and perception by calibrating the sensing equipment, and can also integrate the data of different sensors into a consistent coordinate system to facilitate map creation and positioning accuracy.

[0139] See also Figure 6 , Figure 6 A schematic diagram of the structure of an autonomous navigation and obstacle avoidance device provided in an embodiment of the present invention, the device includes an acquisition unit 601, a model output unit 602, a determination unit 603 and an adjustment unit 604.

[0140] An acquisition unit 601 is used to acquire an initial navigation path, where the initial navigation path includes a plurality of moving target points;

[0141] The acquisition unit 601 is further used to acquire the environment image of the moving target point through the sensor device of the robot;

[0142] The model output unit 602 is used to input the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model; the obstacle information includes obstacle categories corresponding to the obstacles, and the obstacle categories include static obstacles and dynamic obstacles;

[0143] A determination unit 603 is used to determine an obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category;

[0144] The adjustment unit 604 is used to adjust the initial navigation path based on the obstacle information, the road information and the obstacle avoidance algorithm during the navigation process to obtain a target navigation path.

[0145] Obtain the initial navigation path, and obtain the environmental image of the moving target point through the robot's sensor equipment; input the environmental image into the pre-trained image recognition model to obtain the obstacle information and road information output by the image recognition model; the obstacle information includes the obstacle category corresponding to the obstacle, and the obstacle category includes static obstacles and dynamic obstacles; determine the obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category; during the navigation process, adjust the initial navigation path based on the obstacle information, road information and obstacle avoidance algorithm to obtain the target navigation path. Through the above device, the obstacle categories are divided into dynamic obstacles and static obstacles, and the corresponding obstacle avoidance algorithms are set respectively, which improves the adaptability and efficiency of the navigation system when facing different types of obstacles, and realizes the autonomous navigation and obstacle avoidance of the robot in a dynamic environment.

[0146] As a possible implementation manner, if the obstacle category is a static obstacle, the obstacle avoidance algorithm corresponding to the obstacle includes: a two-way search A* algorithm; if the obstacle category of the obstacle is a dynamic obstacle, the obstacle avoidance algorithm corresponding to the obstacle includes: an artificial potential field algorithm.

[0147] As a possible implementation manner, the obstacle information further includes: obstacle coordinates; and the adjustment unit specifically includes:

[0148] Establishing a subunit for establishing an elliptical situation field and determining the semi-axis length of the elliptical situation field;

[0149] a determination subunit, configured to determine whether the robot has a collision risk according to the semi-axis length and the obstacle coordinates;

[0150] The adjustment subunit is used to adjust the initial navigation path based on the road information and the obstacle avoidance algorithm to obtain a target navigation path if it is determined that there is a collision risk.

[0151] As a possible implementation, the image recognition model includes a semantic segmentation sub-model and a target detection sub-model, and the model output unit is specifically used to:

[0152] Inputting the environment image into the semantic segmentation sub-model, so that the semantic segmentation sub-model determines a first category of an object of a preset category in the environment image, and determines road information according to a pixel area whose first category is a road, to obtain the road information output by the semantic segmentation sub-model;

[0153] The environment image is input into the target detection sub-model to obtain obstacle information output by the target detection sub-model.

[0154] As a possible implementation manner, the environment image includes a depth image and a red, green, and blue (RGB) image, and the device further includes:

[0155] A repair unit, configured to repair the hole portion in the depth image by using a bilateral filtering algorithm in combination with the gradient information of the RGB image, so as to obtain a repaired depth image;

[0156] The model output unit is specifically used to input the repaired depth image and RGB image into a pre-trained image recognition model to obtain the obstacle category output by the image recognition model.

[0157] As a possible implementation manner, the adjustment unit further includes:

[0158] A conversion subunit, used for converting the obstacle into a bounding volume through an axial bounding box algorithm;

[0159] The adjustment subunit is further used to adjust the initial navigation path in real time based on the bounding volume, the road information and the obstacle avoidance algorithm to obtain a target navigation path.

[0160] As a possible implementation manner, before acquiring the environmental image of the moving target point through the sensor device of the robot, the device further includes:

[0161] A calibration unit is used to calibrate the sensor device through a calibration algorithm.

[0162] As a possible implementation manner, the semantic segmentation sub-model includes a SegNet model.

[0163] As a possible implementation manner, the target detection sub-model includes a YOLOV3 network model.

[0164] As a possible implementation manner, the acquisition unit is specifically used to:

[0165] Based on the building information and navigation requirements provided by the BIM model, global path planning is carried out to obtain the initial navigation path.

[0166] The present application also provides an autonomous navigation obstacle avoidance device for a robot in a dynamic scene, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the terminal may also include various network interfaces, power supplies and other components.

[0167] It should be noted that the autonomous navigation and obstacle avoidance device for a robot in a dynamic scene provided in an embodiment of the present application has the technical effects of any of the above embodiments, and the embodiments of the present application will not be described in detail here.

[0168] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0169] It should be noted that a computer-readable storage medium provided in an embodiment of the present application has the technical effects of any of the above embodiments, and the embodiments of the present application are not described in detail here.

[0170] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0171] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0172] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A robot autonomous navigation and obstacle avoidance method in a dynamic scene, characterized in that: The method comprises: Acquire an initial navigation path, wherein the initial navigation path includes a plurality of moving target points; Acquiring an environmental image of the moving target point through a sensor device of the robot; Inputting the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model; the obstacle information includes obstacle categories corresponding to the obstacles, and the obstacle categories include static obstacles and dynamic obstacles; The environment image includes a depth image and a red, green and blue (RGB) image; a bilateral filtering algorithm is used to repair the hole part in the depth image in combination with the gradient information of the RGB image to obtain a repaired depth image; the step of inputting the environment image into a pre-trained image recognition model to obtain obstacle information output by the image recognition model includes: inputting the repaired depth image and the RGB image into a pre-trained image recognition model to obtain an obstacle category output by the image recognition model; The hole part in the depth image is repaired by the following formula; ； ； In the formula, and are the pixel values ​​of pixel points p and q respectively, D q is the depth of pixel q; FI p represents the filtered depth map, G e represents the gradient calculation formula, Ω is the area covered by the convolution kernel, represents the distance weight, represents the gray value weight, represents the normalized weight coefficient, μ is the gradient derivative function; Determining an obstacle avoidance algorithm corresponding to the obstacle according to the obstacle category; During the navigation process, the initial navigation path is adjusted based on the obstacle information, the road information and the obstacle avoidance algorithm to obtain a target navigation path.

2. The method according to claim 1, characterized in that If the obstacle category is a static obstacle, the obstacle avoidance algorithm corresponding to the obstacle includes: a bidirectional search A* algorithm; If the obstacle category of the obstacle is a dynamic obstacle, the obstacle avoidance algorithm corresponding to the obstacle includes: an artificial potential field algorithm.

3. The method according to claim 2, characterized in that The obstacle information also includes: obstacle coordinates; if the obstacle category is a dynamic obstacle, adjusting the initial navigation path based on the obstacle information, the road information and the obstacle avoidance algorithm to obtain a target navigation path includes: Establish an elliptical field and determine the semi-axis length of the elliptical field; Determining whether the robot has a collision risk according to the semi-axis length and the obstacle coordinates; If it is determined that there is a collision risk, the initial navigation path is adjusted based on the road information and the obstacle avoidance algorithm to obtain a target navigation path.

4. The method according to claim 1, characterized in that The image recognition model includes a semantic segmentation sub-model and a target detection sub-model. The step of inputting the environment image into a pre-trained image recognition model to obtain obstacle information and road information output by the image recognition model includes: Inputting the environment image into the semantic segmentation sub-model, so that the semantic segmentation sub-model determines a first category of an object of a preset category in the environment image, and determines road information according to a pixel area whose first category is a road, to obtain the road information output by the semantic segmentation sub-model; The environment image is input into the target detection sub-model to obtain obstacle information output by the target detection sub-model.

5. The method according to claim 1, characterized in that The adjusting the initial navigation path based on the obstacle information, the road information and the obstacle avoidance algorithm to obtain a target navigation path includes: The obstacle is converted into a bounding volume by an axial bounding box algorithm; The initial navigation path is adjusted in real time based on the bounding volume, the road information and the obstacle avoidance algorithm to obtain a target navigation path.

6. The method according to claim 1, characterized in that Before acquiring the environment image of the moving target point by the sensing device of the robot, the method further includes: The sensor device is calibrated by a calibration algorithm.

7. The method according to claim 4, characterized in that The semantic segmentation sub-model includes a SegNet model.

8. The method according to claim 4, characterized in that The target detection sub-model includes a YOLOV3 network model.

9. The method according to claim 1, characterized in that: The obtaining of the initial navigation path comprises: Based on the building information provided by the Building Information Model (BIM) and navigation requirements, global path planning is performed to obtain the initial navigation path.

Images