Obstacle recognition model and training method, road obstacle recognition method, autonomous mobile robot and navigation method

Abstract

The embodiment of the present invention provides an obstacle recognition model and training method, a road obstacle recognition method, an autonomous mobile robot and a navigation method. The obstacle recognition model and training method include the following steps: S1, initializing the obstacle recognition model, wherein the obstacle recognition model includes a feature extraction network, a residual connection network and a dense connection network; S2, constructing a training data set, wherein the training data set includes multiple road images, each road image includes an obstacle area and a vehicle drivable area as labels; S3, training the obstacle recognition model based on the training data set, calculating the loss based on a preset loss function, and updating the model parameters. Obstacles are learned through deep learning methods, and the recognition accuracy of obstacles and drivable areas is improved, thereby further enhancing the accuracy and reliability of navigation.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to an obstacle recognition model and training method, a road obstacle recognition method, an autonomous mobile robot and a navigation method. Background Art

[0002] Automated Guided Vehicle (AGV) navigation systems mostly use preset paths or simple obstacle avoidance logic for navigation. These systems may face challenges in complex outdoor road conditions because the unstructured and dynamically changing characteristics of the outdoor environment make it difficult for traditional preset paths and simple obstacle avoidance logic to adapt. The complexity of the outdoor environment includes but is not limited to weather changes, terrain undulations, and movement of obstacles, all of which affect the navigation accuracy and safety of the AGV. Therefore, for complex outdoor road conditions, the AGV navigation system requires more advanced navigation technology and algorithms to ensure accuracy and safety.

[0003] At present, the navigation methods of AGV mainly include electromagnetic navigation, magnetic nail navigation, laser navigation, QR code navigation, inertial navigation, etc. Among them, electromagnetic navigation and magnetic nail navigation are traditional navigation methods. They navigate through preset paths and simple obstacle avoidance logic and are suitable for indoor structured environments. Laser navigation, QR code navigation and inertial navigation provide higher accuracy and adaptability, especially in complex outdoor road conditions.

[0004] For example, laser navigation uses a laser scanner installed on the AGV body to determine the position and direction of the AGV by emitting a laser beam and receiving the reflected signal from the reflector. This method can provide higher navigation accuracy in dynamic environments. However, this navigation method has higher hardware requirements.

[0005] QR code navigation sets QR code labels on the path, and the AGV determines its position and next action by reading these QR codes. This method is suitable for scenarios that require high-precision navigation, but it also requires pre-marking of the environment.

[0006] Inertial navigation uses sensors such as gyroscopes and accelerometers to measure the motion state of the AGV and calculates the position and direction information. This method shows good adaptability and stability in complex outdoor road conditions, but lacks environmental perception. Therefore, the existing technology lacks an effective and economical navigation method for AGVs in real dynamic environments. Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention provides an obstacle recognition model and training method, a road obstacle recognition method, an autonomous mobile robot and a navigation method, which mainly solve the problem that the prior art lacks an effective and economical navigation method for AGVs in real dynamic environments.

[0008] The objective of the present invention is achieved through the following solutions:

[0009] In a first aspect, according to an embodiment of the present invention, a training method for an obstacle recognition model is provided, wherein the obstacle recognition model is used to recognize obstacles and vehicle drivable areas in a given road image, and comprises the following steps: S1, initializing the obstacle recognition model, wherein the obstacle recognition model comprises a feature extraction network, a residual connection network, and a dense connection network; S2, constructing a training data set, wherein the training data set comprises a plurality of road images, each road image comprises an obstacle area and a vehicle drivable area as labels; S3, training the obstacle recognition model based on the training data set, calculating the loss based on a preset loss function, and updating the model parameters.

[0010] According to an embodiment of the present invention, the feature extraction network includes multiple convolutional layers, and each convolutional layer is connected to a batch normalization layer and an activation function layer.

[0011] According to an embodiment of the present invention, the residual connection network includes a residual block, and the network input of each layer in the residual block is the sum of the network output of the previous layer and the network input of the previous layer.

[0012] According to an embodiment of the present invention, the densely connected network includes dense blocks, and the network input of each layer in the dense block is the network output of all previous layers.

[0013] According to an embodiment of the present invention, the preset loss function is cross entropy loss.

[0014] In a second aspect, according to another embodiment of the present invention, there is provided an obstacle recognition model obtained by using the training method as described in the first aspect, wherein the image recognition model includes a feature extraction network, a residual connection network and a dense connection network.

[0015] In a third aspect, according to another embodiment of the present invention, a road obstacle identification method is provided, comprising the following steps: P1, acquiring a road surface image; P2, based on the acquired road surface image, using the obstacle identification model as described in the second aspect to identify obstacle areas and vehicle drivable areas in the road surface image.

[0016] In a fourth aspect, according to another embodiment of the present invention, there is provided a navigation method for an autonomous mobile robot, comprising the following steps: P1, obtaining a destination position and obtaining a road surface image around a vehicle in real time; P2, using a road obstacle recognition method as described in the third aspect to identify obstacle areas and vehicle drivable areas in road surface images; P3, based on the destination position, the obstacle area and the vehicle drivable area, using a preset path planning method to calculate a planned path, and updating the path as the road surface image is acquired in real time.

[0017] According to another embodiment of the present invention, the preset path planning method includes a simulated annealing algorithm, an artificial potential field method, a fuzzy logic algorithm, and a taboo search algorithm.

[0018] In a fifth aspect, according to another embodiment of the present invention, there is provided an autonomous mobile robot, comprising an image sensor and a control system, wherein the control system adopts a navigation method of an autonomous mobile robot as described in the fourth aspect to travel.

[0019] Compared with the prior art, the present invention has the following beneficial effects: by learning obstacles through deep learning methods, it can automatically learn the complex features of images and improve the recognition accuracy of obstacles and drivable areas; thanks to the aforementioned accurate recognition of obstacles, during navigation, the surrounding environment is fully understood and the optimal path can be planned; and real-time path updates are performed by acquiring images in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The embodiments of the present invention are further described below with reference to the accompanying drawings:

[0021] Figure 1 The figure is a flow chart of the obstacle recognition model training method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] As mentioned in the background technology, the prior art lacks an effective and economical navigation method for AGV in real dynamic environment. Figure 1As shown, an embodiment of the present invention provides a training method for an obstacle recognition model, wherein the obstacle recognition model is used to recognize obstacles and vehicle drivable areas in a given road image, and includes the following steps: S1, initializing the obstacle recognition model, wherein the obstacle recognition model includes a feature extraction network, a residual connection network, and a dense connection network; S2, constructing a training data set, wherein the training data set includes multiple road images, each road image includes an obstacle area and a vehicle drivable area as labels; S3, training the obstacle recognition model based on the training data set, calculating the loss based on a preset loss function, and updating the model parameters. By learning obstacles through a deep learning method, it is possible to automatically learn the complex features of the image, improve the recognition accuracy of obstacles and drivable areas, and further enhance the accuracy and reliability of navigation.

[0024] In detail, the training dataset contains outdoor road images and their finely annotated segmentation maps, which clearly distinguish obstacles from drivable areas. In addition, in order to improve the generalization ability of the model, data augmentation is also performed on the images, including random cropping, rotation, color transformation, etc. In addition, the images are preprocessed (scaling, normalization, etc.) according to the requirements of the model input. In addition, the model outputs a segmentation map of the same size as the input image, where the value of each pixel represents the probability of the pixel belonging to an obstacle (1) or a drivable area (0).

[0025] In addition, in order to train a robust deep learning model, it is necessary to prepare a large-scale, diverse data set and simulate different environmental conditions (including but not limited to weather and lighting) through data enhancement techniques (such as random rotation, flipping, and scaling). By creating a detailed model optimization and parameter adjustment, the embodiments of the present invention can further improve the accuracy and real-time performance of obstacle detection.

[0026] According to an embodiment of the present invention, the feature extraction network includes multiple convolutional layers, each of which is connected to a batch normalization layer and an activation function layer. The convolutional layer uses a 3x3 filter with a step size of 1 and a padding of 1. The activation function is a ReLU activation function.

[0027] In order to promote the training of deep networks and prevent the gradient vanishing problem, according to an embodiment of the present invention, the residual connection network includes a residual block, and the network input of each layer in the residual block is the sum of the network output of the previous layer and the network input of the previous layer. The residual block contains 1x1, 3x3 and 1x1 convolution operations to achieve efficient transmission and fusion of features. Similarly, each convolution layer is connected to a batch normalization layer and an activation function layer.

[0028] In order to enhance the transferability of features, according to an embodiment of the present invention, the densely connected network includes dense blocks, and the network input of each layer in the dense block is the network output of all previous layers. Similarly, the dense block contains 1x1, 3x3 and 1x1 convolution operations to achieve efficient transfer and fusion of features, and each convolution layer is connected to a batch normalization layer and an activation function layer.

[0029] In addition, the obstacle recognition model also includes an upsampling layer. Upsampling (bilinear interpolation or transposed convolution) is used at the end of the network to restore the spatial resolution of the feature map, so that the output layer finally outputs a segmentation map with the same resolution as the input image, where each pixel contains the probability of two types of labels (obstacles and drivable areas). In detail, the upsampling layer gradually reduces the number of feature maps while increasing their spatial dimensions until it matches the input image size.

[0030] During the training process, according to an embodiment of the present invention, the preset loss function is the cross entropy loss, and the loss is calculated for each pixel. In addition, the Adam optimizer is also used, which has the characteristics of adaptive learning rate. Some other training parameters are as follows: batch size, 32; number of iterations, 10,000 times; early stopping, if the performance on the validation set does not improve within 10 consecutive epochs, stop training early.

[0031] According to another embodiment of the present invention, an obstacle recognition model is provided, which is obtained by using the training method as described above, wherein the image recognition model includes a feature extraction network, a residual connection network and a dense connection network.

[0032] According to another embodiment of the present invention, a road obstacle recognition method is provided, comprising the following steps: P1, acquiring a road image; P2, based on the acquired road image, using the obstacle recognition model as described above to identify obstacle areas and vehicle drivable areas in the road image.

[0033] According to another embodiment of the present invention, there is provided a navigation method for an autonomous mobile robot, comprising the following steps: P1, acquiring a destination position and acquiring a road surface image around a vehicle in real time; P2, using a road obstacle recognition method as described above to recognize an obstacle area and a vehicle drivable area in the road surface image; P3, based on the destination position, the obstacle area and the vehicle drivable area, using a preset path planning method to calculate a planned path, and updating the path as the road surface image is acquired in real time.

[0034] According to another embodiment of the present invention, the preset path planning method includes a simulated annealing algorithm, an artificial potential field method, a fuzzy logic algorithm, and a taboo search algorithm.

[0035] The difference from the traditional obstacle avoidance logic is that obstacle avoidance usually refers to planning a path to avoid obstacles when the location of obstacles is known. The navigation method of the embodiment of the present invention not only includes obstacle avoidance, but also needs to consider the global path planning from the starting point to the end point, including but not limited to obstacle avoidance, road selection, speed control and other aspects.

[0036] The advantage of the navigation method of the embodiment of the present invention is that it can handle more complex scenes and can better handle dynamically changing environments. In addition, since the traditional intermediate steps are omitted, the algorithm has a faster response speed and can more flexibly adapt to real-time changing traffic conditions.

[0037] According to another embodiment of the present invention, an autonomous mobile robot is provided, including an image sensor and a control system, wherein the control system adopts a navigation method of an autonomous mobile robot as described above to travel. The vehicle control system receives the path planning result of the navigation method, controls the steering, acceleration and braking of the autonomous mobile robot, and realizes autonomous obstacle avoidance.

[0038] In addition, multimodal information fusion technology can also be combined. By combining RGB-D depth cameras, lidar and IMU units, the robot can obtain multimodal information such as three-dimensional visual information and posture angles, which can assist in accurate obstacle detection in complex environments.

[0039] In addition, in adverse weather conditions, such as fog or low-light environments, the reliability of the navigation method can be further enhanced by enhancing the images through the Image Adaptive YOLO (IA-YOLO) framework, and predicting and applying image enhancement parameters through the Differentiable Image Processing (DIP) module and a small convolutional neural network (CNN-PP).

[0040] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0041] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0042] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A training method for an obstacle recognition model, wherein the obstacle recognition model is used to recognize obstacles and vehicle drivable areas in a given road image, characterized in that: The following steps are involved: S1. Initializing the obstacle recognition model, wherein the obstacle recognition model includes a feature extraction network, a residual connection network and a dense connection network; S2. constructing a training data set, wherein the training data set includes a plurality of road images, each road image includes an obstacle area and a vehicle drivable area as labels; S3. Training the obstacle recognition model based on the training data set, calculating the loss based on a preset loss function, and updating the model parameters.

2. The method for training an obstacle recognition model according to claim 1, characterized in that: The feature extraction network includes multiple convolutional layers, and each convolutional layer is connected to a batch normalization layer and an activation function layer.

3. The method for training an obstacle recognition model according to claim 1, characterized in that: The residual connection network includes a residual block, and the network input of each layer in the residual block is the sum of the network output of the previous layer and the network input of the previous layer.

4. The method for training an obstacle recognition model according to claim 1, characterized in that: The densely connected network includes dense blocks, and the network input of each layer in the dense block is the network output of all previous layers.

5. The method for training an obstacle recognition model according to claim 1, characterized in that: The default loss function is cross entropy loss.

6. An obstacle recognition model, characterized in that: The image recognition model is obtained by using the training method described in any one of claims 1 to 5, wherein the image recognition model includes a feature extraction network, a residual connection network and a dense connection network.

7. A method for identifying road obstacles, characterized in that: The following steps are involved: P1. Obtain road surface image; P2. Based on the acquired road surface image, the obstacle recognition model as claimed in claim 6 is used to recognize the obstacle area and the vehicle drivable area in the road surface image.

8. A navigation method for an autonomous mobile robot, characterized in that: The following steps are involved: P1, obtain the destination location and obtain the road image around the vehicle in real time; P2. Using a road obstacle recognition method as claimed in claim 7 to recognize obstacle areas and vehicle drivable areas in a road image; P3. Based on the destination location, the obstacle area and the vehicle drivable area, a preset path planning method is used to calculate the planned path, and the path is updated with the road surface image acquired in real time.

9. The navigation method of an autonomous mobile robot according to claim 8, characterized in that: The preset path planning methods include simulated annealing algorithm, artificial potential field method, fuzzy logic algorithm, and taboo search algorithm.

10. An autonomous mobile robot, comprising an image sensor and a control system, characterized in that: The control system drives using a navigation method of an autonomous mobile robot as described in any one of claims 8-9.

Images