Indoor transport robot lightweight lane line tracking method based on improved U-Net

Abstract

The invention relates to an indoor transport robot lightweight lane line tracking method based on improved U-Net. The method comprises the following steps: acquiring an RGB image in an indoor environment; inputting the acquired RGB image into the lightweight image segmentation model, and extracting lane line region information in the image; wherein the lightweight image segmentation model is obtained by training an improved U-Net model based on a pre-constructed training set; the improvement of the U-Net model comprises the following steps: replacing a common standard convolution structure with a depth separable convolution structure; and performing path extraction and motion planning based on the lane line area information, and driving the mobile robot to perform autonomous navigation driving along the lane line. According to the method, the dependence on hardware computing power can be reduced on the basis of ensuring the segmentation precision, and the adaptability and practicability of the algorithm in a resource-constrained platform are improved.

Description

Technical Field

[0001] This invention relates to the field of robot vision navigation technology, and in particular to a lightweight lane line following method for indoor transport robots based on an improved U-Net. Background Technology

[0002] With the rapid development of artificial intelligence technology, indoor transport robots are widely used in warehousing, logistics, and medical delivery scenarios. Their path planning and navigation capabilities directly affect the system's operational efficiency and intelligence level. Traditional line-following navigation methods mostly rely on manual markings such as magnetic strips and QR codes on the ground, which suffers from high deployment costs, poor flexibility, and weak environmental adaptability. With the development of computer vision and deep learning technologies, more and more research is attempting to use image recognition methods to achieve lane line recognition and path planning. Among them, image segmentation methods based on convolutional neural networks (CNNs) are widely used in lane line extraction tasks.

[0003] In existing technologies, U-Net, as a typical encoder-decoder neural network, has achieved good results in fields such as medical image segmentation and road sign extraction. However, the standard U-Net model has a large number of parameters and high computational complexity, making it unsuitable for resource-constrained indoor mobile robot systems. Furthermore, existing visual tracking methods are mostly focused on outdoor scenes, failing to adequately address issues such as lighting variations, ground reflections, and occlusion interference in complex indoor environments, resulting in robustness and real-time performance that cannot meet practical application requirements. Therefore, there is an urgent need for a lane line tracking method that is lightweight, has high segmentation accuracy, and is suitable for complex indoor environments to improve the navigation performance and environmental adaptability of indoor transport robots.

[0004] Image segmentation, as a key technology in computer vision, is widely used in scene understanding, object recognition, and path planning. Existing image segmentation algorithms can be broadly categorized into two types: traditional image segmentation methods and deep learning-based image segmentation methods. Traditional image segmentation methods primarily rely on manually defined image feature extraction rules, often employing thresholding, edge detection, clustering analysis, and region growing to divide image regions. Typical methods include histogram bimodal segmentation, iterative thresholding, and watershed algorithms. These methods have relatively low computational resource requirements, making them easy to deploy on resource-constrained devices. However, deep neural network models generally have a large number of parameters and high computational complexity, making them difficult to deploy and run in real-time for indoor mobile robot platforms with limited computing power, thus limiting their application effectiveness in practical tasks. Therefore, achieving lightweight network structures while ensuring segmentation accuracy has become a key technical approach to solving these problems. Summary of the Invention

[0005] The purpose of this invention is to provide a lightweight lane-following method for indoor transport robots based on an improved U-Net. This method involves acquiring RGB images of the indoor environment using a camera mounted on the front of the robot; inputting the acquired RGB images into a lightweight image segmentation neural network based on an improved U-Net structure to extract lane-line region information and achieve accurate lane-line segmentation; performing path extraction and motion planning based on the segmentation results to drive the robot to autonomously navigate along the lane lines. Finally, the tracking algorithm is integrated and validated by constructing an indoor transport robot platform.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A lightweight lane-line tracking method for indoor transport robots based on an improved U-Net includes:

[0008] Acquire RGB images in an indoor environment;

[0009] The acquired RGB image is input into a lightweight image segmentation model to extract lane line region information from the image; wherein, the lightweight image segmentation model is obtained by training an improved U-Net model based on a pre-built training set; the improved U-Net model includes: using a depthwise separable convolutional structure instead of a standard convolutional structure;

[0010] Based on the lane line area information, path extraction and motion planning are performed to drive the mobile robot to autonomously navigate along the lane lines.

[0011] Optionally, the pre-built training set includes: collected image samples with lane lines.

[0012] Optionally, training the improved U-Net model based on a pre-built training set includes:

[0013] For the pre-built training set, segmentation labels for lane lines and background regions are constructed;

[0014] The improved U-Net model is trained based on image data with segmentation labels.

[0015] Optionally, constructing segmentation labels for lane lines and background areas includes:

[0016] In the image data of the training set, lane lines are labeled as foreground objects, and the remaining areas are labeled as background.

[0017] During the label creation process, image annotation tools are used to manually annotate the original images, and the annotation information is converted into image format label data for network training.

[0018] Optionally, the expression for the depthwise separable convolutional structure is:

[0019] N c =K×K×C1×C2+C2

[0020] N m =K×K×C1+C1+C1×C2+C2

[0021] N c -N m =C1×(K×K×(C2-1)-C2-1)

[0022] Where K represents the spatial size of the convolution kernel, C1 is the number of channels in the input feature map, C2 is the number of channels in the output feature map, and N... c N represents the total number of parameters in a regular convolution operation. m This represents the total number of parameters for depthwise separable convolution operations.

[0023] Optionally, the improved U-Net model includes:

[0024] Three sets of basic coding modules are used in the encoding stage. Each set of coding modules consists of two DSConv operations and one downsampling operation. The downsampling operation halves the image spatial size while keeping the number of channels unchanged.

[0025] After passing through three sets of encoding modules, the input image feature map is sequentially encoded into intermediate feature representations with different channel numbers;

[0026] During the decoding stage, three basic decoding modules are also used. Each decoding module includes one upsampling operation and two DSConv operations. The upsampling operation is used to double the image space size while keeping the number of channels unchanged.

[0027] Optionally, the improved U-Net model also includes:

[0028] In the decoding stage, a skip connection mechanism is introduced to concatenate the feature map of the corresponding scale in the encoding stage with the upsampled feature map along the channel dimension, thereby achieving the fusion of low-level and high-level features.

[0029] The fused feature maps are then refined through two DSConv operations to produce image feature maps with different numbers of channels, and an indoor lane line segmentation result map is generated.

[0030] The beneficial effects of this invention are as follows:

[0031] This invention acquires RGB images of an indoor environment using a camera mounted on the front of an indoor transport robot. The acquired RGB images are then input into a lightweight image segmentation neural network based on an improved U-Net structure to extract lane line region information, achieving accurate lane line segmentation. Based on the segmentation results, path extraction and motion planning are performed to drive the mobile robot to autonomously navigate along the lane lines. Finally, the tracking algorithm is integrated and verified by constructing an indoor transport robot platform. This invention has the following technical effects:

[0032] (1) It overcomes the problem that the image segmentation results of the existing indoor mobile robot embedded with traditional image segmentation algorithm are easily affected by environmental factors such as changes in lighting in practical applications, which leads to unstable image segmentation results, and effectively improves the robustness and environmental adaptability of the tracking system.

[0033] (2) This paper solves the problem that existing deep learning image segmentation methods in indoor mobile robots have high hardware resource requirements and are difficult to achieve real-time control. By introducing a lightweight neural network structure, the segmentation accuracy is guaranteed while reducing the computational burden, thus meeting the comprehensive requirements of robot systems for real-time performance and low power consumption. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of a lightweight lane line following method for an indoor transportation robot based on an improved U-Net according to an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of a portion of the dataset collected in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of image segmentation labels for data samples collected according to an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the improved UNet module structure according to an embodiment of the present invention;

[0039] Figure 5 The original image and the model training results for 1, 5, 15, 10, 25, 50, 100, 200, and 500 times are shown in the embodiments of the present invention.

[0040] Figure 6 This is a schematic diagram illustrating the verification of the indoor mobile robot tracking method according to an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] This embodiment proposes a lightweight lane-following method for indoor transportation robots based on an improved U-Net, including:

[0044] Acquire RGB images in an indoor environment;

[0045] The acquired RGB image is input into a lightweight image segmentation model to extract lane line region information from the image; the lightweight image segmentation model is obtained by training an improved U-Net model based on a pre-built training set; the improved U-Net model includes: using a depthwise separable convolutional structure instead of a standard convolutional structure;

[0046] Based on lane line area information, path extraction and motion planning are performed to drive the mobile robot to autonomously navigate along the lane lines.

[0047] Specifically, this embodiment proposes a lightweight lane-following method for indoor transport robots based on an improved U-Net; such as Figure 1 As shown in the diagram, this embodiment uses a camera mounted on the front of the indoor transport robot to capture RGB images of the indoor environment. The captured RGB images are then input into a lightweight image segmentation neural network based on an improved U-Net structure to extract lane line region information, achieving accurate lane line segmentation. Based on the segmentation results, path extraction and motion planning are performed to drive the mobile robot to autonomously navigate along the lane lines. Finally, the tracking algorithm is integrated and verified by constructing an indoor transport robot platform.

[0048] Furthermore, the pre-built training set includes: collected image samples with lane lines.

[0049] Furthermore, training the improved U-Net model based on a pre-built training set includes:

[0050] For the pre-built training set, construct segmentation labels for lane lines and background regions;

[0051] The improved U-Net model is trained based on image data with segmentation labels.

[0052] Furthermore, the segmentation labels for lane lines and background areas are constructed as follows:

[0053] In the image data of the training set, lane lines are labeled as foreground objects, and the remaining areas are labeled as background.

[0054] During the label creation process, image annotation tools are used to manually annotate the original images, and the annotation information is converted into image format label data for network training.

[0055] Specifically, in this embodiment, pre-constructing the training set includes the following:

[0056] The dataset used in this embodiment consists of lane line image samples acquired by a camera mounted on an indoor mobile robot. Some of the acquired images are shown below. Figure 2 As shown in the figure. Since the image segmentation neural network model used is based on supervised learning methods in deep learning, high-quality label data plays a crucial role in the model training process. In this invention, the image segmentation task aims to accurately segment the lane line region from the image, thus requiring the construction of segmentation label images for lane lines and background regions. Specifically, this task can be classified as a binary classification image segmentation problem, where lane lines in the image are labeled as foreground objects, and the remaining areas are labeled as background. During the label creation process, the original image is manually labeled using the Labelme image annotation tool to generate corresponding JSON format annotation files. Subsequently, the "labelme_json_to_dataset" function in the Labelme tool is used to convert the annotation information into image format label data that can be used for network training. The annotation results are shown in the figure. Figure 3 As shown in the figure, this embodiment completed the annotation of 500 images, of which 400 were used as the training set and 100 as the test set for training and performance evaluation of the neural network model. The lane line image segmentation dataset constructed by the above method meets the training requirements of lightweight deep learning models, ensuring the accuracy and practicality of the system in actual deployment.

[0057] This embodiment uses a depthwise separable convolutional structure instead of a standard convolutional structure to improve the U-Net model; the details are as follows:

[0058] In the field of image segmentation, the U-Net model, as a classic deep neural network structure, was initially proposed for medical image segmentation tasks. This model has a U-shaped structure and belongs to a typical encoder-decoder architecture: in the encoding stage, four convolutional layers are used to progressively extract and downsample image features; in the decoding stage, four deconvolutional layers combined with skip connections are used to upsample and restore the feature maps, achieving high-resolution segmentation image recovery, and fusing high-level semantic information and low-level spatial detail information from the encoding stage, thereby improving segmentation accuracy. Although the U-Net model was initially applied to medical images, its design is suitable for image features with single targets and clear edges, thus demonstrating good applicability in lane line image segmentation tasks. The indoor lane line images involved in this invention have similar target segmentation features, and therefore, accurate lane line extraction can be achieved using the U-Net structure. However, the original U-Net model has a large number of parameters and a complex network structure, requiring high computational resources, making it unsuitable for deployment and operation on resource-constrained indoor mobile robot platforms, and failing to meet its requirements for real-time performance and low power consumption. To this end, the present invention has made structural lightweight improvements to the original U-Net model, which significantly reduces the network computational burden while maintaining segmentation accuracy as much as possible, thus making it more suitable for real-time tracking control tasks of indoor transport robots in practical application scenarios.

[0059] To achieve lightweight network models and improve deployment efficiency on resource-constrained platforms, this embodiment introduces the design concept of depthwise separable convolution. The position of the depthwise separable convolution is the same as that of ordinary convolution. This method significantly reduces the number of parameters and computational complexity of the model while maintaining feature extraction capabilities, and is widely used in the construction of lightweight convolutional neural networks. Traditional convolution operations simultaneously fuse features in the spatial dimension (i.e., region) and channel dimension during execution, essentially a filter stacking operation with high parameter redundancy. In contrast, depthwise separable convolution decomposes the conventional convolution process into two independent steps, processing the spatial and channel dimensions separately, thereby decoupling spatial and channel information. Specifically, depthwise separable convolution consists of the following two parts: the first part consists of C1 convolutions with a kernel size of K*K*1, and the second part consists of C2 convolutions with a kernel size of 1*1*C1. The parameter count is calculated as shown in Formula 2. The parameter reduction of depthwise separable convolution is shown in Formula 3.

[0060] N c =K×K×C1×C2+C2(1)

[0061] N m=K×K×C1+C1+C1×C2+C2(2)

[0062] N c -N m =C1×(K×K×(C2-1)-C2-1)(3)

[0063] In the formula, K represents the spatial size of the convolution kernel, C1 is the number of channels in the input feature map, C2 is the number of channels in the output feature map, and N... c N represents the total number of parameters in a regular convolution operation. m This represents the total number of parameters for depthwise separable convolution operations.

[0064] Furthermore, the improved U-Net model includes:

[0065] Three sets of basic coding modules are used in the encoding stage. Each set of coding modules consists of two DSConv operations and one downsampling operation. The downsampling operation halves the image spatial size while keeping the number of channels unchanged.

[0066] After passing through three sets of encoding modules, the input image feature map is sequentially encoded into intermediate feature representations with different channel numbers;

[0067] During the decoding stage, three basic decoding modules are also used. Each decoding module includes one upsampling operation and two DSConv operations. The upsampling operation is used to double the image space size while keeping the number of channels unchanged.

[0068] Furthermore, the improved U-Net model also includes:

[0069] In the decoding stage, a skip connection mechanism is introduced to concatenate the feature map of the corresponding scale in the encoding stage with the upsampled feature map along the channel dimension, thereby achieving the fusion of low-level and high-level features.

[0070] The fused feature maps are then refined through two DSConv operations to produce image feature maps with different numbers of channels, and an indoor lane line segmentation result map is generated.

[0071] Specifically, in this embodiment, the improved UNet model is as follows: Figure 4 As shown:

[0072] In this embodiment, a lightweight improvement to the classic U-Net image segmentation network structure was implemented, constructing an improved U-Net model based on Depthwise Separable Convolution (DSConv). This model still employs a U-shaped encoder-decoder structure, aiming to effectively reduce the model's parameter size and computational complexity while maintaining image segmentation accuracy, thereby improving the model's real-time performance on indoor mobile robot platforms. Figure 4 As shown, the improved model employs three sets of basic encoding modules in the encoding stage. Each module consists of two DSConv operations and one downsampling operation. The downsampling operation halves the image spatial size while maintaining the same number of channels. After passing through the three encoding modules, the input image feature map is sequentially encoded into intermediate feature representations with 64, 128, and 256 channels, respectively. In the decoding stage, three sets of basic decoding modules are also used, each consisting of one upsampling operation and two DSConv operations. The upsampling operation doubles the image spatial size while maintaining the same number of channels to recover high-resolution information.

[0073] To further mitigate potential image feature loss during downsampling, a skipconnection mechanism is introduced in the decoding stage to concatenate the feature maps of the corresponding scale from the encoding stage with the upsampled feature maps along the channel dimension, achieving the fusion of low-level and high-level features. The fused feature maps are then refined through two DSConv operations, ultimately outputting image feature maps with 256, 128, and 64 channels respectively, generating indoor lane line segmentation results. Furthermore, a multi-scale feature extraction mechanism is introduced in the initial stage of the network to improve the model's ability to perceive lane lines at different scales, enhancing the robustness and completeness of the segmentation results. Through the above structural design and convolutional operation optimization, the improved U-Net model proposed in this invention significantly reduces the computational overhead while maintaining good segmentation performance, making it more suitable for the combined real-time and computational resource requirements of indoor mobile robots.

[0074] In this embodiment, the environment configuration and training process based on the improved U-Net lightweight image segmentation algorithm are as follows:

[0075] To verify the effectiveness of the proposed improved U-Net lightweight image segmentation algorithm in this embodiment, model training and testing experiments were conducted on the constructed indoor mobile robot lane line image dataset. Furthermore, the algorithm was applied to target detection in air traffic control images for performance verification. This experiment was conducted on the Sugon cloud computing service platform. The hardware configuration of the platform cluster nodes is as follows: each node is equipped with one 32-core x86 architecture processor with a main frequency of 2.5GHz and one NVIDIA Tesla V100 graphics accelerator card; for memory, each node is configured with two 16GB DDR4 2666MHz ECC REG memory modules. Simultaneously, two sets of Sugon Parastor300S parallel storage systems were deployed to provide stable, high-speed, large-capacity data storage support.

[0076] In the improved U-Net lightweight image segmentation model constructed in this invention, the total number of iterations during training is set to 500, the mean squared error (MSE) loss function is selected, the initial learning rate of the optimizer is set to 0.01, and gradient descent is used for weight updates to effectively avoid gradient vanishing or gradient exploding problems during training. To achieve dynamic adjustment and convergence control during the training phase, a learning rate decay strategy is set: every 100 training cycles, the learning rate is scaled by a factor of 0.1 to further improve the convergence stability of the model. Image data needs to be uniformly normalized before being input into the network, and the input and output image sizes are both set to 160×160 to adapt to the network structure and accelerate training convergence.

[0077] After 500 iterations, the training error of the model stabilized at approximately 0.0015, and the test error was approximately 0.02, indicating that the model has good fitting ability and generalization performance on both the training and test sets. To further verify the performance evolution of the model under different training epochs, segmentation results were sampled for the same image after the 1st, 5th, 15th, 20th, 25th, 50th, 100th, 200th, and 400th training epochs. The experimental results are as follows: Figure 5 As shown in the figure, the results demonstrate that with the increase in training epochs, the model segmentation effect gradually improves, the edge contours become clearer, and the prediction accuracy is significantly improved, verifying the effectiveness and feasibility of the network structure and training strategy proposed in this invention.

[0078] Furthermore, this embodiment, based on the Jetson Nano embedded development platform, constructs an indoor mobile robot system integrating image data acquisition, image processing, and automatic control decision-making. This system is used to verify the effectiveness and practicality of the proposed improved U-Net image segmentation algorithm in actual line-following tasks. This embodiment uses the ability of the indoor transport robot to successfully navigate an S-shaped curve (240cm long and 240cm wide) with lane markings at a constant speed of 1m / s as the primary performance verification indicator; simultaneously, the frame rate (FPS, i.e., frames processed per second) of the image segmentation and control program is used as an auxiliary performance evaluation indicator. The overall flow of the line-following program is as follows: Figure 6 As shown in the figure. During the experiment, the indoor transport robot acquires images in real time through an onboard camera and inputs the images into the improved U-Net lightweight network proposed in this invention for lane line segmentation. Based on the segmentation results, path calculation and control command output are then performed to achieve automatic line-following control in an indoor environment. The entire system is deployed on a Jetson Nano platform, verifying the efficiency and stability of the proposed algorithm on embedded devices.

[0079] Experimental results show that the indoor mobile robot based on the algorithm of this invention can stably and accurately complete the automatic passage of S-shaped curves while maintaining a travel speed of 1 m / s. It has good path recognition ability and control robustness. The specific results are shown in Table 1.

[0080] Table 1 Comparison of experimental results

[0081]

[0082] Experimental results show that the improved U-Net image segmentation model proposed in this invention achieves significant weight reduction in its structural design, reducing the number of parameters by approximately 86% compared to the original U-Net model. This optimization effectively reduces the model's dependence on computational resources, enabling the deep learning-based image segmentation algorithm to run stably on embedded platforms such as Jetson Nano. Simultaneously, the optimized model maintains high segmentation accuracy while possessing high operating efficiency, fully meeting the real-time processing and autonomous tracking control requirements of indoor mobile robots for image segmentation tasks, demonstrating good engineering practical value and promising application prospects.

[0083] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A lightweight lane-following method for indoor transport robots based on an improved U-Net, characterized in that, include: Acquire RGB images in an indoor environment; The acquired RGB image is input into a lightweight image segmentation model to extract lane line region information from the image; wherein, the lightweight image segmentation model is obtained by training an improved U-Net model based on a pre-built training set; the improved U-Net model includes: using a depthwise separable convolutional structure instead of a standard convolutional structure; Based on the lane line area information, path extraction and motion planning are performed to drive the mobile robot to autonomously navigate along the lane lines.

2. The lightweight lane-line tracking method for indoor transportation robots based on improved U-Net as described in claim 1, characterized in that, The pre-built training set includes: collected image samples with lane lines.

3. The lightweight lane-line tracking method for indoor transportation robots based on improved U-Net according to claim 1, characterized in that, Training the improved U-Net model based on a pre-built training set includes: For the pre-built training set, construct segmentation labels for lane lines and background regions; The improved U-Net model is trained based on image data with segmentation labels.

4. The lightweight lane-line tracking method for indoor transportation robots based on improved U-Net according to claim 1, characterized in that, The labels for separating lane lines from the background area include: In the training set of image data, lane lines are labeled as foreground objects, and the remaining areas are labeled as background. During the label creation process, image annotation tools are used to manually annotate the original images, and the annotation information is converted into image format label data for network training.

5. The lightweight lane-line tracking method for indoor transportation robots based on improved U-Net according to claim 1, characterized in that, The expression for the depthwise separable convolutional structure is: N c =K×K×C1×C2+C2 N m =K×K×C1+C1+C1×C2+C2 N c -N m =C1×(K×K×(C2-1)-C2-1) Where K represents the spatial size of the convolution kernel, C1 is the number of channels in the input feature map, C2 is the number of channels in the output feature map, and N... c N represents the total number of parameters in a regular convolution operation. m This represents the total number of parameters for depthwise separable convolution operations.

6. The lightweight lane-line tracking method for indoor transportation robots based on improved U-Net according to claim 1, characterized in that, The improved U-Net model includes: Three sets of basic coding modules are used in the encoding stage. Each set of coding modules consists of two DSConv operations and one downsampling operation. The downsampling operation halves the image spatial size while keeping the number of channels unchanged. After passing through three sets of encoding modules, the input image feature map is sequentially encoded into intermediate feature representations with different channel numbers; During the decoding stage, three basic decoding modules are also used. Each decoding module includes one upsampling operation and two DSConv operations. The upsampling operation is used to double the image space size while keeping the number of channels unchanged.

7. The lightweight lane-following method for indoor transport robots based on improved U-Net according to claim 6, characterized in that, The improved U-Net model also includes: In the decoding stage, a skip connection mechanism is introduced to concatenate the feature map of the corresponding scale in the encoding stage with the upsampled feature map along the channel dimension, thereby achieving the fusion of low-level and high-level features. The fused feature maps are then refined through two DSConv operations to produce image feature maps with different numbers of channels, and an indoor lane line segmentation result map is generated.

Images