A lane area calibration method and system based on edge computing platform
By using a multi-scale U-shaped network of edge computing platforms on the emergency lane lane on highways, the problems of inaccuracy and slow processing speed of traditional methods under environmental changes are solved, and high accuracy and robustness are achieved lane inspection ensures traffic safety.
Patent Information
- Application Number
- CN202411161488.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-23
AI Technical Summary
Traditional lane detection methods are susceptible to environmental changes in the emergency lane of highways, resulting in inaccurate and inconsistent calibration results, and slow processing speed, making it difficult to identify vehicle break-in behavior in a timely manner, affecting traffic safety.
Using a multi-scale U-shaped network based on edge computing platform for lane area calibration, mosaic data augmentation and multi-scale feature extraction, combined with encoder, decoder and softmax classifier, the network is built and trained to achieve high accuracy and robustness. lane area calibration.
It significantly improves the accuracy and robustness of lane area calibration, reduces manual operation dependence, reduces network bandwidth burden, and can quickly identify vehicle intrusion behavior in different environments to ensure traffic safety.
Smart Images

Figure CN118691907B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a lane area calibration method and system based on an edge computing platform and a multi-scale U-shaped network. Background Art
[0002] In the emergency lane of the highway, it is a common and dangerous phenomenon for vehicles to break into it at will, which not only violates traffic regulations, but also may cause serious traffic accidents and affect the passage of rescue vehicles in emergency situations. Therefore, the monitoring and management of the emergency lane of the highway is particularly important.
[0003] The area calibration of traditional lane detection methods often relies on manual operation, which may perform well under standard driving conditions, but is easily affected by subjective factors when dealing with environmental changes (such as weather changes, lighting conditions, road conditions, etc.). This process is not only time-consuming and labor-intensive, increasing operating costs, but also leads to inaccurate and inconsistent calibration results. These methods do not handle the situation of vehicles intruding into the emergency lane in a timely manner, especially in high-speed and high-risk environments.
[0004] Deep learning technology, especially convolutional neural networks (CNNs), provides new possibilities for improving the accuracy and robustness of lane area calibration. However, lane area calibration directly applied to emergency lanes on highways still faces challenges. Due to the randomness of vehicle behavior and the variability of environmental conditions on emergency lanes, lane calibration methods are required to work at different scales to adapt to rapidly changing scenarios. At the same time, they must be able to process and analyze a large amount of traffic data in real time, quickly and accurately identify the behavior of vehicles arbitrarily entering emergency lanes, and take corresponding measures in a timely manner, such as issuing warnings and notifying relevant departments, to ensure traffic safety. Therefore, a method that can adapt to rapidly changing scenarios and has high lane detection accuracy and robustness is studied, especially when dealing with key scenarios such as emergency lanes on highways, which can adapt to environmental changes and ensure high accuracy and reliability of lane calibration. Summary of the invention
[0005] In order to overcome the problems existing in the background technology, the present invention provides a lane area calibration method based on an edge computing platform, which can not only significantly reduce the dependence on manual operation and improve the efficiency and accuracy of calibration, but also has low latency, low network burden and sufficient adaptive capabilities. The main contents of the present invention are as follows:
[0006] Step 1: obtain the image set to be processed, perform mosaic data enhancement on the image set and construct a training data set;
[0007] Step 2: Build and train a multi-scale U-shaped network for lane area calibration;
[0008] The multi-scale U-shaped network includes an encoder, a decoder, and a classifier, the encoder includes n dual convolution units and n pooling operation units, each dual convolution unit is followed by a pooling operation unit, the dual convolution unit is used to extract image features, and the pooling operation unit is used to reduce the dimension of data;
[0009] The decoder includes n composite upsampling modules and n composite splicing modules, each composite upsampling module is followed by a composite splicing module, the composite upsampling module is used to transform the dimension of the input feature map, the dual convolution unit is connected to the composite splicing module in the form of a U-Net network, and the composite splicing module splices the feature maps processed by the dual convolution unit and the composite upsampling module to obtain a new feature map;
[0010] Finally, the lane line area is obtained through the softmax pixel-by-pixel classifier, including the emergency lane area, lane line area, and vehicle driving area, as well as the prediction probabilities of these three areas;
[0011] Step 3: Deploy the trained multi-scale U-shaped network on the edge computing platform to calibrate the lane area of the image to be tested.
[0012] Furthermore, in step 1, a gantry-type road monitoring camera is used to collect videos of real scenes in different natural environments, and then the collected videos are framed, and video frames are extracted at certain time intervals, and these video frames are used as a picture set; mosaic data enhancement is performed on the picture set, and the original pictures are first scaled at different scales, and the scaled pictures are randomly spliced so that the spliced pictures meet the size of the original pictures to form a training data set.
[0013] Furthermore, the double convolution unit consists of two convolutional layers, namely the first convolutional layer and the second convolutional layer. The two convolutional layers are operated in series, and the convolution kernel size of the first and second convolutional layers is , the step size is 1, the padding is 0, the number of output channels is different, the activation function of the first convolution layer uses the ReLU activation function, and the activation function of the second convolution layer uses the sigmoid activation function. When the feature map passes through the first convolution layer, the dimension of the feature map is reduced by two dimensions, and when it passes through the second convolution layer, it is reduced by two dimensions again. In general, the feature map is reduced by 4 dimensions when it passes through the double convolution unit.
[0014] Furthermore, the composite upsampling module includes a double convolution unit and an upsampling unit. When the feature map passes through the composite upsampling module, the dimension of the feature map is first reduced by 4 dimensions, and then after passing through the upsampling unit, the dimension of the feature map is doubled.
[0015] Furthermore, the composite stitching module is composed of an adaptive cropping unit and a stitching unit. The composite stitching module accepts the input of two feature maps. After the two feature maps are input into the composite stitching module, they first pass through the adaptive cropping unit, which compares the sizes of the two feature maps and uses the smaller feature map as a template to crop the larger feature map to obtain two feature maps of the same size. Finally, the two feature maps are stitched to obtain a new feature map.
[0016] Furthermore, the processing of the multi-scale U-shaped network is as follows:
[0017] The feature maps output by n double convolution units are P1, P2, P3, P4...Pn. After the input image is reduced by 4 dimensions through the first double convolution unit, the feature map P1 is obtained. After the feature P1 passes through the pooling unit, the dimension is halved. After passing through the second double convolution unit, the feature map P2 is reduced by 4 dimensions. The features P3...Pn are obtained in sequence. The feature output of the nth pooling unit is recorded as C0.
[0018] The inputs of the composite splicing module are the output of the previous composite upsampling module and features Pn...P3, P2, P1. After feature C0 passes through the composite upsampling module, the dimension of C0 is first reduced by 4 dimensions, and then after passing through the upsampling unit, the dimension is doubled to obtain feature m1. After m1 is output from the composite upsampling module, it is input into the first composite splicing module, and is adaptively trimmed with another input feature Pn of the composite splicing module, and then spliced. The output feature is C1. After feature C1 passes through the second composite upsampling module, it is spliced with feature Pn-1 in the second composite upsampling module to obtain feature C2, and the final feature Cn to be predicted is obtained in turn; finally, the lane line area is obtained through the softmax pixel-by-pixel classifier.
[0019] Furthermore, a comprehensive loss function consisting of a classification loss function and a segmentation loss function is used to train the multi-scale U-type network, where the classification loss function is the cross entropy loss function and the segmentation loss function is the Dice loss function.
[0020] Furthermore, the calculation formula of the comprehensive loss function L is as follows:
[0021]
[0022] Among them, λ and β are classification loss functions respectively. And the segmentation loss function The weight of x i represents the i-th sample in the training data set, p(x i ) means that for sample x iThe predicted probability, K is the total number of preset categories, a total of 3 categories are preset, N is the total number of samples in the training data set, C represents the predicted category, if the prediction is correct, C=1, if the prediction is wrong, C=0, y ic Indicates the conversion of sample targets into binary code. i represents the sample number. The sample target refers to the three classes in a sample image i. The three targets are i1, i2, and i3. The target is converted into binary code, which is y ic , the values of ic are i1, i2, i3. For category 1, the corresponding vector is [1, 0, 0]; for category 2, the corresponding vector is [0, 1,0]; for category 3, the corresponding vector is [0, 0, 1]; X and Y represent the predicted mask and actual mask of the sample target respectively, and ∩ represents the intersection.
[0023] The present invention also provides a lane area calibration system based on an edge computing platform, comprising the following units:
[0024] A data processing unit, used to obtain a set of images to be processed, perform mosaic data enhancement processing on the set of images and construct a training data set;
[0025] Network construction and training unit, used to build and train a multi-scale U-shaped network for lane area calibration;
[0026] The multi-scale U-shaped network includes an encoder, a decoder, and a classifier, the encoder includes n dual convolution units and n pooling operation units, each dual convolution unit is followed by a pooling operation unit, the dual convolution unit is used to extract image features, and the pooling operation unit is used to reduce the dimension of data;
[0027] The decoder includes n composite upsampling modules and n composite splicing modules, each composite upsampling module is followed by a composite splicing module, the composite upsampling module is used to transform the dimension of the input feature map, the dual convolution unit is connected to the composite splicing module in the form of a U-Net network, and the composite splicing module splices the feature maps processed by the dual convolution unit and the composite upsampling module to obtain a new feature map;
[0028] Finally, the lane line area is obtained through the softmax pixel-by-pixel classifier, including the emergency lane area, lane line area, and vehicle driving area, as well as the prediction probabilities of these three areas;
[0029] The lane area calibration unit is used to deploy the trained multi-scale U-shaped network on the edge computing platform to realize the lane area calibration of the image to be tested.
[0030] The present invention has the following beneficial effects compared with the prior art:
[0031] The multi-scale U-shaped network improves the model's ability to recognize vehicles of different sizes and complex scenes by extracting feature information at different scales, and enhances the accuracy and robustness of lane area calibration, especially in unstable environments such as lighting changes and weather changes. By processing data locally on the edge computing platform, the need to transmit large amounts of raw data to the cloud or data center can be reduced, thereby reducing the burden on network bandwidth and potentially reducing operating costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 A flow chart of the method provided by the present invention;
[0033] Figure 2 A structural diagram of a multi-scale U-shaped network is provided for the present invention;
[0034] Figure 3 It is a double convolution unit structure diagram;
[0035] Figure 4 This is a structural diagram of the composite splicing module. DETAILED DESCRIPTION
[0036] The technical solution of the present invention will be fully described below in conjunction with the accompanying drawings in the present invention.
[0037] like Figure 1 As shown, an embodiment of the present invention provides a lane area calibration method based on an edge computing platform, comprising the following steps:
[0038] Step 1: Use a gantry-type monitoring camera to obtain a set of images to be processed, perform mosaic data enhancement on the image set and construct a data set;
[0039] Specifically, a gantry-type road monitoring camera is used to collect videos of real scenes in different natural environments. The collected videos are then framed and video frames are extracted at a time interval of 0.5 seconds. These video frames are used as data set images for subsequent training. Mosaic data enhancement is performed on the image set. The original images are first scaled at different scales, and the scaled images are randomly spliced so that the spliced images meet the size of the original images to form a data set.
[0040] Import the dataset images into the Labelme tool, mark the lane lines, lane driving areas and emergency lanes. After labeling, a json file will be generated for each image, and then a mask file will be generated through the json file. The dataset images and the mask file are recombined to obtain the dataset, in which the ratio of the training set to the validation set is 8:2.
[0041] Step 2: Build and train a multi-scale U-shaped network for lane area calibration;
[0042] In view of the low latency, high bandwidth and limited computing power of the edge computing platform, we designed a network structure consisting of an encoder and a decoder. This structure enables efficient lane detection and calibration tasks even with limited computing resources, and dynamically adjusts parameters based on real-time data to adapt to various road and environmental conditions.
[0043] Step 2-1, the encoder includes n double convolution units and n pooling operation units, one double convolution unit is followed by one pooling operation unit, and the two units are alternately connected in series. In this embodiment, n is 4 for the best effect;
[0044] Step 2-1-1, the double convolution unit consists of two convolutional layers, namely the first convolutional layer and the second convolutional layer. The two convolutional layers are operated in series. The convolution kernel size of the first and second convolutional layers is , the stride is 1, the padding is 0, the number of output channels is different, the activation function of the first convolution layer uses the ReLU activation function, and the activation function of the second convolution layer uses the sigmoid activation function. When the feature map passes through the first convolution layer, the dimension of the feature map is reduced by two dimensions, and when it passes through the second convolution layer, it is reduced by two dimensions again. In general, when the feature map passes through the double convolution unit, the feature map is reduced by 4 dimensions;
[0045] In the pooling operation unit, the size of the pooling window is set to 2x2, the step size of the pooling window is set to 2, and the maximum value of the adjacent 4 elements is used as the output. This maximum pooling operation is used to downsample the input data, which can reduce the dimension of the data and halve the dimension of the feature map, reducing the amount of calculation while retaining the feature information.
[0046] Step 2-1-2, the feature maps output by n double convolution units are P1, P2, P3, P4...Pn, respectively. After the input image passes through the first double convolution unit and reduces 4 dimensions, the feature map P1 is obtained. After the feature P1 passes through the pooling operation unit, the dimension is halved. After passing through the second double convolution unit, the feature map P2 is reduced by 4 dimensions. This process is repeated many times to obtain the features P3...Pn. The feature output of the nth pooling operation unit is recorded as C0.
[0047] Step 2-2, on the decoder, based on the idea of the U-Net decoding structure, a fast-executing composite upsampling module and a composite splicing module are constructed. The decoder structure includes n composite upsampling modules and n composite splicing modules, where one composite upsampling module is followed by a composite splicing module. The two modules are alternately connected in series, which simplifies the deconstruction of the network and enhances the real-time processing capability.
[0048] Step 2-2-1, a composite upsampling module, including a double convolution unit and an upsampling unit. When the feature map passes through the composite upsampling module, the dimension of the feature map is first reduced by 4 dimensions (the same as the double convolution module in step 2-1-1), and then after passing through the upsampling unit, the dimension of the feature map is doubled;
[0049] The composite splicing module is composed of an adaptive cropping unit and a splicing unit. The composite splicing module accepts the input of two feature maps. After the two feature maps are input into the composite splicing module, they first pass through the adaptive cropping unit, which compares the sizes of the two feature maps, uses the smaller feature map as a template, and crops the larger feature map to obtain two feature maps of the same size. Finally, the two feature maps are spliced to obtain a new feature map.
[0050] In step 2-2-2, the inputs of the composite splicing module are the output of the last composite upsampling module and features Pn...P3, P2, P1. After feature C0 passes through the composite upsampling module, the dimension of C0 is first reduced by 4 dimensions, and then after passing through the upsampling unit, the dimension is doubled to obtain feature m1. After m1 is output from the composite upsampling module, it is input into the first composite splicing module, and is adaptively trimmed with another input feature Pn of the composite splicing module, and then spliced. The output feature is C1. After feature C1 passes through the second composite upsampling module, it is spliced with feature Pn-1 in the second composite upsampling module to obtain feature C2. This step is repeated multiple times to obtain the final feature Cn that needs to be predicted. The lane line area is obtained through the softmax pixel-by-pixel classifier, including three types of areas: emergency lane area, lane line area, and vehicle driving area, as well as the prediction probabilities of these three areas.
[0051] Step 2-3, during the training process of the multi-scale U-shaped network, the network learns by learning the relationship between the images in the training set and the corresponding masks. Subsequently, the validation set is used to evaluate its performance. The classification task involves classifying the segmented areas and dividing them into predefined categories. Specifically, three categories are set: emergency lanes, lane lines, and normal driving areas. The cross entropy is used as the classification loss function L cls , the segmentation loss uses the Dice loss function; in order to achieve efficient and centralized model training, we adopted a comprehensive loss function strategy, which combines the classification loss and segmentation loss into a total loss function through different weights. Doing so not only promotes information sharing between different tasks, but also simplifies the entire training process. By integrating all optimization objectives into one loss value, the model can simultaneously optimize segmentation accuracy and classification accuracy during training, thereby achieving more efficient learning. The total loss is L:
[0052]
[0053] In the total loss function above, λ and β are the weights of the classification loss function and the segmentation loss function respectively, and x i represents a sample of the data set, p(x i ) means that for sample x i The predicted probability, K is the total number of preset categories, a total of 3 categories are preset, N is the total number of samples in the data set, C represents the predicted category, if the prediction is correct, C=1, if the prediction is wrong, C=0, y ic Indicates the conversion of sample targets into binary code. i represents the sample number. The sample target refers to the three classes in a sample image i. The three targets are i1, i2, and i3. The target is converted into binary code, which is y ic , the values of ic are i1, i2, i3. For category 1, the corresponding vector is [1, 0, 0]; for category 2, the corresponding vector is [0, 1, 0]; for category 3, the corresponding vector is [0, 0, 1]; X and Y represent the predicted mask and actual mask of the sample target respectively, and ∩ represents the intersection.
[0054] Step 2-4, in the actual measurement of the data set constructed by the present invention, the global accuracy of the multi-scale U-shaped network reached 96.43%, and the average intersection-to-union ratio reached 0.6053.
[0055] Step 3: deploy a multi-scale U-shaped network on the edge computing platform. The edge computing platform system configuration includes operating system: Ubuntu 18.04 LTS, 64-bit ARMv8 architecture, processor: ARMv8 Processor rev 0 (v81)×6, graphics card: NVIDIA Tegra Xavier (nvgpu) / integrated, memory capacity: 7.6 GiB, and disk capacity: 14.7 GB.
[0056] The lane area calibration method based on the edge computing platform and the multi-scale U-shaped network has a virtual environment python version of 3.8 and a CUDA version of 11.4 configured on the edge computing platform.
[0057] On the other hand, an embodiment of the present invention further provides a lane area calibration system based on an edge computing platform, comprising the following units:
[0058] A data processing unit, used to obtain a set of images to be processed, perform mosaic data enhancement processing on the set of images and construct a training data set;
[0059] Network construction and training unit, used to build and train a multi-scale U-shaped network for lane area calibration;
[0060] The multi-scale U-shaped network includes an encoder, a decoder, and a classifier, the encoder includes n dual convolution units and n pooling operation units, each dual convolution unit is followed by a pooling operation unit, the dual convolution unit is used to extract image features, and the pooling operation unit is used to reduce the dimension of data;
[0061] The decoder includes n composite upsampling modules and n composite splicing modules, each composite upsampling module is followed by a composite splicing module, the composite upsampling module is used to transform the dimension of the input feature map, the dual convolution unit is connected to the composite splicing module in the form of a U-Net network, and the composite splicing module splices the feature maps processed by the dual convolution unit and the composite upsampling module to obtain a new feature map;
[0062] Finally, the lane line area is obtained through the softmax pixel-by-pixel classifier, including the emergency lane area, lane line area, and vehicle driving area, as well as the prediction probabilities of these three areas;
[0063] The lane area calibration unit is used to deploy the trained multi-scale U-shaped network on the edge computing platform to realize the lane area calibration of the image to be tested.
[0064] The specific implementation method of each unit is the same as each step and will not be described in detail in the present invention.
[0065] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
Claims
1. A lane area calibration method based on an edge computing platform, characterized in that: The steps include: Step 1: obtain the image set to be processed, perform mosaic data enhancement on the image set and construct a training data set; Step 2: Build and train a multi-scale U-shaped network for lane area calibration; The multi-scale U-shaped network includes an encoder, a decoder, and a classifier, the encoder includes n dual convolution units and n pooling operation units, each dual convolution unit is followed by a pooling operation unit, the dual convolution unit is used to extract image features, and the pooling operation unit is used to reduce the dimension of data; The decoder includes n composite upsampling modules and n composite splicing modules, each composite upsampling module is followed by a composite splicing module, the composite upsampling module is used to transform the dimension of the input feature map, the dual convolution unit is connected to the composite splicing module in the form of a U-Net network, and the composite splicing module splices the feature maps processed by the dual convolution unit and the composite upsampling module to obtain a new feature map; Finally, the lane line area is obtained through the softmax pixel-by-pixel classifier, including the emergency lane area, lane line area, and vehicle driving area, as well as the prediction probabilities of these three areas; The processing of the multi-scale U-shaped network is as follows: The feature maps output by n double convolution units are P1, P2, P3, P4...Pn. After the input image is reduced by 4 dimensions through the first double convolution unit, the feature map P1 is obtained. After the feature P1 passes through the pooling unit, the dimension is halved. After passing through the second double convolution unit, the feature map P2 is reduced by 4 dimensions. The features P3...Pn are obtained in sequence. The feature output of the nth pooling unit is recorded as C0. The inputs of the composite splicing module are the output of the last composite upsampling module and features Pn...P3, P2, P1. After the feature C0 passes through the composite upsampling module, the dimension of C0 is first reduced by 4 dimensions, and then after passing through the upsampling unit, the dimension is doubled to obtain the feature m1. After m1 is output from the composite upsampling module, it is input into the first composite splicing module, and is adaptively trimmed with another input feature Pn of the composite splicing module, and then spliced. The output feature is C1. After feature C1 passes through the second composite upsampling module, it is spliced with feature Pn-1 in the second composite upsampling module to obtain feature C2, and the final feature Cn to be predicted is obtained in turn; finally, the lane line area is obtained through the softmax pixel-by-pixel classifier; A comprehensive loss function consisting of a classification loss function and a segmentation loss function is used to train the multi-scale U-shaped network, where the classification loss function is the cross entropy loss function and the segmentation loss function is the Dice loss function. The calculation formula of the comprehensive loss function L is as follows: ; Among them, λ and β are classification loss functions respectively. And the segmentation loss function The weight of x i represents the i-th sample in the training data set, p(x i ) means that for sample x i The predicted probability, K is the total number of preset categories, a total of 3 categories are preset, N is the total number of samples in the training data set, C represents the predicted category, if the prediction is correct, C=1, if the prediction is wrong, C=0, y ic Indicates the conversion of sample targets into binary code. i represents the sample number. The sample target refers to the three classes in a sample image i. The three targets are i1, i2, and i3. The target is converted into binary code, which is y ic , the values of ic are i1, i2, i3. For category 1, the corresponding vector is [1, 0, 0]; for category 2, the corresponding vector is [0, 1, 0]; for category 3, the corresponding vector is [0, 0, 1]; X and Y represent the predicted mask and actual mask of the sample target respectively, and ∩ represents the intersection; Step 3: Deploy the trained multi-scale U-shaped network on the edge computing platform to calibrate the lane area of the image to be tested.
2. The lane area calibration method based on the edge computing platform according to claim 1, characterized in that: In step 1, a gantry-type road monitoring camera is used to collect videos of real scenes in different natural environments, and then the collected videos are framed, and video frames are extracted at certain time intervals, and these video frames are used as a picture set; mosaic data enhancement is performed on the picture set, and the original pictures are first scaled at different scales, and the scaled pictures are randomly spliced so that the spliced pictures meet the size of the original pictures to form a training data set.
3. The lane area calibration method based on the edge computing platform according to claim 1, characterized in that: The double convolution unit consists of two convolutional layers, the first convolutional layer and the second convolutional layer. The two convolutional layers are operated in series. The convolution kernel size of the first and second convolutional layers is 3*3, the step size is 1, the padding is 0, and the number of output channels is different. The activation function of the first convolutional layer uses the ReLU activation function, and the activation function of the second convolutional layer uses the sigmoid activation function. When the feature map passes through the first convolutional layer, the dimension of the feature map is reduced by two dimensions, and when it passes through the second convolutional layer, it is reduced by two dimensions again. In general, the feature map is reduced by 4 dimensions when it passes through the double convolutional unit.
4. The lane area calibration method based on the edge computing platform according to claim 1, characterized in that: The composite upsampling module includes a double convolution unit and an upsampling unit. When the feature map passes through the composite upsampling module, the dimension of the feature map is first reduced by 4 dimensions, and then after passing through the upsampling unit, the dimension of the feature map is doubled.
5. The lane area calibration method based on the edge computing platform according to claim 1, characterized in that: The composite stitching module is composed of an adaptive cropping unit and a stitching unit. The composite stitching module accepts the input of two feature maps. After the two feature maps are input into the composite stitching module, they first pass through the adaptive cropping unit, which compares the sizes of the two feature maps and uses the smaller feature map as a template to crop the larger feature map to obtain two feature maps of the same size. Finally, the two feature maps are stitched to obtain a new feature map.
6. A lane area calibration system based on an edge computing platform, characterized in that: The following units are included: A data processing unit, used to obtain a set of images to be processed, perform mosaic data enhancement processing on the set of images and construct a training data set; Network construction and training unit, used to build and train a multi-scale U-shaped network for lane area calibration; The multi-scale U-shaped network includes an encoder, a decoder, and a classifier, the encoder includes n dual convolution units and n pooling operation units, each dual convolution unit is followed by a pooling operation unit, the dual convolution unit is used to extract image features, and the pooling operation unit is used to reduce the dimension of data; The decoder includes n composite upsampling modules and n composite splicing modules, each composite upsampling module is followed by a composite splicing module, the composite upsampling module is used to transform the dimension of the input feature map, the dual convolution unit is connected to the composite splicing module in the form of a U-Net network, and the composite splicing module splices the feature maps processed by the dual convolution unit and the composite upsampling module to obtain a new feature map; Finally, the lane line area is obtained through the softmax pixel-by-pixel classifier, including the emergency lane area, lane line area, and vehicle driving area, as well as the prediction probabilities of these three areas; The processing of the multi-scale U-shaped network is as follows: The feature maps output by n double convolution units are P1, P2, P3, P4...Pn. After the input image is reduced by 4 dimensions through the first double convolution unit, the feature map P1 is obtained. After the feature P1 passes through the pooling unit, the dimension is halved. After passing through the second double convolution unit, the feature map P2 is reduced by 4 dimensions. The features P3...Pn are obtained in sequence. The feature output of the nth pooling unit is recorded as C0. The inputs of the composite splicing module are the output of the last composite upsampling module and features Pn...P3, P2, P1. After the feature C0 passes through the composite upsampling module, the dimension of C0 is first reduced by 4 dimensions, and then after passing through the upsampling unit, the dimension is doubled to obtain the feature m1. After m1 is output from the composite upsampling module, it is input into the first composite splicing module, and is adaptively trimmed with another input feature Pn of the composite splicing module, and then spliced. The output feature is C1. After feature C1 passes through the second composite upsampling module, it is spliced with feature Pn-1 in the second composite upsampling module to obtain feature C2, and the final feature Cn to be predicted is obtained in turn; finally, the lane line area is obtained through the softmax pixel-by-pixel classifier; A comprehensive loss function consisting of a classification loss function and a segmentation loss function is used to train the multi-scale U-shaped network, where the classification loss function is the cross entropy loss function and the segmentation loss function is the Dice loss function. The calculation formula of the comprehensive loss function L is as follows: ; Among them, λ and β are classification loss functions respectively. And the segmentation loss function The weight of x i represents the i-th sample in the training data set, p(x i ) means that for sample x i The predicted probability, K is the total number of preset categories, a total of 3 categories are preset, N is the total number of samples in the training data set, C represents the predicted category, if the prediction is correct, C=1, if the prediction is wrong, C=0, y ic Indicates the conversion of sample targets into binary code. i represents the sample number. The sample target refers to the three classes in a sample image i. The three targets are i1, i2, and i3. The target is converted into binary code, which is y ic , the values of ic are i1, i2, i3. For category 1, the corresponding vector is [1, 0, 0]; for category 2, the corresponding vector is [0, 1, 0]; for category 3, the corresponding vector is [0, 0, 1]; X and Y represent the predicted mask and actual mask of the sample target respectively, and ∩ represents the intersection; The lane area calibration unit is used to deploy the trained multi-scale U-shaped network on the edge computing platform to realize the lane area calibration of the image to be tested.
