Detection model and method of anti-skid ability of airport wet pavement

Abstract

The present invention proposes a detection model based on scene graph segmentation and multimodality and a model-based airport wet and slippery pavement anti-skid ability detection method. The image segmentation model (APSegNet) used, that is, a model based on an encoder-decoder architecture, can better extract scene graph features at different sizes at the pixel level. The use of horizontal and vertical moving windows can make the convolution function of feature extraction more focused on the specific position of the feature map, introducing more complete location information. The humidity prediction model APHumidNet can use a multimodal feature fusion method to fuse the features of the pavement, sky, greenery and masked aircraft at different angles, so as to detect the humidity of the airport pavement. Due to different humidity, the anti-skid ability of the pavement is affected. In this way, the pavement anti-skid factor in the new image can be calculated and fed back to the driving system to adjust the speed or the braking system in order to achieve a shorter braking distance and ensure driving safety.

Description

Technical Field

[0001] The present invention relates to the fields of anti-skid capability of airport asphalt pavement and artificial intelligence technology, and in particular to a detection model based on scene graph segmentation and multimodality and a model-based anti-skid capability detection method for airport wet and slippery pavement. Background Art

[0002] The safety performance of airport asphalt pavements depends heavily on weather conditions, as slippery surfaces reduce friction and increase the risk of collisions. Therefore, timely feedback on pavement moisture conditions is crucial for safe driving, such as real-time navigation planning and autonomous driving, enabling timely adjustments to the braking system to reduce braking distances and improve driving safety. Traditionally, pavement moisture conditions can be measured using either invasive sensors embedded in the pavement or non-invasive sensors mounted on the pavement. However, invasive sensors are susceptible to damage from traffic and are difficult to maintain and replace. Non-invasive sensors, mounted on the pavement, are more expensive to maintain and are prone to damage over time.

[0003] With the development and application of deep learning in image recognition, this paper utilizes image and video data sources, fully leveraging the rich information contained in images. It proposes a scene graph segmentation and multimodal method and system for detecting the anti-skid capacity of airport wet pavement surfaces. This method uses video-captured airport pavement images and measured humidity data to train an image segmentation model and a weather and humidity prediction model. Based on this model, an unknown airport image can be input to predict the current weather and humidity conditions. This prediction can provide guidance for airport driving. For unmanned driving scenarios, speed and braking system adjustments can be made based on weather conditions, thereby reducing braking distances on wet pavement surfaces and improving driving safety. Summary of the Invention

[0004] The present invention provides a method for training an airport wet pavement anti-skid capability detection model based on scene graph segmentation and multimodality, comprising the following steps:

[0005] S1: Use high-speed cameras to collect high-definition photos of the airport under different weather conditions and measure the humidity of the airport to obtain the humidity status;

[0006] The humidity level is divided into 5 levels: 0-extremely dry, 1-dry, 2-slightly humid, 3-normally humid, and 4-extremely humid.

[0007] S2: Image Annotation

[0008] Image annotation is performed simultaneously in two directions: one is segmentation annotation and the other is humidity annotation;

[0009] S3: In the segmentation and annotation direction, Labelme software is used to outline the contours of four types of objects in the image: road surface, sky, aircraft, and greenery using polygons, and then saved as a JSON file;

[0010] S4: In the humidity labeling direction, put the photos and humidity data collected in S1 into 5 folders from 0 to 4 according to the level. The pictures are placed in the corresponding folders according to the humidity level at the time of shooting, and the labeling is completed;

[0011] S5: Use the images annotated in S3 to train an image segmentation model (APSegNet);

[0012] S6: Based on the road surface humidity corresponding to the images annotated in S3 and the images annotated in S4, a multimodal scene image road surface humidity prediction model is trained to obtain a humidity prediction model (APHumidNet).

[0013] The present invention also provides an airport wet and slippery pavement anti-skid capability detection model constructed by the above method.

[0014] The present invention also provides a method for detecting the anti-skid ability of an airport wet pavement based on the above model, the specific steps of which are as follows:

[0015] (1) Use a high-speed camera to capture a 512x512 image of the airport pavement to be inspected. Load the APSegNet image segmentation model and use it to segment four objects: pavement, sky, aircraft, and greenery. Then load the humidity prediction model and input the segmented object images into the humidity prediction model to predict the pavement humidity status.

[0016] (2) Obtain the corresponding influencing factor through the predicted humidity state, and finally obtain the BPN value under the current anti-slip state of the airport;

[0017] The BPN value under the current anti-slip state = the anti-slip BPN under dry conditions × the impact factor (a) (11).

[0018] In addition, the present invention also provides a server, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein the processor implements the above-mentioned detection method steps when executing the computer program; and a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program implements the above-mentioned detection method steps when executed by the processor.

[0019] The present invention provides a method and system for predicting the humidity of an airport pavement based on an airport scene image segmentation model (APSegNet) and a humidity prediction model (APHumidNet). The method and system can detect the humidity of the airport pavement. Since different humidity levels affect the anti-skid ability of the pavement, the anti-skid factor of the pavement in the new image can be calculated and fed back to the driving system to adjust the speed or the braking system so as to achieve a shorter braking distance and ensure driving safety. The technical advantages of the present invention are mainly reflected in the following aspects: (1) The image segmentation model (APSegNet) architecture used, for the airport scene graph, the encoder-decoder architecture model can better extract the features of the scene graph at different pixel levels, and then the decoder part restores the features to their size, thereby achieving pixel-level segmentation step by step, and then achieving accurate pixel-level segmentation of the four categories of pavement, sky, aircraft, and greenery. (2) The downsampling module is used to extract features from the image. When the size is reduced, the features of the image are retained to the greatest extent according to the color and texture distribution of the road surface, sky, aircraft, and greenery. The upsampling module is used to restore the small-sized features, but ensure that the feature map is not distorted. Finally, it is restored to the original image size and achieves accurate segmentation at the pixel level. (3) For the airport scene image, the moving window block multi-layer perceptron is used. It can perform a moving window block operation on the feature map of the scene image, exchange and fuse feature information in each window, and globally distinguish different categories of road surface, sky, aircraft, and greenery to achieve accurate segmentation. (4) For the airport scene image, the patch embedding module is used to reduce the size of the feature map of the scene image, but the main color and texture features are not lost, thus preparing for the subsequent segmentation. (5) For the airport scene image, the horizontal and vertical moving windows are used, which can make the convolution function of the feature extraction more focused on the specific position of the feature map, such as the obvious position of the sky with white clouds, the green belt with plants, etc., thereby introducing more complete position information of the feature map. (6) For airport scene graphs, deep convolution can effectively separate the features of the pavement, sky, aircraft, and greenery layers, making full use of the effective information at the same location at different levels. (7) The humidity prediction model APHumidNet can use a multimodal feature fusion method to extract the vectors of the pavement, sky, greenery, and masked aircraft. It can fuse the features of the pavement, sky, greenery, and masked aircraft at different angles. The vector represented by this feature contains some connections between the pavement, sky, and greenery, such as: the sky is at the top and the greenery surrounds the pavement; this feature also fully considers the connection between the pavement humidity and the color of the greenery and the scene displayed by the sky, such as: the greenery shows brighter colors on rainy days, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of the scene graph segmentation model (APSegNet);

[0022] Figure 3 This is the schematic diagram of the convolution downsampling and upsampling modules;

[0023] Figure 4 This is a schematic diagram of the structure of the moving window block multilayer perceptron;

[0024] Figure 5 Schematic diagram of patch embedding structure;

[0025] Figure 6 Move the schematic diagram for the horizontal window;

[0026] Figure 7 Move the schematic for the vertical window;

[0027] Figure 8 This is the principle diagram of deep convolution;

[0028] Figure 9 This is the effect picture of the airport scene segmented by APSegNet;

[0029] Figure 10 Schematic diagram of the structure of the humidity detection model (HumiDetNet). DETAILED DESCRIPTION

[0030] The present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be noted that the following examples are intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0031] Example 1

[0032] A training method for an airport wet pavement anti-skid capability detection model based on scene graph segmentation and multimodality, including: Figure 1 The specific process is as follows:

[0033] S1: High-speed cameras are used to collect high-definition photos of the airport under different weather conditions, including clear skies, light rain, heavy rain, frost, and snow. These different weather conditions reflect the different levels of road surface slipperiness and conditions.

[0034] In order to mark the correspondence between the pictures and the road surface humidity, the road surface humidity data needs to be collected while sampling the photos. The humidity data is measured and recorded by a humidity meter.

[0035] The humidity level is divided into five levels: 0 - Very dry, 1 - Dry, 2 - Slightly damp, 3 - Moderately damp, and 4 - Very damp. Each of the five levels corresponds to the specific results measured by the humidity meter.

[0036] Humidity is a meteorological element that indicates the water vapor content and moisture level in the air. Relative humidity is expressed as a percentage of the actual water vapor pressure in the air to the saturated water vapor pressure at the prevailing temperature, rounded to an integer.

[0037] The relative humidity index parameters corresponding to the 5 levels of humidity are:

[0038] 0-Extremely dry <20%

[0039] 1- Dry 20-40%

[0040] 2- Slightly humid 40%-60%

[0041] 3-General humidity 60%-80%

[0042] 4- Extremely humid 80% or more.

[0043] When training the APSegNet model, we collected 5,000 images of sunny, light rain, heavy rain, frost, and snowy weather scenes at three airports in North China, 1,000 images for each weather condition. The collected images were then resized to a basic 512×512 size, and humidity was tested while the images were being collected.

[0044] S2: Image Annotation

[0045] Image annotation is performed simultaneously in two directions: one is segmentation annotation and the other is humidity annotation.

[0046] S3: In the segmentation and annotation direction, use Labelme software to outline the contours of four types of objects in the image: road surface, sky, aircraft, and greenery using polygons, and then save them. The annotation results can be saved as a JSON format file, which contains the coordinate points of the enclosing contours of each type of object in the image.

[0047] S4: In the humidity labeling direction, put the pictures and the humidity data collected by S1 into 5 folders from 0 to 4 according to the levels. Put the pictures into the corresponding folders according to the humidity level at the time of shooting, and the labeling is completed.

[0048] S5: Using the images annotated in S3, train an image segmentation model (APSegNet) that can segment four objects: road surface, sky, aircraft, and greenery. Save the model. Mini-batch stochastic gradient descent is used as the training optimization algorithm. Hyperparameter values ​​are as follows: weight decay factor = 0.0002, momentum = 0.9, learning rate = 0.01, and mini-batch size = 16.

[0049] When the training reaches 100 epochs or the loss function reaches 0.00001, stop training.

[0050] S6: Train a multimodal scene image road surface humidity prediction model using the road surface humidity corresponding to the images annotated in S3 and S4. A humidity prediction model (APHumidNet) is obtained and saved. Mini-batch stochastic gradient descent is used as the training optimization algorithm. Hyperparameter values ​​are as follows: weight decay factor = 0.0003, momentum = 0.9, learning rate = 0.005, mini-batch size = 32. Training is terminated when 100 epochs are reached or the loss function reaches 0.00001.

[0051] The APSegNet in step S5 is a model based on an encoder-decoder architecture. The following describes the model in detail using the segmentation of airport scene images as an example.

[0052] like Figure 2As shown, the size of the captured airport scene image is [3, H, W], where 3 represents the three RGB channels, H represents the width, and W represents the height. In the encoder on the left of the figure, the input image first undergoes a convolutional downsampling module, reducing the size to [C1, H / 2, W / 2]—that is, the number of channels becomes C1, and the width and height are halved. The result is denoted as A1. It then undergoes a moving window block multilayer perceptron downsampling module, reducing the size to [C2, H / 4, W / 4], with the result denoted as A2. It then undergoes a convolutional downsampling module, reducing the size to [C3, H / 8, W / 8], with the result denoted as A3. It then undergoes a moving window block multilayer perceptron downsampling module, reducing the size to [C4, H / 16, W / 16], with the result denoted as A4. Finally, it undergoes another convolutional downsampling module, reducing the size to [C5, H / 32, W / 32], with the result denoted as A5. The decoder is essentially symmetrical to the encoder, except that upsampling is used instead. The encoder output A5 first passes through a convolutional upsampling module, reducing its size to [C4, H / 16, W / 16]. Here, the number of channels is reduced to C4, and the width and height are doubled. The result is denoted as B4. The encoder outputs A4 and B4 are then vector-added and serve as the input to the next module, the Moving Window Block Multilayer Perceptron (MWP) upsampling module. The resulting output has a size of [C3, H / 8, W / 8], denoted as B3. This output is then vector-added with the corresponding encoder output A3 and serves as the input to the next module, the Convolutional Upsampling module. The resulting output has a size of [C2, H / 4, W / 4], denoted as B2. B2 is then vector-added with the corresponding encoder output A2 and serves as the input to the next module, the Moving Window Block Multilayer Perceptron (MWP) upsampling module. The resulting output has a size of [C1, H / 2, W / 2], denoted as B1. B1 is vector-added with the corresponding encoder output A1 and serves as the input to the next module, the Convolutional Upsampling module. The final decoder output has a size of [4, H, W]. The decoder output size is consistent with the encoder image input size in width and height, indicating that each pixel in the image is classified. 4 indicates that the final segmentation category is 4 (road surface, sky, airplane, and greenery).

[0053] For airport scene graphs, the encoder-decoder architecture model can better extract the features of the scene graph at different sizes at the pixel level. The decoder then restores the features to their original size, thereby achieving pixel-level segmentation step by step. This allows for precise pixel-level segmentation of the four categories of pavement, sky, aircraft, and greenery.

[0054] On the basis of the above scheme, Figure 3As shown, both the convolution downsampling and upsampling modules consist of a convolution block, a batch normalization, and an activation block. The convolution kernel size of the convolution block is 3×3, the stride is 1, and the padding is 1. For the downsampling module, each activation block is followed by a maximum pooling block with a window size of 2×2; and then for the upsampling module, each activation block is followed by a bilinear interpolation block. For the airport scene image, the downsampling module is used to extract features from the image. When the size is reduced, the features of the image are retained to the greatest extent according to the color and texture distribution of the pavement, sky, aircraft, and greenery. The upsampling module is used to restore the small-sized features, but ensure that the feature map will not be distorted. Finally, it is restored to the original image size, but achieves accurate segmentation at the pixel level.

[0055] Based on the above scheme, the structure of the moving window block multilayer perceptron is as follows Figure 4 As shown in the figure, the input feature map has a size of [C, W, H], where C is the number of channels, W is the width, and H is the height. It first passes through a patch embedding block, resulting in an output of size [W / 2 × H / 2, 2C], denoted by M. After matrix deformation, the size becomes [2C, W / 2, H / 2]. Then, it passes through a horizontal sliding window, resulting in an output of size [W / 2 × H / 2, 2C]. It then passes through fully connected layer 1, with the size unchanged. This is followed by a depthwise convolutional block, with the output size unchanged. This is followed by an activation function, with no size change. Matrix deformation follows, with the size becoming [2C, W / 2, H / 2]. This is followed by a vertical sliding window block, with the output size becoming [W / 2 × H / 2, 2C]. This is followed by fully connected layer 2, with the output size still [W / 2 × H / 2, 2C], denoted by N. Vector addition of M and N yields the output of the moving window patch multilayer perceptron, with the size [W / 2 × H / 2, 2C]. For airport scene graphs, a moving window blocking multi-layer perceptron is used. It can perform moving window blocking operations on the feature graph of the scene graph, exchange and fuse feature information in each window, and globally distinguish different categories of pavement, sky, aircraft, and greenery to achieve accurate segmentation.

[0056] Based on the above scheme, the structure of the patch embedding block is as follows: Figure 5As shown in the figure, the input feature map size is [C, W, H], where C is the number of channels, W is the width, and H is the height. After the projection convolution, the size becomes [2C, W / 2, H / 2], which is equivalent to doubling the number of channels and reducing the width and height to 1 / 2 of the original. Next, matrix flattening is performed, and the size becomes [2C, W / 2×H / 2], which is equivalent to going from 3 dimensions to 2 dimensions. Then, matrix deformation is performed, and the size becomes [W / 2×H / 2, 2C]. Finally, after batch normalization, the output size is [W / 2×H / 2, 2C]. For the airport scene image, the patch embedding module is used to reduce the size of the scene image feature map, but the main color and texture features are not lost, thus preparing for the subsequent segmentation.

[0057] Based on the above scheme, the working principle diagrams of the above moving window in the horizontal and vertical directions are as follows: Figure 6 and Figure 7 As shown in . By moving the window operation, the horizontal or vertical feature data on the channel is moved, which will help subsequent convolution and other functions focus on the specific position of the feature map, thereby introducing more location information of the feature map blocks. Figure 6 As shown, assuming that the original feature map is divided into 25 feature blocks of 5×5, and numbered from 1 to 25, after the horizontal window is moved, it is equivalent to moving columns 1 and 2 to the right of columns 3, 4, and 5, and the feature map becomes as follows Figure 6 As shown on the right; Figure 7 As shown, after executing the vertical window movement, it is equivalent to moving the data of the following three columns to the top, and the feature map becomes as follows Figure 7 As shown on the right, this helps create a random window, introducing position information along both the horizontal and vertical coordinates. For the airport scene image, using a horizontal and vertical moving window allows the convolution function for feature extraction to focus more closely on specific locations in the feature map, such as prominent areas of the sky with white clouds and green belts with plants, thereby introducing more complete location information in the feature map.

[0058] On the basis of the above scheme, Figure 8 The diagram below shows the principle behind the depthwise convolution block. This is a channel-wise convolution, meaning each convolution kernel corresponds to a specific channel of the input. Depthwise convolution separates the features of each layer, effectively leveraging the available information from different layers at the same spatial location. For the airport scene image, depthwise convolution effectively separates the features of the pavement, sky, aircraft, and greenery layers, leveraging the available information at the same location at different levels.

[0059] Therefore, the moving window block multilayer perceptron ( Figure 4 The calculation formula is as follows:

[0060] M=Tokenize(X in ) (1)

[0061] T MT =MatrixTransform(M) (2)

[0062] X shift =Shift W (T MT ) (3)

[0063] Y = GELU(DWConv(FC1(X shift ))) (4)

[0064] Y MT =MatrixTransform(Y) (5)

[0065] Y shift =Shift H (Y MT ) (6)

[0066] N=FC2(Y shift ) (7)

[0067] Output=M+N (8)

[0068] In the above formula: Tokenize represents patch embedding, MattixTransform represents matrix transformation, Shift W Indicates moving window calculation (horizontally), Shift H Indicates moving window calculation (vertical), FC1 and FC2 indicate fully connected layers, DWConv indicates depthwise convolution, and GELU indicates activation function. in represents the input vector of the moving window block multilayer perceptron, M represents the output of the patch embedding, T MT Represents the result of matrix deformation, X shift represents the result of the horizontal moving window calculation, Y represents the result after full connection, depth convolution and activation, Y MT Represents the result of the second matrix deformation, Y shift Represents the result of the vertical moving window calculation, M represents N represents the result of the second full connection calculation.

[0069] Figure 9 The image below shows the image segmentation of an airport scene. The left side shows the photo, and the right side shows the segmented image. Through segmentation, four objects—the pavement, the sky, the aircraft, and the greenery—are identified from the original image with different colors or patterns. These objects can then be cut out from the original image, specifically by pixel matching.

[0070] Based on the above scheme, the process of the humidity prediction model (APHumidNet) in step S6 is as follows: Figure 10 As shown, from the segmented image of the airport scene above, segmentations of four object categories, namely pavement, sky, aircraft, and greenery, are obtained. Then, the aircraft image is masked so that the portions of the pavement, sky, and greenery containing the aircraft appear black. This is to eliminate interference from the aircraft image on the subsequent moisture detection of the pavement, sky, and greenery objects. The segmented pavement, sky, and greenery objects are thus obtained, with the aircraft appearing as a black mask. To extract features from the pavement, sky, and greenery images, a pre-trained ResNet deep residual network (DRN) is used. ResNet employs residual learning to address the degradation problem of deep learning. The pre-trained ResNet DRN model is derived from ResNet (https: / / arxiv.org / abs / 1512.03385v1); it proposes the idea of ​​residual learning, which protects information integrity by directly routing input information to the output. The entire network only needs to learn the difference between the input and output, simplifying the learning objective and difficulty.

[0071] Specifically, the segmented image is fed into a ResNet, and the output vector of the penultimate layer before the fully connected layer is taken as the extracted image feature vector. This yields feature vectors for the sky, road surface, and greenery, each of size [C, W, H], where C is the number of channels, W is the width, and H is the height. To fully integrate the features of the road surface, sky, and greenery for subsequent humidity classification, a multimodal fusion approach is employed to fully integrate the features of the road surface, sky, and greenery at different angles. The sky feature vector is denoted as A, the road surface feature vector as B, and the greenery feature vector as C. The road surface feature vector is transposed to obtain D, of size [H, W]. A and D are then vector-multiplied to obtain E, of size [W, W]. A softmax calculation is performed on E to obtain F, of size [W, W], denoted as F. The greenery feature vectors C and F are vector-multiplied to obtain G, of size [W, H]. To achieve the purpose of classification, a fully connected layer is added to convert the vector into the number of categories to be classified, classes = 5. After the softmax function, the probability output of 5 categories is obtained, namely 0 - extremely dry, 1 - dry, 2 - slightly humid, 3 - moderately humid, and 4 - extremely humid. The calculation formula is (9):

[0072]

[0073] For the segmented road surface, sky, green objects, and masked aircraft, ResNet is used for feature extraction. This is a pre-trained classic model that can extract the pixel texture of the image, specifically the features of water droplets or wet parts on the surface of the road surface, sky, and green objects, into vectors, thereby preserving the image characteristics to the greatest extent.

[0074] For the extracted vectors of the pavement, sky, greenery, and masked aircraft, a multimodal feature fusion method is used to fuse the features of the pavement, sky, greenery, and masked aircraft at different angles. The vector represented by this feature contains some connections between the pavement, sky, and greenery, such as the sky being at the top and the greenery surrounding the pavement; this feature also fully considers the connection between the pavement humidity and the color of the greenery and the scene displayed by the sky, such as the brighter color of the greenery on rainy days.

[0075] This humidity prediction model (APHumidNet) achieves excellent results by using a dataset of segmented images of airport scenes at a specific airport for training. We use accuracy to measure the effectiveness of training. For example, if the true category of a sample is 2, if the predicted category is 2, it is T(True), otherwise it is F(False). The accuracy is calculated as Equation (10):

[0076] Accuracy = T / (T+F) × 100% (10)

[0077] The APHumidNet model segments and fuses these airport scene images to predict humidity levels, achieving an excellent accuracy of 84.3%, which can guide aircraft speed. Because varying humidity levels affect the pavement's skid resistance, the model calculates the pavement's skid resistance factor from the new image and feeds it back to the driving system, enabling speed and braking adjustments to minimize braking distances and ensure safe driving.

[0078] Using the above method, a model for detecting the anti-skid capability of airport wet pavement based on scene graph segmentation and multimodality can be trained.

[0079] Example 2

[0080] A method for detecting the anti-skid ability of an airport wet pavement based on the model trained in Example 1, such as Figure 1 The specific steps are as follows:

[0081] (1) For a new unknown airport pavement driving scene, a high-speed camera is used to capture an image of the airport pavement with an image size of 512x512. The APSegNet image segmentation model saved in step S5 is loaded and the segmentation model is used to segment four objects: pavement, sky, aircraft, and greenery. The humidity prediction model obtained in step S6 is then loaded and the images of the segmented objects are input into the humidity prediction model to predict the pavement humidity status in the new image.

[0082] (2) Due to the different humidity levels, the anti-skid ability of the pavement is affected. The corresponding influencing factor is obtained through the predicted humidity state. Finally, the BPN value of the airport's current anti-skid state is obtained and fed back to the driving system to adjust the speed or the braking system in order to achieve a shorter braking distance and ensure driving safety.

[0083] Anti-slip force is typically measured using a pendulum test in clear, dry weather. A rubber block is attached to the underside of the pendulum. When the pendulum swings freely from a certain height, the slider contacts the test surface. Friction between the two causes energy loss, limiting the pendulum's return to a certain height. The greater the surface friction, the smaller the return height. The friction pendulum test yields the friction pendulum value (BPN). Higher humidity reduces anti-slip force. Specific influencing factors are shown in Table 1:

[0084] Table 1 Relationship between pavement moisture status and influencing factor indicators

[0085] Wet condition of the road surface Impact Factor (a) 0-Extremely dry 1.2 1- Drying 1.0 2- Slightly damp 0.76 3-Generally humid 0.55 4- Extremely humid 0.41

[0086] Then the current anti-slip force state value can be calculated by the formula:

[0087] BPN value under current anti-slip state = anti-slip BPN under dry conditions × influence factor (a) (11)

[0088] The BPN value calculated under the current anti-skid state is transmitted to the driving system for speed adjustment or braking system adjustment to achieve a shorter braking distance and ensure driving safety.

[0089] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as above in terms of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make some changes or modifications to equivalent embodiments using the technical contents disclosed above. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A training method for an airport wet pavement anti-skid capability detection model based on scene graph segmentation and multimodality, characterized in that: The steps include: S1: Use high-speed cameras to collect high-definition photos of the airport under different weather conditions and measure the humidity of the airport to obtain the humidity status; The humidity level is divided into 5 levels: 0-extremely dry, 1-dry, 2-slightly humid, 3-normally humid, and 4-extremely humid. S2: Image Annotation Image annotation is performed simultaneously in two directions: one is segmentation annotation and the other is humidity annotation; S3: In the segmentation and annotation direction, Labelme software is used to outline the contours of four types of objects in the image: road surface, sky, aircraft, and greenery using polygons, and then saved as a JSON file; S4: In the humidity labeling direction, put the photos and humidity data collected in S1 into 5 folders from 0 to 4 according to the level. The pictures are placed in the corresponding folders according to the humidity level at the time of shooting, and the labeling is completed; S5: Use the images annotated in S3 to train an image segmentation model APSegNet; S6: Based on the road surface humidity corresponding to the images annotated in S3 and S4, a multimodal scene image road surface humidity prediction model is trained to obtain a humidity prediction model APHumidNet; The APSegNet in step S5 is a model based on the encoder-decoder architecture. Specifically, the input image size is [3, H, W]. In the encoder part, the input image first passes through a convolution downsampling module and the size becomes [C1, H / 2, W / 2], that is, the channel becomes C1, the width and height become 1 / 2 of the original, and the result is recorded as A1; then it passes through a moving window block multilayer perceptron downsampling, the size becomes [C2, H / 4, W / 4], and the result is recorded as A2; followed by a convolution downsampling, the size becomes [C3, H / 8, W / 8], and the result is recorded as A3; then it passes through a moving window block multilayer perceptron downsampling module, the size becomes [C4, H / 16, W / 16], and the result is recorded as A4; finally, it passes through a convolution downsampling, the size becomes [C5, H / 32, W / 32], and the result is recorded as A5; the output A5 of the encoder first passes through a convolution upsampling module, the size becomes [C4, H / 16, W / 16], where the number of channels becomes C4, the width and height become twice the original, and the result is recorded as B4; then the encoder output A4 and B4 are vector-added and used as the input of the next module moving window block multilayer perceptron-upsampling. The output size is [C3, H / 8, W / 8], and the result is recorded as B3; then the output and the corresponding encoder output A3 are vector-added and used as the input of the next module convolution upsampling. The output size is [C2,H / 4,W / 4], and the result is recorded as B2; B2 is then vector-added with the corresponding encoder output A2 and used as the input of the next module moving window block multilayer perceptron-upsampling. The output size is [C1,H / 2,W / 2], and the result is recorded as B1; B1 and the corresponding encoder output A1 are vector-added and used as the input of the next module convolution upsampling. Finally, the output size of the decoder is [4,H,W].

2. The method for training a model for detecting the anti-skid capability of wet and slippery road surfaces of an airport based on scene graph segmentation and multimodality according to claim 1, characterized in that: The convolutional downsampling and upsampling modules are composed of a convolution block, a batch normalization and an activation block; the convolution block has a convolution kernel size of 3×3, a stride of 1, and a padding of 1; for the downsampling module, each activation block is followed by a maximum pooling block with a window size of 2×2; and then for the upsampling module, each activation block is followed by a bilinear interpolation block.

3. The method for training a model for detecting the anti-skid capability of a wet and slippery road surface of an airport based on scene graph segmentation and multimodality according to claim 1, characterized in that: The input feature map of the moving window block multilayer perceptron is of size [C, W, H]. It first passes through a patch embedding block, with an output size of [W / 2×H / 2, 2C], denoted as M. After matrix deformation, the size becomes [2C, W / 2, H / 2]. Then, it passes through a horizontal moving window, with an output size of [W / 2× H / 2, 2C]. Then, it passes through the fully connected layer 1, with the size unchanged. After that, it is followed by a depthwise convolution block, with the output size unchanged. This is followed by an activation function, which does not change the size; then the matrix is ​​transformed, and the size becomes [2C, W / 2, H / 2]; this is followed by a vertical moving window block, and the output size becomes [W / 2× H / 2, 2C]; then there is a fully connected layer 2, and the output size is still [W / 2×H / 2, 2C], denoted as N; the vector addition of M and N is used to obtain the output of the moving window block multilayer perceptron, with a size of [W / 2× H / 2, 2C].

4. The method for training a model for detecting the anti-skid capability of wet and slippery road surfaces of an airport based on scene graph segmentation and multimodality according to claim 3, characterized in that: In the patch embedding block, the input feature map size is [C, W, H]. After projection convolution, the size becomes [2C, W / 2, H / 2]. Next, matrix flattening is performed, and the size becomes [2C, W / 2×H / 2]. Finally, after batch normalization, the output size is [W / 2×H / 2, 2C].

5. The method for training a model for detecting the anti-skid capability of wet and slippery road surfaces of an airport based on scene graph segmentation and multimodality according to claim 1, characterized in that: In step S6, the humidity prediction model APHumidNet adopts the pre-trained ResNet deep residual network.

6. A model for detecting the anti-skid capability of an airport wet pavement constructed by the method according to claims 1 to 5.

7. A method for detecting the anti-skid ability of an airport wet pavement based on the model of claim 6, characterized in that: The specific steps are as follows: (1) Use a high-speed camera to take a 512x512 image of the airport pavement to be inspected. Load the APSegNet image segmentation model and use it to segment four objects: pavement, sky, aircraft, and greenery. Then load the humidity prediction model and input the segmented object images into the humidity prediction model to predict the pavement humidity status. (2) Obtain the corresponding influencing factor through the predicted humidity state, and finally obtain the BPN value under the current anti-slip state of the airport; The BPN value under the current anti-slip state = the anti-slip BPN under dry conditions × the impact factor (a) (11).

8. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor implements the method steps of claim 7 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: The computer program implements the method steps of claim 7 when executed by a processor.

Images