Zebra crossing detection method and system based on intelligent intersection

Abstract

The present application discloses a zebra crossing detection method and system based on intelligent intersections. The method includes obtaining a zebra crossing image dataset; inputting multiple zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, outputting multiple first zebra crossing feature maps; inputting the multiple first zebra crossing feature maps in parallel into a spatial attention network and a channel attention network of the zebra crossing detection model for feature extraction and fusion, outputting multiple second zebra crossing feature maps; inputting the multiple second zebra crossing feature maps into an upsampling network of the zebra crossing detection model for image size restoration, and outputting the predicted pixel category corresponding to each pixel; constructing a loss function based on the real pixel category and the predicted pixel category, and training and optimizing the zebra crossing detection model based on the loss function to obtain a trained zebra crossing detection model, and detecting the position of the zebra crossing in the zebra crossing image to be tested.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent urban traffic management, and in particular to a zebra crossing detection method and system based on intelligent intersections. Background Art

[0002] In recent years, with the accelerating pace of urbanization, the number of motor vehicles has continued to increase, leading to problems such as urban traffic congestion and parking conflicts. Therefore, improving the intelligence level of urban traffic management is a social issue that needs to be addressed. Increasing the intelligence level of urban traffic management can effectively improve the current traffic management situation.

[0003] An intelligent urban traffic management system relies on artificial intelligence algorithms, cloud service platforms, intelligent hardware devices, and edge computing devices to accurately collect, process, and provide feedback on traffic information in real time. This system includes capturing evidence and issuing alerts for traffic violations such as running red lights and speeding, providing guidance and parking space records for roadside parking, and providing real-time updates, predictions, and information on traffic congestion. Intelligent traffic management can effectively improve urban road traffic conditions, provide traffic management departments with more evidence of vehicle violations and irregular driving, and enable more intelligent application and management of the entire traffic flow, thereby positively promoting various aspects of urban traffic management and driving safety.

[0004] Zebra crossing detection is a crucial component of intelligent traffic management. At intersections with heavy traffic, efficient and accurate zebra crossing detection is crucial for improving safe passage. However, traditional zebra crossing detection methods generate a minimum bounding rectangle for the crossing, resulting in narrow, distorted, or worn zebra crossings from certain viewing angles and fisheye lenses. Traditional zebra crossing detection methods, which use rectangular frames, are less effective for zebra crossing detection, resulting in low detection accuracy and hindering intelligent urban management.

[0005] Application Contents

[0006] The purpose of this application is to solve the technical problem of low accuracy of traditional zebra crossing detection methods. To achieve the above purpose, this application provides a zebra crossing detection method and system based on intelligent intersections.

[0007] This application provides a zebra crossing detection method based on an intelligent intersection, comprising:

[0008] Obtain a zebra crossing image dataset, where the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations;

[0009] Inputting the plurality of zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, and outputting a plurality of first zebra crossing feature maps;

[0010] Inputting the multiple first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting multiple second zebra crossing feature maps;

[0011] Inputting the plurality of second zebra crossing feature maps into the upsampling network of the zebra crossing detection model to perform image size restoration, and outputting the predicted pixel category corresponding to each pixel;

[0012] Constructing a loss function according to the real pixel category and the predicted pixel category, and performing model training and optimization on the zebra crossing detection model according to the loss function to obtain a trained zebra crossing detection model;

[0013] The position of the zebra crossing in the zebra crossing image to be detected is detected according to the trained zebra crossing detection model.

[0014] In one embodiment, the zebra crossing image dataset is obtained, and the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations, including:

[0015] Acquire a first image dataset containing zebra crossings, and perform pixel-by-pixel semantic segmentation and annotation on the first image dataset to obtain a first zebra crossing image dataset;

[0016] Acquire a second image dataset containing a zebra crossing, where the second image dataset and the first image dataset are image datasets acquired in different scenes;

[0017] Performing model training on a cyclic generative adversarial network based on the first image dataset and the second image dataset to obtain a trained cyclic generative adversarial network;

[0018] Inputting the first zebra crossing image dataset into the trained recurrent generative adversarial network, and outputting a second zebra crossing image dataset simulating different scenarios;

[0019] The zebra crossing image dataset is obtained according to the first zebra crossing image dataset and the second zebra crossing image dataset.

[0020] In one embodiment, the step of inputting the plurality of zebra crossing images containing real pixel category labels into a feature extraction network of a zebra crossing detection model for feature extraction and outputting a plurality of first zebra crossing feature maps includes:

[0021] Performing feature extraction on the plurality of zebra crossing images containing real pixel category annotations according to the pooling layer of the feature extraction network to obtain a first feature atlas;

[0022] Performing feature extraction on the first feature atlas according to the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer of the feature extraction network to obtain the plurality of first zebra crossing feature maps;

[0023] The ratio of the number of output channels of the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer is 1:2:4:8.

[0024] In one embodiment, the step of inputting the plurality of first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting a plurality of second zebra crossing feature maps, comprises:

[0025] Input each of the first zebra crossing feature maps into a horizontal dimension spatial attention network, perform average pooling and vertical replication expansion on the first zebra crossing feature map to obtain a horizontal attention parameter feature map;

[0026] Input each of the first zebra crossing feature maps into the longitudinal dimension spatial attention network, perform average pooling and horizontal replication expansion on the first zebra crossing feature maps, and obtain a longitudinal attention parameter feature map;

[0027] Adding the horizontal attention parameter feature map and the vertical attention parameter feature map pixel by pixel to obtain a spatial attention parameter feature map;

[0028] Inputting the spatial attention parameter feature map into a normalization layer, normalizing the spatial attention parameters, and obtaining normalized spatial attention parameters;

[0029] The normalized spatial attention parameter is multiplied pixel by pixel by the first zebra crossing feature map to obtain a spatial attention weighted feature map.

[0030] In one embodiment, the step of inputting the plurality of first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting a plurality of second zebra crossing feature maps, further comprises:

[0031] Input each of the first zebra crossing feature maps into two identical convolutional layers, and output a first feature map and a second feature map;

[0032] Reshaping the first feature map and the second feature map to obtain a first reshaped feature map and a second reshaped feature map;

[0033] performing feature transposition on the second reshaped feature map to obtain a second transposed feature map;

[0034] Performing a matrix multiplication operation on the first reshaped feature map and the second transposed feature map to obtain a channel feature map, and normalizing the feature map parameters of the channel feature map to obtain a channel dimension attention feature map;

[0035] Reshaping the first zebra crossing feature map to obtain a first zebra crossing reshaped feature map;

[0036] Performing a matrix multiplication operation on the channel dimension attention feature map and the first zebra crossing reshaping feature map to obtain a reweighted feature map, and performing feature map reshaping on the reweighted feature map to obtain a channel attention weighted feature map;

[0037] The spatial attention weighted feature map corresponding to each of the first zebra crossing feature maps and the channel attention weighted feature map are added pixel by pixel to obtain the multiple second zebra crossing feature maps.

[0038] In one embodiment, inputting the plurality of second zebra crossing feature maps into the upsampling network of the zebra crossing detection model to perform image size restoration and outputting the predicted pixel category corresponding to each pixel includes:

[0039] Inputting each of the second zebra crossing feature maps into the five deconvolution layers of the upsampling network to restore the image size to the image size of the original zebra crossing image, thereby obtaining a plurality of second zebra crossing feature maps of the original image size;

[0040] According to the second zebra crossing feature map of each original image size, a predicted pixel category corresponding to each pixel is obtained.

[0041] In one embodiment, the present application provides a zebra crossing detection system based on a smart intersection, comprising:

[0042] An image data acquisition module is used to acquire a zebra crossing image dataset, wherein the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations;

[0043] a feature extraction network module, configured to input the plurality of zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, and output a plurality of first zebra crossing feature maps;

[0044] An attention network module is used to input the multiple first zebra crossing feature maps into the spatial attention network and channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and output multiple second zebra crossing feature maps;

[0045] an upsampling module, configured to input the plurality of second zebra crossing feature maps into an upsampling network of the zebra crossing detection model to restore image size and output a predicted pixel category corresponding to each pixel;

[0046] A model training module is used to construct a loss function according to the real pixel category and the predicted pixel category, and perform model training and optimization on the zebra crossing detection model according to the loss function to obtain a trained zebra crossing detection model;

[0047] The detection module is used to detect the position of the zebra crossing in the zebra crossing image to be detected based on the trained zebra crossing detection model.

[0048] In one embodiment, the image data acquisition module includes:

[0049] a first zebra crossing image dataset acquisition module, configured to acquire a first image dataset containing zebra crossings, and perform pixel-by-pixel semantic segmentation and annotation on the first image dataset to obtain a first zebra crossing image dataset;

[0050] A second image data set acquisition module, configured to acquire a second image data set containing a zebra crossing, wherein the second image data set and the first image data set are image data sets acquired in different scenes;

[0051] A cyclic generative adversarial network module, configured to perform model training on a cyclic generative adversarial network based on the first image dataset and the second image dataset to obtain a trained cyclic generative adversarial network;

[0052] A second zebra crossing image dataset acquisition module is configured to input the first zebra crossing image dataset into the trained recurrent generative adversarial network and output a second zebra crossing image dataset simulating different scenarios;

[0053] The zebra crossing image data set acquisition module is configured to obtain the zebra crossing image data set according to the first zebra crossing image data set and the second zebra crossing image data set.

[0054] In one embodiment, the feature extraction network module includes:

[0055] A pooling layer module, configured to perform feature extraction on the plurality of zebra crossing images containing real pixel category annotations according to the pooling layer of the feature extraction network to obtain a first feature atlas;

[0056] a full convolutional layer module, configured to perform feature extraction on the first feature atlas according to the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer of the feature extraction network to obtain the plurality of first zebra crossing feature maps;

[0057] The ratio of the number of output channels of the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer is 1:2:4:8.

[0058] In one embodiment, the attention network module includes:

[0059] A horizontal attention parameter feature map acquisition module is used to input each of the first zebra crossing feature maps into a horizontal dimensional spatial attention network, perform average pooling and vertical replication expansion on the first zebra crossing feature maps, and obtain a horizontal attention parameter feature map;

[0060] A longitudinal attention parameter feature map acquisition module is used to input each of the first zebra crossing feature maps into the longitudinal dimensional spatial attention network, perform average pooling and horizontal replication expansion on the first zebra crossing feature maps, and obtain a longitudinal attention parameter feature map;

[0061] a spatial attention parameter feature map acquisition module, configured to add the horizontal attention parameter feature map and the vertical attention parameter feature map pixel by pixel to obtain a spatial attention parameter feature map;

[0062] A normalized spatial attention parameter acquisition module is used to input the spatial attention parameter feature map into the normalization layer, normalize the spatial attention parameter, and obtain the normalized spatial attention parameter;

[0063] A spatial attention weighted feature map acquisition module is used to multiply the normalized spatial attention parameter by the first zebra crossing feature map pixel by pixel to obtain a spatial attention weighted feature map.

[0064] In one embodiment, the attention network module further includes:

[0065] A convolution module, configured to input each of the first zebra crossing feature maps into two identical convolution layers, and output a first feature map and a second feature map;

[0066] A first feature reshaping module is configured to reshape the first feature map and the second feature map to obtain a first reshaped feature map and a second reshaped feature map;

[0067] a feature transposition module, configured to perform feature transposition on the second reshaped feature map to obtain a second transposed feature map;

[0068] a channel-dimensional attention feature map acquisition module, configured to perform a matrix multiplication operation on the first reshaped feature map and the second transposed feature map to obtain a channel feature map, and perform feature map parameter normalization on the channel feature map to obtain a channel-dimensional attention feature map;

[0069] A second feature reshaping module is configured to reshape the first zebra crossing feature map to obtain a first zebra crossing reshaped feature map;

[0070] A channel attention weighted feature map acquisition module is used to perform a matrix multiplication operation on the channel dimension attention feature map and the first zebra crossing reshaping feature map to obtain a reweighted feature map, and perform feature map reshaping on the reweighted feature map to obtain a channel attention weighted feature map;

[0071] The second zebra crossing feature map acquisition module is used to add the spatial attention weighted feature map and the channel attention weighted feature map corresponding to each of the first zebra crossing feature maps pixel by pixel to obtain the multiple second zebra crossing feature maps.

[0072] In one embodiment, the upsampling module includes:

[0073] An image resizing module is configured to input each of the second zebra crossing feature maps into the five deconvolution layers of the upsampling network to restore the image size to the image size of the original zebra crossing image, thereby obtaining a plurality of second zebra crossing feature maps of the original image size;

[0074] The pixel category acquisition module is used to obtain the predicted pixel category corresponding to each pixel according to the second zebra crossing feature map of each original image size.

[0075] In the above-mentioned zebra crossing detection method and system based on intelligent intersections, the obtained zebra crossing image dataset is used as a model training set, and is sequentially input into the feature extraction network, spatial attention network, channel attention network and upsampling network of the zebra crossing detection model. A loss function is constructed based on the real pixel category and the predicted pixel category to train and optimize the model, and a trained zebra crossing detection model is obtained, which is used to detect the position of the zebra crossing in the zebra crossing image to be tested. Therefore, the zebra crossing detection method based on intelligent intersections provided by the present application enhances spatial feature extraction by adding a dual attention mechanism, can effectively utilize zebra crossing data with spatial characteristics of slender strip shapes, and at the same time enhances feature extraction in the channel dimension, can improve the robustness of zebra crossing detection, accurately extract the spatial features and inter-channel features of the zebra crossing, improve the detection accuracy of the zebra crossing detection model, thereby improving the detection effect of the zebra crossing, and is conducive to the realization of intelligent management of the city. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flowchart of the steps of the zebra crossing detection method based on smart intersections provided by this application.

[0077] Figure 2This is a schematic diagram of the structure of the zebra crossing detection system based on the intelligent intersection provided by this application. DETAILED DESCRIPTION

[0078] The technical solution of the present application is further described in detail below through the accompanying drawings and examples.

[0079] See Figure 1 , this application provides a zebra crossing detection method based on smart intersections, including:

[0080] S10, obtaining a zebra crossing image dataset, where the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations;

[0081] S20, inputting a plurality of zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, and outputting a plurality of first zebra crossing feature maps;

[0082] S30, inputting the multiple first zebra crossing feature maps into the spatial attention network and channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting multiple second zebra crossing feature maps;

[0083] S40, inputting the plurality of second zebra crossing feature maps into an upsampling network of a zebra crossing detection model to restore the image size, and outputting a predicted pixel category corresponding to each pixel;

[0084] S50, constructing a loss function based on the real pixel category and the predicted pixel category, and performing model training and optimization on the zebra crossing detection model based on the loss function to obtain a trained zebra crossing detection model;

[0085] S60: Detect the position of the zebra crossing in the zebra crossing image to be detected based on the trained zebra crossing detection model.

[0086] In this embodiment, in S10, a zebra crossing image dataset is acquired via a traffic intersection surveillance camera. Traffic intersection surveillance cameras can capture images from multiple perspectives, and the captured image data includes zebra crossing image data from multiple perspectives, including horizontal, vertical, and diagonal. The zebra crossings in the images captured by the cameras are semantically segmented and annotated pixel by pixel using a polygon annotation tool. Multiple zebra crossing images with true pixel category annotations are obtained, forming a zebra crossing image dataset. Pixel category annotations include two pixel categories: background and zebra crossing. By knowing the category of the pixels in the image, the location of the zebra crossing can be determined.

[0087] In S20, the feature extraction network, spatial attention network, channel attention network, and upsampling network constitute the zebra crossing detection model. The feature extraction network is used to adjust the size of the feature maps of multiple zebra crossing images and enhance feature extraction by expanding the number of feature channels. This allows for feature extraction in each zebra crossing image, facilitating the determination of the specific location of the zebra crossing within the image.

[0088] In S30, the spatial attention network and the channel attention network are connected in parallel to form an attention mechanism network. The spatial attention network is used to enhance the spatial feature extraction of zebra crossings. The channel attention network is used to effectively enhance the extraction of semantic features by capturing the interdependencies between different channels. The spatial attention network and the channel attention network operate in parallel to simultaneously extract features in both spatial and channel dimensions, improving the detection accuracy of the zebra crossing detection model and accurately determining the location of the zebra crossing.

[0089] In S40, the second zebra crossing feature map, formed by feature extraction through the spatial attention network and the channel attention network, is restored to its original size as input to the feature extraction network through an upsampling operation, and the predicted pixel category corresponding to each pixel in the restored image size is output accordingly. The feature extraction network, spatial attention network, channel attention network, and upsampling network constitute a zebra crossing detection model. The input of the zebra crossing detection model is multiple zebra crossing images with real pixel category annotations, and the output is the predicted pixel category corresponding to each pixel.

[0090] In S50, the predicted pixel category corresponding to each pixel predicted by the zebra crossing detection model is compared with the actual pixel category, a loss function of the zebra crossing detection model is constructed, and the model is trained and optimized to form a trained zebra crossing detection model.

[0091] In S60, the zebra crossing image to be tested can be an image containing a zebra crossing captured by a traffic intersection surveillance camera. The zebra crossing image to be tested is input into a trained zebra crossing detection model, which outputs the pixel category corresponding to each pixel, thereby determining whether each pixel corresponds to a zebra crossing category. Thus, the specific location of the zebra crossing is determined based on the multiple pixels corresponding to the zebra crossing category. Therefore, the location of the zebra crossing in the zebra crossing image to be tested is detected using the trained zebra crossing detection model.

[0092] The zebra crossing detection method based on smart intersections provided by the present application uses the acquired zebra crossing image dataset as a model training set, and sequentially inputs it into the feature extraction network, spatial attention network, channel attention network, and upsampling network of the zebra crossing detection model. A loss function is constructed based on the real pixel category and the predicted pixel category to train and optimize the model, thereby obtaining a trained zebra crossing detection model for detecting the position of the zebra crossing in the zebra crossing image to be tested. Therefore, the zebra crossing detection method based on smart intersections provided by the present application enhances spatial feature extraction by adding a dual attention mechanism, can effectively utilize zebra crossing data with spatial characteristics of slender strip shapes, and at the same time enhances feature extraction in the channel dimension, can improve the robustness of zebra crossing detection, accurately extract the spatial features and inter-channel features of the zebra crossing, improve the detection accuracy of the zebra crossing detection model, thereby improving the detection effect of the zebra crossing, and is conducive to the realization of intelligent management of the city.

[0093] In one embodiment, S10, a zebra crossing image dataset is obtained. The zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations, including:

[0094] S110, obtaining a first image dataset containing zebra crossings, and performing pixel-by-pixel semantic segmentation and annotation on the first image dataset to obtain a first zebra crossing image dataset;

[0095] S120, obtaining a second image dataset containing a zebra crossing, where the second image dataset and the first image dataset are image datasets obtained in different scenes;

[0096] S130, performing model training on a recurrent generative adversarial network based on the first image dataset and the second image dataset to obtain a trained recurrent generative adversarial network;

[0097] S140, inputting the first zebra crossing image dataset into the trained recurrent generative adversarial network, and outputting a second zebra crossing image dataset simulating different scenarios;

[0098] S150 , obtaining a zebra crossing image dataset according to the first zebra crossing image dataset and the second zebra crossing image dataset.

[0099] In this embodiment, the first image dataset is image data acquired by a surveillance camera at a traffic intersection. Semantic segmentation and annotation are performed on each image in the first image dataset to obtain a first zebra crossing image dataset. Each image in the first zebra crossing image dataset is annotated with a real pixel category, namely, a background category and a zebra crossing category. The second image dataset and the first image dataset are image datasets acquired under different scenes. Different scenes can be understood as different weather conditions, such as sunny, rainy, foggy, and other weather conditions, or different times, such as morning, noon, evening, and other time scenes. The second image dataset and the first image dataset are image data acquired under different weather conditions.

[0100] Model training is performed using the unpaired second image dataset and the first image dataset as training data. A recurrent generative adversarial network (RGN) is used to construct the generator and discriminator networks for a rainy or foggy scene data generation model, resulting in a trained RGN. The first zebra crossing image dataset under different weather conditions is input into the trained RGN, which then outputs a second zebra crossing image dataset that simulates weather conditions different from the first zebra crossing image dataset. The second zebra crossing image dataset can be simulated zebra crossing data under different weather conditions, such as rainy or foggy. The first and second zebra crossing image datasets under different weather conditions are fused together to form a zebra crossing image dataset, which serves as the model training set for optimizing the training of the zebra crossing detection model. By fusing the zebra crossing image datasets under different scenarios, the training data for the zebra crossing detection model is more comprehensive. By simulating zebra crossing conditions under different scenarios through data augmentation, the accuracy of zebra crossing detection in special cases is improved, thereby enhancing the accuracy of the zebra crossing detection model.

[0101] In one embodiment, S20, multiple zebra crossing images containing real pixel category annotations are input into a feature extraction network of a zebra crossing detection model for feature extraction, and multiple first zebra crossing feature maps are output, including:

[0102] S210, performing feature extraction on a plurality of zebra crossing images containing real pixel category annotations according to a pooling layer of a feature extraction network to obtain a first feature atlas;

[0103] S220, performing feature extraction on the first feature atlas according to the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer of the feature extraction network to obtain a plurality of first zebra crossing feature maps;

[0104] Among them, the ratio of the number of output channels of the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer is 1:2:4:8.

[0105] In this embodiment, a network structure with five stages of pooling and full convolution forms a feature extraction network. In the first stage, a pooling layer is used to reduce the size of the feature map of multiple zebra crossing images containing real pixel category annotations to reduce the parameters of subsequent model calculations and improve the detection efficiency of the zebra crossing detection model. The pooling layer uses the maximum pooling method for calculation, and the pooling step size is set to 2 to extract features from the zebra crossing image. A convolution layer can be connected after the pooling layer to expand the number of feature channels and enhance the feature extraction of the image. The convolution kernel size of the convolution layer connected to the pooling layer is 1×1, the step size is 1, and the number of channels is increased from 3 to 32.

[0106] Stages 2 through 5 are all fully convolutional computations, consisting of the first, second, third, and fourth fully convolutional layers, respectively. Each of these layers consists of three convolutional layers. The convolution kernel size of the first and second layers of each fully convolutional layer is 3×3, and the kernel size of the third layer is 1×1. The convolution stride of the first layer is 2, while that of the second and third layers is 1. The number of output channels of the first and second layers remains unchanged, while the number of output channels of the third layer is doubled. The first, second, and third convolutional layers are each connected to a normalization layer and an activation function layer. The output channel counts of the first, second, third, and fourth fully convolutional layers are in a ratio of 1:2:4:8, which are 64, 128, 256, and 512, respectively. Normalization layers include, but are not limited to, instance normalization and adaptive instance normalization. Non-linear activation layers include, but are not limited to, ReLU, Leaky ReLU, and other non-linear activation functions. The input image size of multiple zebra crossing images containing real pixel category annotations to the feature extraction network is 3×1920×1280 (C×H×W, Channel×Height×Width, number of channels×height×width). After the five-stage operation of the feature extraction network, namely the pooling layer, the first fully convolutional layer, the second fully convolutional layer, the third fully convolutional layer, and the fourth fully convolutional layer, multiple first zebra crossing feature maps with a feature map size of 512×60×40 are obtained.

[0107] In this embodiment, a feature extraction network formed by a pooling layer and multiple fully convolutional layers is used to reduce the size of feature maps and extract features from multiple zebra crossing images containing real pixel category annotations, so as to reduce the parameters of subsequent model calculations, which is beneficial to improving the detection accuracy of the zebra crossing detection model.

[0108] In one embodiment, S30, the plurality of first zebra crossing feature maps are respectively and parallelly input into the spatial attention network and the channel attention network of the zebra crossing detection model for feature extraction and fusion, and a plurality of second zebra crossing feature maps are output, including:

[0109] S310, inputting each first zebra crossing feature map into a horizontal dimension spatial attention network, performing average pooling and vertical replication expansion on the first zebra crossing feature map to obtain a horizontal attention parameter feature map;

[0110] S320, inputting each first zebra crossing feature map into the longitudinal dimension spatial attention network, performing average pooling and horizontal replication expansion on the first zebra crossing feature map to obtain a longitudinal attention parameter feature map;

[0111] S330, adding the horizontal attention parameter feature map and the vertical attention parameter feature map pixel by pixel to obtain a spatial attention parameter feature map;

[0112] S340, inputting the spatial attention parameter feature map into a normalization layer, normalizing the spatial attention parameters, and obtaining normalized spatial attention parameters;

[0113] S350, multiply the normalized spatial attention parameter by the first zebra crossing feature map pixel by pixel to obtain a spatial attention weighted feature map.

[0114] In this embodiment, the spatial attention network and the channel attention network form an attention mechanism network, and the two parts are connected in parallel. Through the spatial attention network, the spatial feature extraction of the zebra crossing can be enhanced. The horizontal dimension spatial attention network and the vertical dimension spatial attention network fully consider the horizontal or vertical strip shape of the zebra crossing in the image, and then fully consider the characteristics of the zebra crossing for feature extraction. The spatial attention part performs parallel attention calculations from the horizontal and vertical dimensions respectively, which can enhance the feature extraction of the zebra crossing. The input of the spatial attention part is the first zebra crossing feature map output by the feature extraction network, and the feature map size is 512×60×40.

[0115] The calculation process of the spatial attention network in the horizontal dimension is to perform an average pooling operation with a pooling step of (1, 40) on the first zebra crossing feature map, transform the feature map size to 512×60×1, and replicate the feature map after average pooling along the vertical direction so that the feature parameters of each vertical direction are the same, obtaining a feature map size of 512×60×40, and obtaining a horizontal attention parameter feature map.

[0116] The calculation process of the spatial attention network in the vertical dimension is similar to that in the horizontal dimension. The input is the first zebra crossing feature map with a size of 512×60×40, which is the output feature map of the feature extraction network. The feature map is subjected to an average pooling operation with a pooling step of (60, 1). The feature map size is transformed to 512×1×40, and the feature map after average pooling is replicated and expanded horizontally so that the feature parameters of each horizontal direction are the same, resulting in a feature map size of 512×60×40, and a vertical attention parameter feature map is obtained.

[0117] The horizontal and vertical attention parameter feature maps are added pixel by pixel to obtain the overall spatial attention parameter feature map. The spatial attention parameter feature map is input into a normalization layer to normalize the attention parameters. The normalized attention parameters are then multiplied pixel by pixel with the first zebra crossing feature map (i.e., the input feature maps of the horizontal and vertical spatial attention networks) to implement the feature weighting operation of spatial attention and obtain the spatial attention weighted feature map. The output feature map size is 512×60×40.

[0118] In one embodiment, S30, the plurality of first zebra crossing feature maps are respectively and parallelly input into the spatial attention network and the channel attention network of the zebra crossing detection model for feature extraction and fusion, and a plurality of second zebra crossing feature maps are output, further comprising:

[0119] S360: Input each first zebra crossing feature map into two identical convolutional layers, and output a first feature map and a second feature map;

[0120] S370, performing feature map reshaping on the first feature map and the second feature map respectively to obtain a first reshaped feature map and a second reshaped feature map;

[0121] S380, performing feature transposition on the second reshaped feature map to obtain a second transposed feature map;

[0122] S390, performing a matrix multiplication operation on the first reshaped feature map and the second transposed feature map to obtain a channel feature map, and performing feature map parameter normalization on the channel feature map to obtain a channel dimension attention feature map;

[0123] S391, reshape the first zebra crossing feature map to obtain a first zebra crossing reshaped feature map;

[0124] S392, performing a matrix multiplication operation on the channel dimension attention feature map and the first zebra crossing reshaped feature map to obtain a reweighted feature map, and performing feature map reshaping on the reweighted feature map to obtain a channel attention weighted feature map;

[0125] S393, adding the spatial attention weighted feature map and the channel attention weighted feature map corresponding to each first zebra crossing feature map pixel by pixel to obtain multiple second zebra crossing feature maps.

[0126] In this embodiment, two identical convolutional layer calculation processes, feature map reshaping processes, feature transposition processes, matrix multiplication operations, feature map parameter normalization processes, and feature map pixel-by-pixel addition processes constitute a channel attention network. Through the channel attention network, the interdependence between different channels can be obtained, effectively enhancing the extraction of semantic features. The channel attention network and the spatial attention network operate in parallel. The input of the channel attention network is the first zebra crossing feature map with a feature map size of 512×60×40 output by the feature extraction network. The first zebra crossing feature map output by the feature extraction network is passed through two convolutional layers with a convolution kernel size of 1×1 to obtain two feature maps, namely the first feature map and the second feature map. The feature map sizes of the first feature map and the second feature map are the same, both 512×60×40. The first feature map and the second feature map are reshaped to form a first reshaped feature map and a second reshaped feature map. The feature map sizes of the first reshaped feature map and the second reshaped feature map become 512×(60×40). The second reshaped feature map is transposed to obtain a second transposed feature map, with a feature map size of 2400 × 512. The first reshaped feature map is matrix multiplied with the transposed second reshaped feature map (i.e., the second transposed feature map) to obtain a channel feature map. The feature map size of the channel feature map is 512 × 512. After normalizing the feature map parameters of the channel feature map, the channel-dimensional attention feature map is obtained.

[0127] The first zebra crossing feature map input to the channel attention network is reshaped from 512×60×40 to 512×2400, obtaining the first zebra crossing reshaped feature map. The channel-dimensional attention feature map is matrix-multiplied with the reshaped first zebra crossing feature map to obtain a reweighted feature map of size 512×2400. The reweighted feature map is then reshaped to obtain the channel-dimensional attention weighted feature map. The channel-dimensional attention weighted feature map has a feature map size of 512×60×40, which enables feature reweighting in the channel dimension. The feature output of the spatial attention component is pixel-by-pixel added to the feature output of the channel attention component to obtain the second zebra crossing feature map of size 512×60×40.

[0128] The zebra crossing detection method based on smart intersections provided by this application enhances spatial feature extraction by adding a dual attention mechanism, can effectively utilize zebra crossing data with spatial characteristics of slender strip shapes, and at the same time enhances feature extraction in the channel dimension, which can improve the robustness of zebra crossing detection, accurately extract the spatial features and inter-channel features of zebra crossings, and improve the detection accuracy of the zebra crossing detection model, thereby improving the detection effect of zebra crossings, which is conducive to the realization of intelligent management of cities.

[0129] In one embodiment, S40, the plurality of second zebra crossing feature maps are input into an upsampling network of a zebra crossing detection model to restore the image size, and the predicted pixel category corresponding to each pixel is output, including:

[0130] S410, inputting each second zebra crossing feature map into five deconvolution layers of an upsampling network to restore the image size to the image size of the original zebra crossing image, thereby obtaining a plurality of second zebra crossing feature maps of the original image size;

[0131] S420: Obtain a predicted pixel category corresponding to each pixel according to the second zebra crossing feature map of each original image size.

[0132] In this embodiment, the upsampling part is the third part of the zebra crossing detection model, and the upsampling part uses deconvolution to restore the image size. The input of the upsampling part is the second zebra crossing feature map formed after passing through the spatial attention network and the channel attention network. The second zebra crossing feature map is a feature map with a size of 512×60×40. The second zebra crossing feature map is restored to the original input image size through the deconvolution process. It can be understood as the original image size input to the zebra crossing detection model, or as the image size input to the feature extraction network, that is, 3×1920×1280. The convolution kernel size of the 5 deconvolution layers is 3×3, and the convolution step size is 2.

[0133] The zebra detection model consists of a feature extraction network, a spatial attention network, a channel attention network, and an upsampling network. Multiple zebra crossing images with true pixel category annotations are fed into the feature extraction network. After passing through the spatial and channel attention networks in parallel, the upsampling network outputs a second zebra crossing feature map at the original image size, corresponding to the predicted pixel category for each pixel.

[0134] In one embodiment, S50 constructs a loss function based on the real pixel categories and the predicted pixel categories, and trains and optimizes the zebra crossing detection model based on the loss function to obtain a trained zebra crossing detection model, including:

[0135] S510, constructing a loss function based on the real pixel category and the predicted pixel category. The loss function includes:

[0136]

[0137] Among them, C represents the number of pixel categories, P c Indicates the probability of predicting the pixel category as background category or zebra crossing category, y c represents the true pixel category.

[0138] S520: Perform model training and optimization on the zebra crossing detection model according to the loss function to obtain a trained zebra crossing detection model.

[0139] In this embodiment, a cross-entropy loss function can be used as the loss function. The zebra crossing detection model is trained and optimized according to the loss function to obtain a trained zebra crossing detection model. The pixel category of each pixel in the zebra crossing image to be tested is predicted, thereby determining the specific location of the zebra crossing.

[0140] See Figure 2 The present application provides a zebra crossing detection system 100 based on a smart intersection. The zebra crossing detection system 100 based on a smart intersection includes an image data acquisition module 10, a feature extraction network module 20, an attention network module 30, an upsampling module 40, a model training module 50, and a detection module 60. The image data acquisition module 10 is used to acquire a zebra crossing image dataset, which includes a plurality of zebra crossing images with real pixel category annotations. The feature extraction network module 20 is used to input the plurality of zebra crossing images with real pixel category annotations into the feature extraction network of the zebra crossing detection model for feature extraction, and output a plurality of first zebra crossing feature maps. The attention network module 30 is used to input the plurality of first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction fusion, and output a plurality of second zebra crossing feature maps.

[0141] The upsampling module 40 is used to input the multiple second zebra crossing feature maps into the upsampling network of the zebra crossing detection model to restore the image size and output the predicted pixel category corresponding to each pixel.

[0142] The model training module 50 is used to construct a loss function based on the real pixel categories and the predicted pixel categories, and train and optimize the zebra crossing detection model based on the loss function to obtain a trained zebra crossing detection model. The detection module 60 is used to detect the position of the zebra crossing in the zebra crossing image to be tested based on the trained zebra crossing detection model.

[0143] In this embodiment, the relevant description of the image data acquisition module 10 may refer to the relevant description of S10 in the above embodiment. The relevant description of the feature extraction network module 20 may refer to the relevant description of S20 in the above embodiment. The relevant description of the attention network module 30 may refer to the relevant description of S30 in the above embodiment. The relevant description of the upsampling module 40 may refer to the relevant description of S40 in the above embodiment. The relevant description of the model training module 50 may refer to the relevant description of S50 in the above embodiment. The relevant description of the detection module 60 may refer to the relevant description of S60 in the above embodiment.

[0144] In one embodiment, the image data acquisition module 10 includes a first zebra crossing image dataset acquisition module, a second image dataset acquisition module, a recurrent generative adversarial network module, a second zebra crossing image dataset acquisition module, and a zebra crossing image dataset acquisition module. The first zebra crossing image dataset acquisition module is configured to acquire a first image dataset containing zebra crossings and perform pixel-by-pixel semantic segmentation and annotation on the first image dataset to obtain a first zebra crossing image dataset. The second image dataset acquisition module is configured to acquire a second image dataset containing zebra crossings, where the second image dataset and the first image dataset are image datasets acquired in different scenes. The recurrent generative adversarial network module is configured to train a recurrent generative adversarial network model based on the first and second image datasets to obtain a trained recurrent generative adversarial network. The second zebra crossing image dataset acquisition module is configured to input the first zebra crossing image dataset into the trained recurrent generative adversarial network and output a second zebra crossing image dataset simulating a different scene. The zebra crossing image dataset acquisition module is configured to obtain a zebra crossing image dataset based on the first and second zebra crossing image datasets.

[0145] In this embodiment, the description of the first zebra crossing image dataset acquisition module can refer to the description of S110 in the above embodiment. The description of the second image dataset acquisition module can refer to the description of S120 in the above embodiment. The description of the recurrent generative adversarial network module can refer to the description of S130 in the above embodiment. The description of the second zebra crossing image dataset acquisition module can refer to the description of S140 in the above embodiment. The description of the zebra crossing image dataset acquisition module can refer to the description of S150 in the above embodiment.

[0146] In one embodiment, the feature extraction network module 20 includes a pooling layer module and a fully convolutional layer module. The pooling layer module is configured to perform feature extraction on multiple zebra crossing images containing real pixel category annotations based on the pooling layer of the feature extraction network to obtain a first feature atlas. The fully convolutional layer module is configured to perform feature extraction on the first feature atlas based on the first, second, third, and fourth fully convolutional layers of the feature extraction network to obtain multiple first zebra crossing feature maps.

[0147] Among them, the ratio of the number of output channels of the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer is 1:2:4:8.

[0148] In this embodiment, the description of the pooling layer module can refer to the description of S210 in the above embodiment. The description of the full convolution layer module can refer to the description of S220 in the above embodiment.

[0149] In one embodiment, the attention network module 30 includes a horizontal attention parameter feature map acquisition module, a vertical attention parameter feature map acquisition module, a spatial attention parameter feature map acquisition module, a normalized spatial attention parameter acquisition module, and a spatial attention weighted feature map acquisition module. The horizontal attention parameter feature map acquisition module is used to input each first zebra crossing feature map into the horizontal dimension spatial attention network, perform average pooling and vertical replication expansion on the first zebra crossing feature map, and obtain a horizontal attention parameter feature map. The vertical attention parameter feature map acquisition module is used to input each first zebra crossing feature map into the vertical dimension spatial attention network, perform average pooling and horizontal replication expansion on the first zebra crossing feature map, and obtain a vertical attention parameter feature map.

[0150] The spatial attention parameter feature map acquisition module is used to perform pixel-by-pixel summation of the horizontal attention parameter feature map and the vertical attention parameter feature map to obtain the spatial attention parameter feature map. The normalized spatial attention parameter acquisition module is used to input the spatial attention parameter feature map into the normalization layer to normalize the spatial attention parameters to obtain the normalized spatial attention parameters. The spatial attention weighted feature map acquisition module is used to perform pixel-by-pixel multiplication of the normalized spatial attention parameters with the first zebra crossing feature map to obtain the spatial attention weighted feature map.

[0151] In this embodiment, the relevant description of the horizontal attention parameter feature map acquisition module may refer to the relevant description of S310 in the above embodiment. The relevant description of the longitudinal attention parameter feature map acquisition module may refer to the relevant description of S320 in the above embodiment. The relevant description of the spatial attention parameter feature map acquisition module may refer to the relevant description of S330 in the above embodiment. The relevant description of the normalized spatial attention parameter acquisition module may refer to the relevant description of S340 in the above embodiment. The relevant description of the spatial attention weighted feature map acquisition module may refer to the relevant description of S350 in the above embodiment.

[0152] In one embodiment, the attention network module 30 further includes a convolution module, a first feature reshaping module, a feature transposition module, a channel-dimensional attention feature map acquisition module, a second feature reshaping module, a channel-attention weighted feature map acquisition module, and a second zebra crossing feature map acquisition module. The convolution module is used to input each first zebra crossing feature map into two identical convolution layers, and output a first feature map and a second feature map. The first feature reshaping module is used to perform feature map reshaping on the first feature map and the second feature map, respectively, to obtain a first reshaped feature map and a second reshaped feature map. The feature transposition module is used to perform feature transposition on the second reshaped feature map, to obtain a second transposed feature map. The channel-dimensional attention feature map acquisition module is used to perform matrix multiplication operation on the first reshaped feature map and the second transposed feature map, to obtain a channel feature map, and to perform feature map parameter normalization on the channel feature map, to obtain a channel-dimensional attention feature map.

[0153] The second feature reshaping module is used to reshape the feature map of the first zebra crossing feature map to obtain the first zebra crossing reshaped feature map. The channel attention weighted feature map acquisition module is used to perform matrix multiplication operation on the channel dimension attention feature map and the first zebra crossing reshaped feature map to obtain a reweighted feature map, and reshape the reweighted feature map to obtain a channel attention weighted feature map. The second zebra crossing feature map acquisition module is used to perform pixel-by-pixel addition of the spatial attention weighted feature map and the channel attention weighted feature map corresponding to each first zebra crossing feature map to obtain multiple second zebra crossing feature maps.

[0154] In this embodiment, the relevant description of the convolution module can refer to the relevant description of S360 in the embodiment. The relevant description of the first feature reshaping module can refer to the relevant description of S370 in the embodiment. The relevant description of the feature transposition module can refer to the relevant description of S380 in the embodiment. The relevant description of the channel dimension attention feature map acquisition module can refer to the relevant description of S390 in the embodiment. The relevant description of the second feature reshaping module can refer to the relevant description of S391 in the embodiment. The relevant description of the channel attention weighted feature map acquisition module can refer to the relevant description of S392 in the embodiment. The relevant description of the second zebra crossing feature map acquisition module can refer to the relevant description of S393 in the embodiment.

[0155] In one embodiment, the upsampling module includes an image resizing module and a pixel category acquisition module. The image resizing module is configured to input each second zebra crossing feature map into the five deconvolution layers of the upsampling network to restore the image size to the original zebra crossing image, thereby obtaining multiple second zebra crossing feature maps of the original image size. The pixel category acquisition module is configured to obtain a predicted pixel category corresponding to each pixel based on each second zebra crossing feature map of the original image size.

[0156] In this embodiment, the description of the image size restoration module can refer to the description of S410 in the above embodiment. The description of the pixel category acquisition module can refer to the description of S420 in the above embodiment.

[0157] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of this disclosure. The accompanying method claims provide elements of the various steps in an exemplary order and are not intended to be limited to a specific order or hierarchy.

[0158] Those skilled in the art will also appreciate that the various illustrative logical blocks, modules, and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, modules, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0159] The various illustrative logic blocks described in the embodiments of the present application, or modules can be implemented or operated by the design of a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination thereof. The general-purpose processor can be a microprocessor, alternatively, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0160] The steps of the methods or algorithms described in the embodiments of the present application can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium can be provided in an ASIC, which can be provided in a user terminal. Alternatively, the processor and the storage medium can also be provided in different components in the user terminal.

[0161] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application. It should be understood that the above description is only the specific implementation methods of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A zebra crossing detection method based on intelligent intersection, characterized in that: include: Obtain a zebra crossing image dataset, where the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations; Inputting the plurality of zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, and outputting a plurality of first zebra crossing feature maps; Inputting the multiple first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting multiple second zebra crossing feature maps; Inputting the plurality of second zebra crossing feature maps into the upsampling network of the zebra crossing detection model to perform image size restoration, and outputting the predicted pixel category corresponding to each pixel; Constructing a loss function according to the real pixel category and the predicted pixel category, and performing model training and optimization on the zebra crossing detection model according to the loss function to obtain a trained zebra crossing detection model; Detecting the position of the zebra crossing in the zebra crossing image to be detected according to the trained zebra crossing detection model; The step of inputting the plurality of first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting a plurality of second zebra crossing feature maps, comprises: Input each of the first zebra crossing feature maps into a horizontal dimension spatial attention network, perform average pooling and vertical replication expansion on the first zebra crossing feature map to obtain a horizontal attention parameter feature map; Input each of the first zebra crossing feature maps into the longitudinal dimension spatial attention network, perform average pooling and horizontal replication expansion on the first zebra crossing feature maps, and obtain a longitudinal attention parameter feature map; Adding the horizontal attention parameter feature map and the vertical attention parameter feature map pixel by pixel to obtain a spatial attention parameter feature map; Inputting the spatial attention parameter feature map into a normalization layer, normalizing the spatial attention parameters, and obtaining normalized spatial attention parameters; The normalized spatial attention parameter is multiplied pixel by pixel by the first zebra crossing feature map to obtain a spatial attention weighted feature map.

2. The zebra crossing detection method based on intelligent intersection according to claim 1, characterized in that: The zebra crossing image dataset is obtained, and the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations, including: Acquire a first image dataset containing zebra crossings, and perform pixel-by-pixel semantic segmentation and annotation on the first image dataset to obtain a first zebra crossing image dataset; Acquire a second image dataset containing a zebra crossing, where the second image dataset and the first image dataset are image datasets acquired in different scenes; Performing model training on a cyclic generative adversarial network based on the first image dataset and the second image dataset to obtain a trained cyclic generative adversarial network; Inputting the first zebra crossing image dataset into the trained recurrent generative adversarial network, and outputting a second zebra crossing image dataset simulating different scenarios; The zebra crossing image dataset is obtained according to the first zebra crossing image dataset and the second zebra crossing image dataset.

3. The zebra crossing detection method based on intelligent intersection according to claim 1, characterized in that: The step of inputting the plurality of zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, and outputting a plurality of first zebra crossing feature maps, comprises: Performing feature extraction on the plurality of zebra crossing images containing real pixel category annotations according to the pooling layer of the feature extraction network to obtain a first feature atlas; Performing feature extraction on the first feature atlas according to the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer of the feature extraction network to obtain the plurality of first zebra crossing feature maps; The ratio of the number of output channels of the first full convolutional layer, the second full convolutional layer, the third full convolutional layer, and the fourth full convolutional layer is 1:2:4:

8.

4. The zebra crossing detection method based on intelligent intersection according to claim 1, characterized in that: The step of inputting the plurality of first zebra crossing feature maps into the spatial attention network and the channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and outputting a plurality of second zebra crossing feature maps, further comprises: Input each of the first zebra crossing feature maps into two identical convolutional layers, and output a first feature map and a second feature map; Reshaping the first feature map and the second feature map to obtain a first reshaped feature map and a second reshaped feature map; performing feature transposition on the second reshaped feature map to obtain a second transposed feature map; Performing a matrix multiplication operation on the first reshaped feature map and the second transposed feature map to obtain a channel feature map, and normalizing the feature map parameters of the channel feature map to obtain a channel dimension attention feature map; Reshaping the first zebra crossing feature map to obtain a first zebra crossing reshaped feature map; Performing a matrix multiplication operation on the channel dimension attention feature map and the first zebra crossing reshaping feature map to obtain a reweighted feature map, and performing feature map reshaping on the reweighted feature map to obtain a channel attention weighted feature map; The spatial attention weighted feature map corresponding to each of the first zebra crossing feature maps and the channel attention weighted feature map are added pixel by pixel to obtain the multiple second zebra crossing feature maps.

5. The zebra crossing detection method based on intelligent intersection according to claim 1, characterized in that: Inputting the plurality of second zebra crossing feature maps into the upsampling network of the zebra crossing detection model to perform image size restoration, and outputting a predicted pixel category corresponding to each pixel, comprises: Inputting each of the second zebra crossing feature maps into the five deconvolution layers of the upsampling network to restore the image size to the image size of the original zebra crossing image, thereby obtaining a plurality of second zebra crossing feature maps of the original image size; According to the second zebra crossing feature map of each original image size, a predicted pixel category corresponding to each pixel is obtained.

6. A zebra crossing detection system based on intelligent intersection, characterized in that: include: An image data acquisition module is used to acquire a zebra crossing image dataset, wherein the zebra crossing image dataset includes a plurality of zebra crossing images with real pixel category annotations; a feature extraction network module, configured to input the plurality of zebra crossing images containing real pixel category annotations into a feature extraction network of a zebra crossing detection model for feature extraction, and output a plurality of first zebra crossing feature maps; An attention network module is used to input the multiple first zebra crossing feature maps into the spatial attention network and channel attention network of the zebra crossing detection model in parallel for feature extraction and fusion, and output multiple second zebra crossing feature maps; an upsampling module, configured to input the plurality of second zebra crossing feature maps into an upsampling network of the zebra crossing detection model to restore image size and output a predicted pixel category corresponding to each pixel; A model training module is used to construct a loss function according to the real pixel category and the predicted pixel category, and perform model training and optimization on the zebra crossing detection model according to the loss function to obtain a trained zebra crossing detection model; A detection module, configured to detect the position of a zebra crossing in a zebra crossing image to be detected based on the trained zebra crossing detection model; The attention network module includes: A horizontal attention parameter feature map acquisition module is used to input each of the first zebra crossing feature maps into a horizontal dimensional spatial attention network, perform average pooling and vertical replication expansion on the first zebra crossing feature maps, and obtain a horizontal attention parameter feature map; A longitudinal attention parameter feature map acquisition module is used to input each of the first zebra crossing feature maps into the longitudinal dimensional spatial attention network, perform average pooling and horizontal replication expansion on the first zebra crossing feature maps, and obtain a longitudinal attention parameter feature map; a spatial attention parameter feature map acquisition module, configured to add the horizontal attention parameter feature map and the vertical attention parameter feature map pixel by pixel to obtain a spatial attention parameter feature map; A normalized spatial attention parameter acquisition module is used to input the spatial attention parameter feature map into the normalization layer, normalize the spatial attention parameter, and obtain the normalized spatial attention parameter; A spatial attention weighted feature map acquisition module is used to multiply the normalized spatial attention parameter by the first zebra crossing feature map pixel by pixel to obtain a spatial attention weighted feature map.

7. The zebra crossing detection system based on intelligent intersection according to claim 6, characterized in that: The image data acquisition module includes: a first zebra crossing image dataset acquisition module, configured to acquire a first image dataset containing zebra crossings, and perform pixel-by-pixel semantic segmentation and annotation on the first image dataset to obtain a first zebra crossing image dataset; A second image data set acquisition module, configured to acquire a second image data set containing a zebra crossing, wherein the second image data set and the first image data set are image data sets acquired in different scenes; A cyclic generative adversarial network module, configured to perform model training on a cyclic generative adversarial network based on the first image dataset and the second image dataset to obtain a trained cyclic generative adversarial network; A second zebra crossing image dataset acquisition module is configured to input the first zebra crossing image dataset into the trained recurrent generative adversarial network and output a second zebra crossing image dataset simulating different scenarios; The zebra crossing image data set acquisition module is configured to obtain the zebra crossing image data set according to the first zebra crossing image data set and the second zebra crossing image data set.

8. The zebra crossing detection system based on intelligent intersection according to claim 6, characterized in that: The feature extraction network module includes: A pooling layer module, configured to perform feature extraction on the plurality of zebra crossing images containing real pixel category annotations according to the pooling layer of the feature extraction network to obtain a first feature atlas; A full convolutional layer module is used to perform feature extraction on the first feature map set according to the first full convolutional layer, the second full convolutional layer, the third full convolutional layer and the fourth full convolutional layer of the feature extraction network to obtain the multiple first zebra crossing feature maps.

9. The zebra crossing detection system based on intelligent intersection according to claim 6, characterized in that: The attention network module also includes: A convolution module, configured to input each of the first zebra crossing feature maps into two identical convolution layers, and output a first feature map and a second feature map; A first feature reshaping module is configured to reshape the first feature map and the second feature map to obtain a first reshaped feature map and a second reshaped feature map; a feature transposition module, configured to perform feature transposition on the second reshaped feature map to obtain a second transposed feature map; a channel-dimensional attention feature map acquisition module, configured to perform a matrix multiplication operation on the first reshaped feature map and the second transposed feature map to obtain a channel feature map, and perform feature map parameter normalization on the channel feature map to obtain a channel-dimensional attention feature map; A second feature reshaping module is configured to reshape the first zebra crossing feature map to obtain a first zebra crossing reshaped feature map; A channel attention weighted feature map acquisition module is used to perform a matrix multiplication operation on the channel dimension attention feature map and the first zebra crossing reshaping feature map to obtain a reweighted feature map, and perform feature map reshaping on the reweighted feature map to obtain a channel attention weighted feature map; The second zebra crossing feature map acquisition module is used to add the spatial attention weighted feature map and the channel attention weighted feature map corresponding to each of the first zebra crossing feature maps pixel by pixel to obtain the multiple second zebra crossing feature maps.

10. The zebra crossing detection system based on intelligent intersection according to claim 6, characterized in that: The upsampling module includes: An image resizing module is configured to input each of the second zebra crossing feature maps into the five deconvolution layers of the upsampling network to restore the image size to the image size of the original zebra crossing image, thereby obtaining a plurality of second zebra crossing feature maps of the original image size; The pixel category acquisition module is used to obtain the predicted pixel category corresponding to each pixel according to the second zebra crossing feature map of each original image size.

Images