A traffic flow prediction method and device, computer equipment and storage medium

Abstract

The embodiment of the application belongs to the technical field of image processing, and relates to a traffic flow prediction method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a high-altitude panoramic image and a road surface close-range image of a target traffic area; performing image expansion on the high-altitude panoramic image and the road surface close-range image to obtain a panoramic expanded image and a close-range expanded image; calling a traffic flow estimation network model; performing image feature extraction on the panoramic expanded image and the close-range expanded image according to a backbone network to obtain panoramic image features and close-range image features; performing image feature fusion on the panoramic image features and the close-range image features according to a neck network to obtain fused image features; and performing model prediction on the fused image features according to a head network to obtain traffic flow prediction data. The application can be used for related image processing in a financial technology business system, and improves the accuracy and reliability of traffic flow prediction.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a traffic flow prediction method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the acceleration of urbanization and the continuous growth of car ownership, traffic congestion has become an increasingly serious problem, making accurate traffic flow forecasting a key aspect of solving traffic issues. Accurate traffic flow forecasting can help traffic management departments formulate reasonable traffic management strategies in advance, optimize traffic light timing, improve road efficiency, and reduce traffic congestion and energy consumption.

[0003] There is a traffic flow prediction method that uses sensor devices (such as inductive loops, radar sensors, etc.) to detect vehicles in traffic areas, thereby predicting traffic flow data.

[0004] However, traditional methods generally suffer from low accuracy and reliability. Specifically, sensor devices are expensive to install and maintain, and are easily affected by environmental factors such as severe weather and equipment failure, which affect the accuracy and stability of data acquisition. In addition, sensor devices can usually only acquire local traffic information, making it difficult to comprehensively reflect the traffic conditions of the entire traffic area, thus limiting the accuracy and scope of traffic flow prediction. Summary of the Invention

[0005] The purpose of this application is to provide a traffic flow prediction method, device, computer equipment, and storage medium to solve the problems of low accuracy and reliability of traditional methods.

[0006] To address the aforementioned technical problems, this application provides a traffic flow prediction method, employing the following technical solution: Acquire aerial panoramic images and close-up images of the road surface of the target traffic area; The high-altitude panoramic image and the close-up road image are augmented to obtain an augmented panoramic image and an augmented close-up image; The traffic flow estimation network model is invoked, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; Based on the backbone network, image features are extracted from the panoramic augmented image and the close-up augmented image to obtain panoramic image features and close-up image features; Based on the neck network, image features of the panoramic image and the close-up image are fused to obtain fused image features; Based on the head network, the fused image features are used to perform model prediction to obtain traffic flow prediction data.

[0007] To address the aforementioned technical problems, this application also provides a traffic flow prediction device, which employs the following technical solution: The image acquisition module is used to acquire aerial panoramic images of the target traffic area and close-up images of the road surface. An image augmentation module is used to augment the high-altitude panoramic image and the close-up road image to obtain an augmented panoramic image and an augmented close-up image. The model invocation module is used to invoke the traffic flow estimation network model, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; The image feature extraction module is used to extract image features from the panoramic augmented image and the close-up augmented image based on the backbone network to obtain panoramic image features and close-up image features; An image feature fusion module is used to fuse the panoramic image features and the close-up image features based on the neck network to obtain fused image features; The model prediction module is used to perform model prediction on the fused image features based on the head network to obtain traffic flow prediction data.

[0008] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the traffic flow prediction method as described above.

[0009] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the traffic flow prediction method as described above.

[0010] This application provides a traffic flow prediction method, comprising: acquiring a high-altitude panoramic image and a close-up image of the road surface of a target traffic area; augmenting the high-altitude panoramic image and the close-up image of the road surface to obtain an augmented panoramic image and an augmented close-up image; calling a traffic flow estimation network model, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; extracting image features from the augmented panoramic image and the augmented close-up image based on the backbone network to obtain panoramic image features and close-up image features; fusing the panoramic image features and the close-up image features based on the neck network to obtain fused image features; and performing model prediction on the fused image features based on the head network to obtain traffic flow prediction data. Compared with the prior art, this application improves the accuracy and reliability of traffic flow prediction by acquiring a high-altitude panoramic image and a close-up image of the road surface of a target traffic area, augmenting the images, and then using a traffic flow estimation network model to extract, fuse, and predict features from the augmented images, thereby providing more effective decision support for traffic management. Attached Figure Description

[0011] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart illustrating the implementation of the traffic flow prediction method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the traffic flow prediction device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0014] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0016] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0017] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0018] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0019] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0020] It should be noted that the traffic flow prediction method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the traffic flow prediction device is generally set in the server / terminal device.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0022] Continue to refer to Figure 2 The diagram shows a flowchart of an embodiment of the traffic flow prediction method according to this application. The traffic flow prediction method includes steps S201, S202, S203, S204, S205, and S206.

[0023] In step S201, an aerial panoramic image of the target traffic area and a close-up image of the road surface are acquired.

[0024] In this embodiment of the application, the target traffic area refers to the specific geographical area that is of particular interest in the traffic flow prediction task, and for which traffic flow information needs to be acquired and predicted.

[0025] In this application embodiment, the high-altitude panoramic image refers to a road image that covers a large area of ​​roads and traffic conditions. It is mainly used to provide an overall overview of the traffic area and helps to understand the macroscopic distribution of traffic flow.

[0026] In this application embodiment, a close-up road surface image refers to an image that can clearly capture detailed information about vehicles, such as vehicle model and license plate, and is mainly used to provide a more accurate basis for accurately identifying and counting the number of vehicles.

[0027] In this embodiment of the application, by simultaneously acquiring images from two different perspectives, the application can comprehensively obtain traffic information of the traffic area, providing a rich data foundation for subsequent traffic flow prediction.

[0028] In step S202, the high-altitude panoramic image and the close-up road image are augmented to obtain an augmented panoramic image and an augmented close-up image.

[0029] In the embodiments of this application, image augmentation is mainly used to increase the amount of image data and improve the generalization ability of the model.

[0030] In practical applications, the number of images collected may be limited, and directly using these images for model training may lead to overfitting and an inability to accurately predict traffic flow in different scenarios. This application utilizes image augmentation techniques, such as rotation, flipping, scaling, and adding noise, to generate more image samples, enabling the model to learn more diverse features and patterns, thereby improving the model's prediction accuracy and robustness.

[0031] In step S203, the traffic flow estimation network model is invoked, which includes a backbone network, a neck network, and a head network.

[0032] In this embodiment, the traffic flow estimation network model adopts a hierarchical structure design. Different layers of the network have different functions, which can process and analyze the image step by step to achieve accurate prediction of traffic flow. The traffic flow estimation network model can be an improved YOLOv5 detection model, which includes a backbone network, a neck network, and a head network. The backbone network is used to extract image feature information. The neck network is used to fuse the features extracted by the backbone network, making the features learned by the neck network more diverse, and then handing them over to the head network for detection, thereby improving the network performance. The head network is used to predict the type and location of the target, and makes the prediction using the features fused by the neck network.

[0033] In step S204, image features are extracted from the panoramic augmented image and the close-up augmented image based on the backbone network to obtain panoramic image features and close-up image features.

[0034] In the embodiments of this application, the backbone network refers to a deep convolutional neural network (CNN) (such as ResNet, VGG, etc.). These networks have powerful feature extraction capabilities and can automatically learn features at different levels from images, including low-level edge and texture features and high-level semantic features.

[0035] In this embodiment of the application, feature extraction is performed on panoramic and close-up expanded images respectively to obtain image features from two different perspectives, providing a foundation for subsequent feature fusion.

[0036] In step S205, the panoramic image features and the close-up image features are fused according to the neck network to obtain fused image features.

[0037] In the embodiments of this application, the role of the neck network is to fuse features from different sources and at different levels, so that the model can make full use of various feature information and improve the ability to predict traffic flow.

[0038] In the embodiments of this application, the feature fusion method includes concatenation, addition, etc.

[0039] In this embodiment, feature fusion organically combines global features of a panoramic image with local features of a close-up image to form a more comprehensive and accurate image feature representation. In step S206, traffic flow prediction data is obtained by performing model prediction on the fused image features based on the head network.

[0040] In this embodiment, the head network consists of a fully connected layer and an output layer. The fully connected layer further processes and transforms the fused image features, while the output layer generates traffic flow prediction data based on the output of the fully connected layer. Specifically, the output layer can use different activation functions, such as the Softmax function for multi-classification problems and the linear activation function for regression problems, depending on the task requirements of traffic flow prediction.

[0041] In some optional implementations of the embodiments of this application, the traffic flow estimation network model described above can use a lightweight feature extraction network GhostNet to replace the original convolutional network in the backbone network. Specifically, GhostNet is used in the backbone network to obtain a large number of feature maps, which not only preserves global features but also reduces the overall network computation, thereby making the network lightweight while maintaining accuracy.

[0042] In some optional implementations of the embodiments of this application, the traffic flow estimation network model introduces an attention mechanism (Efficient Channel Attention, ECA) after the concatenation operation of the neck network. Specifically, the input feature map is subjected to global average pooling, transforming the input feature map from a matrix of [h,w,c] into a vector of [1,1,c], where h is the width, w is the length, and c is the number of channels. An adaptive one-dimensional convolution kernel size is calculated based on the number of channels in the input feature map. The one-dimensional convolution kernel size is used in the one-dimensional convolution to obtain the weight for each channel of the feature map. Finally, the normalized weights are multiplied with the original input feature map channel by channel to generate a weighted feature map.

[0043] In some optional implementations of the embodiments of this application, the above-mentioned traffic flow estimation network model introduces a BiFPN structure on the basis of the original multi-scale fusion detection module "Feature Pyramid Network (FPN)" + "Path Aggregation Network (PAN)" structure. Specifically, the feature maps of different scales in the backbone network and the feature maps after concatenation in the neck network are added together and then convolved, which increases the amount of information in the image features.

[0044] In some optional implementations of the embodiments of this application, the above-described traffic flow estimation network model replaces the Silu (Sigmoid Linear Unit) activation function in the neck network with a nonlinear activation function.

[0045] This application provides a traffic flow prediction method, comprising: acquiring a high-altitude panoramic image and a close-up image of the road surface of a target traffic area; augmenting the high-altitude panoramic image and the close-up image of the road surface to obtain an augmented panoramic image and an augmented close-up image; calling a traffic flow estimation network model, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; extracting image features from the augmented panoramic image and the augmented close-up image based on the backbone network to obtain panoramic image features and close-up image features; fusing the panoramic image features and the close-up image features based on the neck network to obtain fused image features; and performing model prediction based on the fused image features based on the head network to obtain traffic flow prediction data. Compared with the prior art, this application improves the accuracy and reliability of traffic flow prediction by acquiring a high-altitude panoramic image and a close-up image of the road surface of a target traffic area, augmenting the images, and then using a traffic flow estimation network model to extract, fuse, and predict features from the augmented images, thereby providing more effective decision support for traffic management.

[0046] In some optional implementations of the embodiments of this application, the steps of image augmentation of the high-altitude panoramic image and the close-up road image to obtain the panoramic augmented image and the close-up augmented image specifically include the following steps: Rotate the aerial panoramic image and the close-up road surface image respectively to obtain the panoramic augmented image and the close-up augmented image; and / or Image shifting is performed on both the aerial panoramic image and the close-up road surface image to obtain panoramic augmented images and close-up augmented images, respectively; and / or Image scaling is performed on both the aerial panoramic image and the close-up road surface image to obtain panoramic augmented images and close-up augmented images, respectively; and / or Image enhancement is performed on both the aerial panoramic image and the close-up road surface image to obtain the panoramic augmented image and the close-up augmented image, respectively; and / or Noise injection is performed on both the aerial panoramic image and the close-up road surface image to obtain the panoramic augmented image and the close-up augmented image, respectively; and / or Contrast transformations were performed on the aerial panoramic image and the close-up road image to obtain the panoramic augmented image and the close-up augmented image, respectively.

[0047] In this embodiment, image data augmentation includes one or more of the following processes: image rotation, image movement, image scaling, image enhancement, noise injection, and contrast transformation. Specifically: (1) Image rotation: The system performs image rotation operations on both the aerial panoramic image and the close-up road image. The rotation angle can be set according to actual needs, such as 30°, 45°, 60°, etc. By rotating the image, the content can be displayed from different perspectives, increasing the diversity of the image's viewpoint. For high-altitude panoramic images, rotation can simulate scenes captured from different flight angles. For close-up images of the road surface, rotation can present the appearance features of the target object in different directions. After rotation processing, corresponding panoramic and close-up images are obtained. (2) Image movement: Image movement operations can be performed on both the aerial panoramic image and the close-up road image. The direction of movement can be horizontal, vertical, or any diagonal direction, and the distance of movement can also be adjusted according to specific requirements. Image movement can simulate scenes captured by image acquisition devices at different locations, increasing the positional diversity of images. For example, for high-altitude panoramic images, movement can show the relative positional changes of different areas in the image; for close-up images of roads, movement can simulate the imaging of target objects at different locations. After movement processing, corresponding panoramic and close-up images are obtained. (3) Image scaling: The system performs image scaling operations on both the aerial panoramic image and the close-up road image. The scaling ratio can be set to different values, such as 0.8x, 1.2x, 1.5x, etc. By scaling, the size of the image can be changed to simulate the image effect taken at different distances. For high-altitude panoramic images, scaling can present a large-scale scene at different heights or shooting distances. For close-up images of the road surface, scaling can highlight or weaken the details of the target object. After scaling, the corresponding panoramic and close-up images are obtained. (4) Image enhancement: Image enhancement operations were performed on both the aerial panoramic image and the close-up road image. Various methods can be used for image enhancement, such as histogram equalization and sharpening. Histogram equalization can adjust the grayscale distribution of the image, enhance the contrast of the image, and make the details of the image clearer. Sharpening can highlight the edges and texture information of the image and improve the image clarity. Image enhancement can improve image quality, making the expanded image more suitable for practical applications. After enhancement, corresponding panoramic and close-up expanded images are obtained.

[0048] (5) Noise injection: Noise injection is performed on both the aerial panoramic image and the close-up road image. The noise type can include Gaussian noise, salt-and-pepper noise, etc., and the intensity of the injected noise can be adjusted according to the actual situation. Noise injection can simulate the interference factors encountered during image acquisition in real-world scenarios, increasing the noise diversity of the image. After noise injection processing, corresponding panoramic and close-up augmented images are obtained, making the augmented images closer to the actual acquired images and improving the robustness of the model.

[0049] (6) Contrast transformation: Contrast transformation is performed on both the aerial panoramic image and the close-up road image. Contrast transformation can be achieved by adjusting the image's contrast parameters, such as setting the contrast to different multiples. Contrast transformation can alter the brightness of an image, highlighting or diminishing certain features. For aerial panoramic images, contrast transformation can better showcase the brightness differences between different areas; for close-up road images, contrast transformation can make the outlines of target objects clearer. After contrast transformation processing, corresponding panoramic and close-up images are obtained.

[0050] Compared with existing technologies, this application comprehensively expands high-altitude panoramic images and close-up road images by using a variety of techniques such as image rotation, movement, scaling, enhancement, noise injection, and contrast transformation. This generates panoramic and close-up expanded images with rich diversity, providing more abundant and representative image data for fields such as intelligent traffic monitoring and urban planning, thereby improving the performance and accuracy of related systems.

[0051] In some optional implementations of the embodiments of this application, the steps of extracting image features from the panoramic augmented image and the close-up augmented image based on the backbone network to obtain panoramic image features and close-up image features specifically include the following steps: Based on the traffic flow detection model, vehicles are detected in the near-field augmented image to obtain the number of vehicles, and the number of vehicles is vectorized into a vehicle number vector. Based on the backbone network, image features of the panoramic augmented image and the close-up augmented image are extracted respectively to obtain the initial panoramic image features and the initial close-up image features; The vehicle quantity vector is combined with the initial panoramic image features and the initial close-up image features respectively to obtain the panoramic image features and the close-up image features.

[0052] In this embodiment of the application, the application can perform vehicle detection on the close-up augmented image according to the pre-trained traffic flow detection model. The traffic flow detection model can adopt a deep learning-based target detection algorithm, such as Faster R-CNN, YOLO, etc. Through the traffic flow detection model, vehicles in the close-up augmented image can be accurately detected and the number of vehicles can be counted.

[0053] In this embodiment of the application, the obtained vehicle quantity can be vector quantized to obtain a vehicle quantity vector. The vector quantization method can be selected according to actual needs. For example, one-hot encoding can be used to map the vehicle quantity into a fixed-dimensional vector. Assuming that the possible range of the vehicle quantity is 0 to N, the vehicle quantity vector can be a vector of length N+1, where the element corresponding to the vehicle quantity position is 1 and the rest are 0.

[0054] In this embodiment of the application, the application can use a backbone network to extract image features from the panoramic augmented image and the close-up augmented image respectively. Through the forward propagation of the backbone network, the initial panoramic image features of the panoramic augmented image and the initial close-up image features of the close-up augmented image can be extracted. These initial image features contain visual information of the image, such as color, texture, shape, etc.

[0055] In this application embodiment, the obtained vehicle quantity vector can be merged with the obtained initial panoramic image features and initial close-up image features respectively using a first feature merging method. The first feature merging can be performed in various ways. For example, it can be performed by concatenation, which concatenates the vehicle quantity vector with the initial image features in the channel dimension to obtain the fused panoramic image features and close-up image features.

[0056] In practical applications, assuming the vehicle quantity vector has dimension M, the initial panoramic image feature dimension is C1×H1×W1 (where C1 is the number of channels, H1 is the height, and W1 is the width), and the initial close-up image feature dimension is C2×H2×W2 (where C2 is the number of channels, H2 is the height, and W2 is the width). After stitching, the panoramic image feature dimension becomes (C1+M)×H1×W1, and the close-up image feature dimension becomes (C2+M)×H2×W2.

[0057] Compared with existing technologies, this application enhances the ability of image features to express road scenes by incorporating vehicle quantity information into image features in the form of vectors, thereby improving the accuracy and reliability of subsequent related tasks.

[0058] In some optional implementations of the embodiments of this application, the steps of extracting image features of the panoramic augmented image and the close-up augmented image from the backbone network to obtain the initial panoramic image features and the initial close-up image features specifically include the following steps: The global color distribution of the panoramic augmented image and the close-up augmented image are obtained respectively to obtain the panoramic color features and the close-up color features; The texture details of the panoramic augmented image and the close-up augmented image are obtained respectively to obtain the panoramic texture features and the close-up texture features; The shape structure of the panoramic augmented image and the close-up augmented image are obtained respectively, and the panoramic shape features and the close-up shape features are obtained. Panoramic color features, panoramic texture features, and panoramic shape features are aligned, and then the aligned panoramic color features, panoramic texture features, and panoramic shape features are fused to obtain the initial panoramic image features. Near-field color features, near-field texture features, and near-field shape features are aligned, and the aligned near-field color features, near-field texture features, and near-field shape features are fused to obtain the initial near-field image features.

[0059] In this application embodiment, the global color distribution can be captured through color histogram or HSV (Hue, Saturation, Value) / Lab (Lab color space) spatial statistics, or local color patterns can be extracted through shallow features of a convolutional neural network (CNN) to obtain the global color distribution of the image. Specifically: (1) Color histogram: Convert the image to HSV space, count the pixel distribution of each channel (e.g., 8 bins / channel), and generate a 256-dimensional vector; (2) CNN shallow features: Use a pre-trained model (such as the first two layers of ResNet-18) to extract low-level features, and obtain color-related feature vectors (such as 512-dimensional) through global average pooling (GAP).

[0060] In this application embodiment, the application may employ Local Binary Patterns (LBP), Gabor filter banks, or intermediate layer features of convolutional neural networks (CNN) to capture texture details. Specifically: (1) LBP: Calculate the binary code of the 8-neighborhood of each pixel and calculate the global histogram (such as the classic 59-dimensional LBP): (2) Gabor filter: Use multi-scale, multi-directional Gabor kernel convolution image to extract texture frequency domain features (e.g., 40-dimensional: 5 scales × 8 directions); (3) CNN intermediate layer: extract the layer2 output of ResNet-18 (feature map size halved) and obtain texture features (e.g. 256 dimensions) through GAP.

[0061] In this application embodiment, the shape structure can be described by high-level semantic features of edge detection (such as the Canny operator), contour extraction (such as OpenCV's findContours), or convolutional neural networks (CNN). Specifically: (1) Edge features: Apply the Canny operator to generate a binary edge map and statistically analyze the edge direction histogram (e.g., 16 bins); (2) Contour features: Extract the body contour point set and calculate Hu invariant moments (7-dimensional) or Zernike moments (high-order shape descriptors); (3) CNN high-level features: use the layer 4 output of ResNet-18 (high semantic features), and then obtain shape-related features (such as 512 dimensions) through global average pooling (GAP).

[0062] In this embodiment, due to the large differences in the dimensions of different features (e.g., 59 dimensions in LBP vs. 512 dimensions in CNN), direct concatenation would cause the model to be biased towards high-dimensional features. This application avoids this bias by aligning and unifying the dimensions of the features. Specifically, this application can perform feature alignment using fully connected layer projection or 1×1 convolution. (1) Fully connected layer projection: Add an independent FC layer to each feature branch and map them to the same dimension (e.g., project them to 256 dimensions). (2) 1×1 convolution: If the feature is a 2D feature map (such as CNN output), use 1×1 convolution to adjust the number of channels.

[0063] In this application embodiment, the above feature fusion can be achieved through simple concatenation, attention mechanism fusion, or gated fusion. Specifically: (1) Simple splicing: directly splice the aligned features along the channel dimension, retaining all information; (2) Attention mechanism fusion: Dynamically weighting different features through self-attention or cross-attention; (3) Gated fusion: The contribution of each mode is dynamically adjusted by learning gated signals.

[0064] Compared with existing technologies, this application generates initial panoramic / close-up image features by independently extracting color, texture, and shape features from panoramic / close-up images, followed by feature alignment and fusion. In some optional implementations of the embodiments of this application, the step of fusing panoramic image features and close-up image features based on the neck network to obtain fused image features specifically includes the following steps: Image feature fusion is performed on panoramic image features and close-up image features to obtain initial fused image features; Global average pooling is performed on the features of the initial fused image to obtain the global average pooling value, which includes the number of channels; Calculate the size of the one-dimensional convolution kernel for the fused image features based on the number of channels; Perform one-dimensional convolution based on the one-dimensional convolution kernel size to obtain the channel weights for each channel; The initial fused image features are multiplied channel by channel according to the channel weights to obtain the fused image features.

[0065] In this embodiment, the application can fuse panoramic image features and close-up image features to obtain initial fused image features. The fusion method can employ a common concatenation operation, concatenating the panoramic image features and close-up image features along the channel dimension. Assuming the panoramic image features have a dimension of C1×H1×W1 (where C1 is the number of channels, H1 is the height, and W1 is the width), and the close-up image features have a dimension of C2×H2×W2, after concatenation, the dimension of the initial fused image features becomes (C1+C2)×H×W (where H and W can be uniformly processed according to actual conditions, such as using interpolation to make H1=H2=H and W1=W2=W).

[0066] In this embodiment, the initial fused image features can be subjected to global average pooling to obtain a global average pooling value. Global average pooling is an operation that averages the feature values ​​of each channel across the spatial dimensions (height and width), thereby transforming the initial fused image features from a three-dimensional tensor ((C1+C2)×H×W) into a two-dimensional vector ((C1+C2)×1×1). Each element in this two-dimensional vector corresponds to the global average pooling value of a channel, which contains the global information of that channel across the entire image.

[0067] In this embodiment, the one-dimensional convolution kernel size for fused image features can be calculated based on the number of channels (i.e., C1+C2). The calculation of the one-dimensional convolution kernel size can be determined based on empirical formulas or experimental adjustments; for example, k=max(3, Calculated in the manner of ×2+1), where This indicates a floor operation. The calculated kernel size can adaptively adjust according to changes in the number of channels, ensuring the effectiveness of the convolution operation while avoiding increased computation due to an excessively large kernel.

[0068] In this embodiment, a one-dimensional convolution operation can be performed based on the calculated one-dimensional convolution kernel size to obtain the channel weights for each channel. The one-dimensional convolution operation is performed along the channel dimension, using the global average pooling value as input. Through learning the convolution kernel, the correlation and importance information between channels can be captured. After one-dimensional convolution, a one-dimensional vector with the same dimension as the number of channels is obtained. Each element in this vector is the channel weight of the corresponding channel, and its value is typically between 0 and 1, representing the importance of that channel in the fused features.

[0069] In this embodiment, the initial fused image features can be multiplied channel by channel based on the calculated channel weights to obtain the final fused image features. Channel-by-channel multiplication means multiplying the feature value of each channel of the initial fused image features with its corresponding channel weight. This allows for reweighting of the initial fused image features according to the importance of the channels, highlighting the feature information of important channels and suppressing redundant information of unimportant channels, thereby obtaining higher quality fused image features.

[0070] Compared with existing technologies, this application utilizes a specific fusion strategy to fully leverage information from panoramic and close-up image features, generating high-quality fused image features and improving the performance of computer vision tasks.

[0071] In some optional implementations of the embodiments of this application, the step of performing model prediction on the fused image features based on the head network to obtain traffic flow prediction data specifically includes the following steps: The panoramic image features, close-up image features, and fused image features are combined using a second feature merging process to obtain the merged image features. The merged image features are input into the first fully connected layer for the first feature dimensionality reduction, resulting in preliminary nonlinear relationship features. The initial nonlinear relationship features are input into the second fully connected layer for second feature dimensionality reduction to obtain intermediate nonlinear relationship features. The intermediate nonlinear relationship features are input into the output layer for feature mapping to obtain traffic flow prediction data.

[0072] In this embodiment, panoramic image features, close-up image features, and fused image features can be combined using a second feature merging process to obtain merged image features. This second feature merging can employ methods such as feature stitching or weighted feature fusion. For example, feature stitching connects three feature vectors in a specific order to form a longer feature vector; weighted feature fusion assigns a weight to each feature based on its importance, and then adds the weighted features together to obtain the merged image features.

[0073] In this embodiment, the head network is a deep learning model whose structure can be designed according to specific needs. The head network includes a first fully connected layer, a second fully connected layer, and an output layer. Specifically: (1) The first fully connected layer is mainly used to map high-dimensional features to intermediate dimensions and extract preliminary nonlinear relationships, where: Weight matrix: shape [6144, 1024]; Bias vector: shape

[1024] ; Output dimensions: [batch_size, 1024]; (2) The second fully connected layer is mainly used to further compress the feature dimension and enhance the model's expressive power, wherein: Weight matrix: shape [1024, 512]; Bias vector: shape is

[512] ; Output dimensions: [batch_size, 512]; (3) The output layer is mainly used to map intermediate features to the final predicted value (traffic flow), where: Weight matrix: shape [512,1] (single-output regression task); Bias vector: shape is [1]; Output dimension: [batch_size,1], which is the traffic flow prediction value for each sample.

[0074] In this embodiment, the merged image features are input into a pre-trained head network for model prediction to obtain traffic flow prediction data. During the training phase, the head network is trained using a large amount of image data labeled with real traffic flow data, and the network parameters are adjusted so that the network can learn the mapping relationship between image features and traffic flow. During the prediction phase, the merged image features are input into the trained head network, and the network outputs traffic flow prediction data.

[0075] Compared with existing technologies, this application combines panoramic image features, close-up image features, and fused image features into a second feature, and then inputs the combined image features into the head network for model prediction, thereby obtaining more accurate traffic flow prediction data and overcoming the problem of low traffic flow prediction accuracy in existing technologies.

[0076] In some optional implementations of the embodiments of this application, after the above step of performing model prediction on the fused image features based on the head network to obtain traffic flow prediction data, the following step is further included: The system calls a preset traffic expert system and inputs high-altitude panoramic images, close-up road surface images, and traffic flow prediction data into the traffic expert system for system prediction to obtain traffic flow inference data. The model loss function is calculated based on traffic flow prediction data and traffic flow inference data. The first parameter of the traffic flow estimation network model is optimized based on the model loss function.

[0077] In this embodiment, the application can invoke a pre-defined traffic expert system. This traffic expert system is an intelligent system built upon the knowledge and experience of experts in the traffic field, capable of in-depth analysis and reasoning of traffic conditions. Aerial panoramic images, close-up road surface images, and traffic flow prediction data are input into the traffic expert system. The aerial panoramic images provide macroscopic information such as the overall road layout and vehicle distribution; the close-up road surface images more clearly display detailed vehicle characteristics, such as vehicle type and driving status; and the traffic flow prediction data provides the traffic expert system with preliminary estimation results. Based on this input information, and combined with built-in traffic rules, traffic flow models, historical data, and other knowledge, the traffic expert system performs system prediction to obtain traffic flow inference data.

[0078] In this embodiment, the application can calculate a model loss function based on traffic flow prediction data and traffic flow inference data. The model loss function is an indicator that measures the difference between the prediction results of the traffic flow estimation network model and the inference results of the traffic expert system. Commonly used loss functions include mean squared error (MSE) and mean absolute error (MAE). For example, when using mean squared error as the loss function, its calculation formula is:

[0079] in, For the sample size, Data for inferring traffic flow. This provides traffic flow prediction data. By calculating the loss function, the prediction error of the traffic flow estimation network model can be quantified.

[0080] In this embodiment, the traffic flow estimation network model can be optimized for its first parameter based on the model loss function. The purpose of parameter optimization is to adjust the parameters of the traffic flow estimation network model so that the model's prediction results are closer to the inference results of the traffic expert system, thereby reducing prediction errors. Commonly used parameter optimization algorithms include stochastic gradient descent (SGD) and Adam. Taking stochastic gradient descent as an example, its parameter update formula is:

[0081] in, These are the model parameters at the current moment. For learning rate, For the loss function with respect to the parameters The gradient is calculated. By iteratively updating the parameters, the loss function gradually decreases, thereby improving the prediction accuracy of the traffic flow estimation network model.

[0082] Compared with existing technologies, this application improves the accuracy and reliability of traffic flow estimation by combining the preliminary prediction results of the traffic flow estimation network model with the inference results of the traffic expert system, calculating the model loss function and optimizing the parameters of the traffic flow estimation network model, thereby overcoming the problems of limited prediction accuracy and lack of optimization mechanism of single model in existing technologies.

[0083] In some optional implementations of the embodiments of this application, the traffic flow prediction data includes vehicle prediction location data. After the step of performing model prediction on the fused image features based on the head network to obtain the traffic flow prediction data, the following steps are also included: Acquire vehicle real location data corresponding to high-altitude panoramic images and close-up road surface images; The prediction accuracy is calculated based on the predicted vehicle location data and the actual vehicle location data. The prediction accuracy is expressed as:

[0084]

[0085]

[0086] Wherein, F1 represents prediction accuracy, P represents precision accuracy, R represents recall accuracy, TP represents the number of vehicles detected, FP represents the number of vehicles that are not actually vehicles but are detected as vehicles, and FN represents the number of vehicles that are actually vehicles but are not detected. The second parameter of the traffic flow estimation network model is optimized based on the prediction accuracy.

[0087] In this embodiment, the application can acquire vehicle real-time location data corresponding to high-altitude panoramic images and close-up road surface images. This data can be acquired in various ways, such as by installing high-precision sensors (e.g., lidar, millimeter-wave radar) at specific locations on the road. These sensors can detect vehicle location information in real time and transmit the data to a data processing center; or by using pre-marked map data combined with vehicle trajectory information to calculate the vehicle's real-time location at the corresponding moment in the image.

[0088] In this embodiment, the prediction accuracy can be calculated based on vehicle predicted location data (predicted by a traffic flow estimation network model) and actual vehicle location data. By calculating the F1 score, the accuracy and comprehensiveness of the model's prediction can be comprehensively measured.

[0089] In this embodiment, the traffic flow estimation network model can be optimized for a second parameter based on the prediction accuracy. Specifically, the calculated F1 score is used as the optimization objective, and optimization algorithms (such as stochastic gradient descent, Adam algorithm, etc.) are employed to adjust the parameters of the traffic flow estimation network model. Taking stochastic gradient descent as an example, the parameter update formula is:

[0090] in, These are the model parameters at the current moment. For learning rate, For the loss function with respect to the parameters The gradient.

[0091] In practical applications, the loss function can be constructed based on the F1 score; for example, using 1... F1, as the loss function, gradually increases its value by iteratively updating the parameters, thereby improving the model's prediction accuracy.

[0092] Compared with existing technologies, this application improves the accuracy and reliability of traffic flow estimation by acquiring the real location data of vehicles corresponding to the image, calculating the prediction accuracy, and optimizing the second parameter of the traffic flow estimation network model based on the accuracy, thereby overcoming the problem of lacking an effective feedback optimization mechanism in existing technologies.

[0093] In some optional implementations of the embodiments of this application, if both the model loss function and the loss function constructed from the F1 score are calculated simultaneously during the process of adjusting the model parameters, this application assigns weights to the model loss function and the loss function constructed from the F1 score, and then merges the assigned weights to obtain a merged loss function with respect to the parameters. The gradient is calculated, and finally, the merged loss function is calculated with respect to the parameters. The gradient is input into the stochastic gradient descent method described above, thereby obtaining the adjustment parameters of the traffic flow estimation network model.

[0094] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0095] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0096] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0097] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0098] Further reference Figure 3 As a response to the above Figure 2 The present application provides an embodiment of a traffic flow prediction device, which is similar to the method shown. Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0099] like Figure 3 As shown, the traffic flow prediction device 200 of this application embodiment includes: Image acquisition module 210 is used to acquire aerial panoramic images and close-up images of the road surface of the target traffic area; Image augmentation module 220 is used to augment high-altitude panoramic images and close-up road surface images to obtain panoramic augmented images and close-up augmented images; The model calling module 230 is used to call the traffic flow estimation network model, which includes a backbone network, a neck network, and a head network. The image feature extraction module 240 is used to extract image features from the panoramic augmented image and the close-up augmented image based on the backbone network, so as to obtain panoramic image features and close-up image features. The image feature fusion module 250 is used to fuse panoramic image features and close-up image features based on the neck network to obtain fused image features. The model prediction module 260 is used to perform model prediction on the features of the fused image based on the head network to obtain traffic flow prediction data.

[0100] In this embodiment, a traffic flow prediction device 200 is provided, comprising: an image acquisition module 210 for acquiring a high-altitude panoramic image and a close-up image of the road surface of a target traffic area; an image augmentation module 220 for augmenting the high-altitude panoramic image and the close-up image of the road surface to obtain an augmented panoramic image and an augmented close-up image; a model invocation module 230 for invoking a traffic flow estimation network model, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; an image feature extraction module 240 for extracting image features from the augmented panoramic image and the augmented close-up image based on the backbone network to obtain panoramic image features and close-up image features; an image feature fusion module 250 for fusing the panoramic image features and the close-up image features based on the neck network to obtain fused image features; and a model prediction module 260 for performing model prediction on the fused image features based on the head network to obtain traffic flow prediction data. Compared with existing technologies, this application improves the accuracy and reliability of traffic flow prediction by acquiring high-altitude panoramic images and close-up images of the road surface of the target traffic area, and then performing image augmentation processing. The augmented images are then used to extract, fuse, and predict features using a traffic flow estimation network model, thereby providing more effective decision support for traffic management.

[0101] In some optional implementations of the embodiments of this application, the above-mentioned image augmentation module includes: The image rotation submodule is used to rotate the aerial panoramic image and the close-up road image respectively to obtain the panoramic augmented image and the close-up augmented image; and / or The image shifting submodule is used to shift the aerial panoramic image and the close-up road image respectively to obtain the panoramic augmented image and the close-up augmented image; and / or The image scaling submodule is used to scale the aerial panoramic image and the close-up road image respectively, to obtain the panoramic expanded image and the close-up expanded image; and / or The image enhancement submodule is used to enhance the aerial panoramic image and the close-up road image respectively, to obtain the panoramic augmented image and the close-up augmented image; and / or The noise injection submodule is used to inject noise into the high-altitude panoramic image and the close-up road image respectively, to obtain the panoramic augmented image and the close-up augmented image; and / or The contrast transformation submodule is used to perform contrast transformation on the high-altitude panoramic image and the close-up road image respectively, to obtain the panoramic augmented image and the close-up augmented image.

[0102] In some optional implementations of the embodiments of this application, the above-mentioned image feature extraction module includes: The vehicle detection submodule is used to detect vehicles in the near-field augmented image based on the traffic flow detection model, obtain the number of vehicles, and vectorize the number of vehicles into a vehicle number vector. The image feature extraction submodule is used to extract image features from the panoramic augmented image and the close-up augmented image based on the backbone network, respectively, to obtain the initial panoramic image features and the initial close-up image features; The first feature merging submodule is used to merge the vehicle quantity vector with the initial panoramic image features and the initial close-up image features to obtain the panoramic image features and the close-up image features.

[0103] In some optional implementations of the embodiments of this application, the above-mentioned image feature extraction submodule includes: The color feature acquisition unit is used to acquire the global color distribution of the panoramic augmented image and the close-up augmented image respectively, and obtain the panoramic color features and the close-up color features. The texture feature acquisition unit is used to acquire the texture details of the panoramic augmented image and the close-up augmented image respectively, and obtain the panoramic texture features and the close-up texture features. The shape feature acquisition unit is used to acquire the shape structure of the panoramic augmented image and the close-up augmented image respectively, and obtain the panoramic shape features and the close-up shape features; The feature alignment unit is used to align panoramic color features, panoramic texture features, and panoramic shape features, and to fuse the aligned panoramic color features, panoramic texture features, and panoramic shape features to obtain the initial panoramic image features. The feature fusion unit is used to align the near-field color features, near-field texture features, and near-field shape features, and then fuse the aligned near-field color features, near-field texture features, and near-field shape features to obtain the initial near-field image features.

[0104] In some optional implementations of the embodiments of this application, the above-mentioned image feature fusion module includes: The image feature fusion submodule is used to fuse panoramic image features and close-up image features to obtain initial fused image features. The average pooling submodule is used to perform global average pooling on the initial fused image features to obtain a global average pooling value, which includes the number of channels. The kernel calculation submodule is used to calculate the size of the one-dimensional convolution kernel for fusing image features based on the number of channels; The one-dimensional convolution submodule is used to perform one-dimensional convolution based on the size of the one-dimensional convolution kernel to obtain the channel weights of each channel; The channel-by-channel multiplication submodule is used to multiply the initial fused image features channel by channel according to the channel weights to obtain the fused image features.

[0105] In some optional implementations of the embodiments of this application, the above-mentioned model prediction module includes: The second feature merging submodule is used to merge panoramic image features, close-up image features and fused image features to obtain merged image features. The model prediction submodule is used to input the merged image features into the head network for model prediction to obtain traffic flow prediction data.

[0106] In some optional implementations of the embodiments of this application, the traffic flow prediction device 200 includes: The system prediction module is used to call the preset traffic expert system and input the high-altitude panoramic image, the road surface close-up image and the traffic flow prediction data into the traffic expert system to make system predictions and obtain traffic flow inference data. The loss function calculation module is used to calculate the model loss function based on traffic flow prediction data and traffic flow inference data; The first parameter optimization module is used to optimize the first parameter of the traffic flow estimation network model based on the model loss function.

[0107] In some optional implementations of the embodiments of this application, the traffic flow prediction device 200 includes: The real vehicle location acquisition module is used to acquire real vehicle location data corresponding to the high-altitude panoramic image and the close-up road surface image; The prediction accuracy calculation module is used to calculate the prediction accuracy based on the vehicle's predicted location data and the vehicle's actual location data. The prediction accuracy is expressed as:

[0108]

[0109]

[0110] Wherein, F1 represents prediction accuracy, P represents precision accuracy, R represents recall accuracy, TP represents the number of vehicles detected, FP represents the number of vehicles that are not actually vehicles but are detected as vehicles, and FN represents the number of vehicles that are actually vehicles but are not detected. The second parameter optimization module is used to optimize the second parameter of the traffic flow estimation network model based on the prediction accuracy.

[0111] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of this application.

[0112] Computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected via a system bus. It should be noted that only computer device 300 with components 310-330 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0113] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0114] The memory 310 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 may be an internal storage unit of the computer device 300, such as the hard disk or memory of the computer device 300. In other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 310 may include both internal storage units and external storage devices of the computer device 300. In the embodiments of this application, the memory 310 is typically used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for traffic flow prediction methods. In addition, the memory 310 can also be used to temporarily store various types of data that have been output or will be output.

[0115] In some embodiments, processor 320 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 320 is typically used to control the overall operation of computer device 300. In embodiments of this application, processor 320 is used to execute computer-readable instructions stored in memory 310 or process data, such as computer-readable instructions for running a traffic flow prediction method.

[0116] The network interface 330 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 300 and other electronic devices.

[0117] The computer equipment provided in this application acquires aerial panoramic images and close-up images of the road surface of the target traffic area, performs image augmentation processing, and then uses a traffic flow estimation network model to extract, fuse, and predict features from the augmented images, thereby improving the accuracy and reliability of traffic flow prediction and providing more effective decision support for traffic management.

[0118] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the traffic flow prediction method described above.

[0119] The computer-readable storage medium provided in this application acquires aerial panoramic images and close-up images of the road surface of the target traffic area, performs image augmentation processing, and then uses a traffic flow estimation network model to extract, fuse, and predict features from the augmented images, thereby improving the accuracy and reliability of traffic flow prediction and providing more effective decision support for traffic management.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0121] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A traffic flow prediction method, characterized in that, Includes the following steps: Acquire aerial panoramic images and close-up images of the road surface of the target traffic area; The high-altitude panoramic image and the close-up road image are augmented to obtain an augmented panoramic image and an augmented close-up image; The traffic flow estimation network model is invoked, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; Based on the backbone network, image features are extracted from the panoramic augmented image and the close-up augmented image to obtain panoramic image features and close-up image features; Based on the neck network, image features of the panoramic image and the close-up image are fused to obtain fused image features; Based on the head network, the fused image features are used to perform model prediction to obtain traffic flow prediction data.

2. The traffic flow prediction method according to claim 1, characterized in that, The step of augmenting the high-altitude panoramic image and the close-up road image to obtain an augmented panoramic image and an augmented close-up image specifically includes the following steps: The aerial panoramic image and the close-up road image are rotated respectively to obtain the panoramic augmented image and the close-up augmented image; and / or Image shifting is performed on the high-altitude panoramic image and the close-up road image respectively to obtain the panoramic augmented image and the close-up augmented image; and / or The aerial panoramic image and the road surface close-up image are scaled up respectively to obtain the panoramic expanded image and the close-up expanded image; and / or Image enhancement is performed on the high-altitude panoramic image and the close-up road image respectively to obtain the panoramic augmented image and the close-up augmented image; and / or Noise is injected into the high-altitude panoramic image and the close-up road image respectively to obtain the panoramic augmented image and the close-up augmented image; and / or The high-altitude panoramic image and the close-up road image are subjected to contrast transformation to obtain the panoramic augmented image and the close-up augmented image, respectively.

3. The traffic flow prediction method according to claim 1, characterized in that, The step of extracting image features from the panoramic augmented image and the close-up augmented image based on the backbone network to obtain panoramic image features and close-up image features specifically includes the following steps: The vehicle detection model is used to detect vehicles in the near-view augmented image to obtain the number of vehicles, and the number of vehicles is vectorized into a vehicle number vector. Based on the backbone network, image features of the panoramic augmented image and the close-up augmented image are extracted respectively to obtain initial panoramic image features and initial close-up image features; The vehicle quantity vector is combined with the initial panoramic image features and the initial close-up image features respectively to obtain the panoramic image features and the close-up image features.

4. The traffic flow prediction method according to claim 3, characterized in that, The step of extracting image features from the panoramic augmented image and the close-up augmented image based on the backbone network to obtain initial panoramic image features and initial close-up image features specifically includes the following steps: The global color distribution of the panoramic augmented image and the close-up augmented image are obtained respectively to obtain the panoramic color features and the close-up color features; The texture details of the panoramic augmented image and the close-up augmented image are obtained respectively to obtain panoramic texture features and close-up texture features; The shape structure of the panoramic augmented image and the close-up augmented image are obtained respectively to obtain panoramic shape features and close-up shape features; The panoramic color features, panoramic texture features, and panoramic shape features are aligned, and the aligned panoramic color features, panoramic texture features, and panoramic shape features are fused to obtain the initial panoramic image features. The foreground color features, foreground texture features, and foreground shape features are aligned, and the aligned foreground color features, foreground texture features, and foreground shape features are fused to obtain the initial foreground image features.

5. The traffic flow prediction method according to claim 1, characterized in that, The step of fusing the panoramic image features and the close-up image features based on the neck network to obtain the fused image features specifically includes the following steps: The panoramic image features and the close-up image features are fused to obtain initial fused image features; Global average pooling is performed on the initial fused image features to obtain a global average pooling value, wherein the global average pooling value includes the number of channels; The size of the one-dimensional convolution kernel for the fused image features is calculated based on the number of channels. Perform one-dimensional convolution based on the one-dimensional convolution kernel size to obtain the channel weights for each channel; The initial fused image features are multiplied channel by channel according to the channel weights to obtain the fused image features.

6. The traffic flow prediction method according to claim 1, characterized in that, The head network includes a first fully connected layer, a second fully connected layer, and an output layer. The step of performing model prediction on the fused image features based on the head network to obtain traffic flow prediction data specifically includes the following steps: The panoramic image features, the close-up image features, and the fused image features are combined using a second feature to obtain the merged image features; The merged image features are input into the first fully connected layer for first feature dimensionality reduction to obtain preliminary nonlinear relationship features; The preliminary nonlinear relationship features are input into the second fully connected layer for second feature dimensionality reduction to obtain intermediate nonlinear relationship features. The intermediate nonlinear relationship features are input into the output layer for feature mapping to obtain the traffic flow prediction data.

7. The traffic flow prediction method according to claim 1, characterized in that, After the step of performing model prediction on the fused image features based on the head network to obtain traffic flow prediction data, the following step is also included: A preset traffic expert system is invoked, and the high-altitude panoramic image, the close-up image of the road surface, and the traffic flow prediction data are input into the traffic expert system for system prediction to obtain traffic flow inference data; Calculate the model loss function based on the traffic flow prediction data and the traffic flow inference data; The first parameter of the traffic flow estimation network model is optimized based on the model loss function.

8. A traffic flow prediction device, characterized in that, include: The image acquisition module is used to acquire aerial panoramic images of the target traffic area and close-up images of the road surface. An image augmentation module is used to augment the high-altitude panoramic image and the close-up road image to obtain an augmented panoramic image and an augmented close-up image. The model invocation module is used to invoke the traffic flow estimation network model, wherein the traffic flow estimation network model includes a backbone network, a neck network, and a head network; The image feature extraction module is used to extract image features from the panoramic augmented image and the close-up augmented image based on the backbone network to obtain panoramic image features and close-up image features; An image feature fusion module is used to fuse the panoramic image features and the close-up image features based on the neck network to obtain fused image features; The model prediction module is used to perform model prediction on the fused image features based on the head network to obtain traffic flow prediction data.

9. A computer device, comprising a memory and a processor, characterized in that, The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of the traffic flow prediction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the traffic flow prediction method as described in any one of claims 1 to 7.

Images