Multimodal edge cloud vehicle-road data collaborative driving decision system and method based on dual perspectives

Abstract

The present invention discloses a multimodal edge cloud vehicle-road data collaboratively driven decision-making system and method based on dual-perspective view, including a roadside safety warning system before a bus enters a station, which is used to issue safety warnings for dangers on the roadside of a platform before the bus enters the platform, and a passenger bus route prediction system is used to predict bus routes for passengers with dual-perspective visual feature detection and boarding and alighting behaviors. In the present invention, a platform edge computing module performs fast feature matching, uses a Haar feature classifier to retain facial feature areas, and then uses a fast hash matching algorithm based on PCA algorithm to reduce the dimension of data to achieve real-time user data encoding, automatically identify passengers' boarding and alighting behaviors, use a time series analysis algorithm to perform pattern recognition and trend prediction of historical data, dispatch vehicles to divert passengers, and use data redundancy elimination technology to optimize user data processing, optimize data transmission and storage, and ensure the effective integration of real-time and historical data.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a multimodal edge cloud vehicle-road data collaborative driving decision-making system and method based on dual perspectives. Background Art

[0002] In my country, even in large and medium-sized cities with relatively developed rail transit, the bus system is still an important part of urban passenger transportation and an important urban infrastructure. The quality of its planning and operation is crucial to the sustainable development of the city. Smart bus uses GPS / Beidou positioning technology, 3G / 4G communication technology, GIS geographic information system technology, combined with the operating characteristics of buses, to build a bus intelligent dispatching system to plan and dispatch routes and vehicles, realize intelligent scheduling, and improve the utilization rate of buses. At the same time, through the construction of a complete video surveillance system, it can realize the monitoring and management of buses, stations and stations. Smart bus is an inevitable model for the future development of public transportation, which is of great significance to alleviating the increasingly serious traffic congestion problem.

[0003] Existing buses use front-mounted surveillance cameras at bus stops for the driver's reference, which cannot effectively analyze and avoid platform hazards, thereby increasing the risk of buses entering the platform. They cannot predict the bus routes of passengers at the platform, resulting in insufficient or excessive bus allocation, affecting the normal order of passengers taking the bus. For this reason, we propose a multimodal edge cloud vehicle-road data collaboratively driven decision-making system and method based on dual perspectives. Summary of the invention

[0004] The purpose of the present invention is to provide a multimodal edge cloud vehicle-road data collaborative driving decision-making system and method based on dual perspectives to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solutions: a multimodal edge cloud vehicle-road data collaborative driving decision system based on dual perspectives, comprising:

[0006] A roadside safety warning system for buses before they enter a station. The roadside safety warning system for buses before they enter a station is used to issue safety warnings for dangers on the roadside of a platform before the bus enters the platform.

[0007] A passenger bus route prediction system is used to predict bus routes for passengers based on dual-view visual feature detection and boarding and alighting behaviors, so as to achieve reasonable allocation of buses.

[0008] Preferably, the roadside safety warning system before the bus enters the station includes:

[0009] A video acquisition module, which is installed at the front of the bus and at the bus stop and is used to collect image information;

[0010] An OBU module, wherein the OBU module is installed on a bus;

[0011] An RSU module, the RSU module is installed at the bus stop, the OBU module and the RSU module are used together to trigger the connection 100 meters before the bus enters the bus stop area, the roadside safety warning system before the bus enters the station is triggered to run, and image data is transmitted to the bus;

[0012] A platform edge computing module, which is installed at the bus station and is used to process and transmit image data;

[0013] A platform voice reminder module is installed at a bus stop and is used to issue safety warnings to passengers.

[0014] Preferably, the roadside safety warning system before the bus enters the station further includes a dangerous area demarcation module, and the dangerous area demarcation module is used to demarcate the dangerous area of ​​the bus station area according to historical data and real-time data of the bus station;

[0015] The dangerous area demarcation module demarcates the dangerous area after the OBU module and the RSU module trigger the connection. The roadside safety warning system before the bus enters the station receives the bus station parking position, speed, direction and running route data, and updates the bus station data in real time through the connection between the OBU module and the RSU module.

[0016] Preferably, the dangerous area demarcation module uses a dynamic graph convolutional network to analyze and demarcate dangerous areas of bus stops;

[0017] The model calculation formula of the dynamic graph convolutional network is shown in formulas (1) and (2):

[0018] (1);

[0019] (2);

[0020] Among them, A i (t) represents the output feature of node i at time point t, indicating the danger level;

[0021] Represents the attention weight from node i to node j at time point t;

[0022] W(t) and b(t) represent learnable weights and biases;

[0023] h j (t) represents the feature vector of node j at time point t;

[0024] Leaky ReLU is the activation function, expressed as Leaky ReLU(x)=max(0.01x,x). When the input value x is less than 0, it is multiplied by 0.01, and the rest remains unchanged;

[0025] i represents the i-th bus stop; represents the bus j entering the i-th bus stop;

[0026] exp represents the exponential function, defined as exp(x)=e x ;

[0027] a represents a parameter vector, which is used to learn the relationship between neighbor nodes;

[0028] a T Represents the transpose of the parameter vector a, used for dot product calculation;

[0029] h k (t) represents the feature vector of node k at time point t.

[0030] Preferably, after determining the location information of the dangerous area, the dangerous area demarcation module converts the geographical coordinates into image coordinates and displays them in the image captured by the video acquisition module at the bus stop. The specific process is as follows:

[0031] Receiving dangerous area location information: receiving dangerous area location information including geographic coordinates calculated by the dynamic graph convolutional network based on historical data and real-time data of bus stops;

[0032] Coordinate conversion: through the calibration of the video acquisition module, the perspective transformation coordinate conversion formula is used to convert the geographic coordinates into two-dimensional image coordinates;

[0033] Box drawing: Determine the position and size of the box based on the converted image coordinates, dynamically adjust the color and size of the box according to the danger level, and draw it on the real-time video stream or static image;

[0034] Real-time update and display: Dynamically adjust the position and size of the box on the image to reflect the latest danger zone location information, and display the updated image in real time on the bus interface.

[0035] Preferably, the roadside safety warning system before the bus enters the station also includes a pedestrian and vehicle collision warning module, and the pedestrian and vehicle collision warning module is used to automatically identify pedestrians and vehicles in the bus station;

[0036] After the OBU module and the RSU module are triggered to connect, the pedestrian and vehicle collision warning module predicts the potential collision risk between the bus and the pedestrian in the dangerous area according to the dangerous area position demarcated by the dangerous area demarcation module, and issues an early warning through the platform voice reminder module;

[0037] The pedestrian and vehicle collision warning module uses an improved sequence-to-sequence model for collision warning:

[0038] The calculation formula of the improved sequence-to-sequence model is shown in formulas (3), (4) and (5):

[0039] (3);

[0040] (4);

[0041] (5);

[0042] in, Represents the output of the control gate, regulating the flow of information;

[0043] represents the predicted collision risk index;

[0044] represents the linear transformation of the update gate;

[0045] represents the bias term of the update gate;

[0046] σ represents the sigmoid activation function, which maps the input value to between (0, 1);

[0047] represents the hidden state at the previous time point t-1, and the input at the current time point t The concatenation vector of including the position and speed of pedestrians and vehicles;

[0048] Represents an element-by-element multiplication operation;

[0049] tanh represents the hyperbolic tangent function, which maps the input value to between (-1, 1);

[0050] c t Represents the state of the memory unit at the current time point t;

[0051] Represents the activation value of the forget gate at the current time point t;

[0052] Represents the state of the memory unit at the previous time point t-1;

[0053] Represents the activation value of the input gate at the current time point t;

[0054] Represents the new information at the current time point t.

[0055] Preferably, the passenger bus route prediction system comprises:

[0056] A visual feature detection module installed at a bus stop and inside a bus;

[0057] The platform edge computing module is used to calculate visual features in time-sharing. The platform edge computing module performs fast feature matching when the bus enters the bus station. After the platform edge computing module uses the Haar feature classifier to retain the facial feature area, it uses the fast hash matching algorithm based on the PCA algorithm to reduce the dimension of the data and realizes real-time user data encoding, and encodes it into an 8-byte hash value T1 for automatic identification of passengers' boarding and getting on the bus: when the visual feature detection module at the bus station first detects the feature code, and the visual feature detection module in the bus detects and matches the same, it is the passenger's boarding state, and the matching state and data are cleared after the matching is completed; when the visual feature detection module in the bus first detects the feature code, the visual feature detection module at the bus station detects the same feature code and matches the same, it is the passenger's getting off state, and the matching state is cleared after the matching is completed; and the passenger's boarding location, alighting location, all stops, boarding time, and riding duration are recorded, and encoded into an 8-byte hash value T2;

[0058] The cloud system performs data analysis in its spare time. The cloud system uses a time series analysis algorithm to perform pattern recognition and trend prediction of historical data. The visual features of the passenger's mobile phone, the boarding location, and the boarding time are used as input, and the destination location and ride duration are used as output for training. The system predicts the passenger's destination location, all the stations, and ride duration. The visual features and the prediction results are encoded into a 16-byte hash value and sent to the platform edge computing module for storage, where the visual features are the first 8 bytes T1 and the last 8 bytes are the prediction results T3.

[0059] Decode the hash value T2 to obtain the predicted results of the passengers' getting off location, all the stops they will take and the duration of the ride. The bus stops in the predicted results are counted. When the predicted number of passengers at a certain stop exceeds twice the bus capacity, the operator is reminded to dispatch vehicles to divert the passengers.

[0060] Preferably, the passenger bus route prediction system further includes a data optimization processing module that cooperates with the platform edge computing module and the cloud system. The data optimization processing module uses data redundancy elimination technology to optimize the processing of user data. The specific process is as follows:

[0061] A unique hash value is generated for each data block, and duplicate data blocks are quickly identified and eliminated by comparing hash values. The platform edge computing module uses CDC technology to process user data collected in real time, segment data blocks and generate hash values. When the hash value of a data block matches the hash value of an existing historical data block, it is identified as a duplicate data block and is no longer uploaded to the cloud system. For unknown new data blocks, they are uploaded to the cloud system when the passenger bus route prediction system is idle for further in-depth analysis and pattern recognition, thereby optimizing data transmission and storage.

[0062] Preferably, after the OBU module and the RSU module trigger the connection, the dangerous area demarcation module uses a dynamic graph convolutional network model to calculate the dynamic data for analyzing and demarcating the dangerous area of ​​the bus station and transmits it to the platform edge computing module. The platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength status;

[0063] The dynamic data compression algorithm uses a combination of a convolutional neural network and a long short-term memory network to optimize the compression task. The convolutional neural network is used to extract data features, and the long short-term memory network is used to connect changes in strength conditions and adjust the dynamic data compression rate accordingly.

[0064] The dynamic data compression algorithm dynamically adjusts the dynamic data compression rate by calculating formula (6):

[0065] (6);

[0066] Among them, C t represents the data compression rate at time point t;

[0067] σ represents the sigmoid activation function, which maps the data compression rate to between (0,1);

[0068] W c represents the weight matrix;

[0069] h t-1 represents the hidden state of the long short-term memory network at time point t-1;

[0070] x t A feature vector representing the current network state extracted by the convolutional neural network;

[0071] bc represents the bias term;

[0072] The dynamic data compression algorithm learns the importance and urgency of different types of data, and can give priority to ensuring the transmission quality of key data when the OBU module and the RSU module trigger the connection, monitor the network bandwidth and traffic status in real time, and dynamically adjust the compression rate.

[0073] A multimodal edge cloud vehicle-road data collaborative driving decision method based on dual perspectives is applied to a multimodal edge cloud vehicle-road data collaborative driving decision system based on dual perspectives, comprising the following steps:

[0074] A: When the bus enters the bus station area 100 meters before, the OBU module on the bus and the RSU module at the bus station trigger the connection, the roadside safety warning system before the bus enters the station is triggered, the video acquisition module collects image information, and transmits the collected image information to the platform edge computing module;

[0075] B: Then the dangerous area delineation module updates the bus stop data in real time through the connection between the OBU module and the RSU module, and uses a dynamic graph convolutional network to analyze and delineate the dangerous area of ​​the bus stop. After determining the location information of the dangerous area, the geographic coordinates are converted into image coordinates and displayed in the image taken by the video acquisition module at the bus stop;

[0076] C: The platform voice reminder module issues safety warnings to passengers when the bus enters the bus station, and automatically identifies pedestrians and vehicles in the bus station through the pedestrian and vehicle collision warning module, and predicts the potential collision risk between buses and pedestrians in the dangerous area, and issues warnings through the platform voice reminder module;

[0077] D: The platform edge computing module performs fast feature matching when the bus enters the bus station. After using the Haar feature classifier to retain the facial feature area, the platform edge computing module uses the fast hash matching algorithm to achieve real-time user data encoding after reducing the dimension of the data based on the PCA algorithm, and automatically identifies the passengers' boarding and alighting behaviors, and stores their data in the cloud system;

[0078] E: The cloud system uses a time series analysis algorithm to perform pattern recognition and trend prediction on historical data. It decodes the hash value T2 to obtain the predicted results of the passengers' alighting location, all the stops they will take, and the duration of their ride at the bus stop, and dispatches vehicles to divert passengers. At the same time, the platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength, giving priority to ensuring the transmission quality of key data.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] 1. In the present invention, the platform edge computing module performs fast feature matching when the bus enters the bus station. After the platform edge computing module uses the Haar feature classifier to retain the facial feature area, it uses the fast hash matching algorithm to realize real-time user data encoding based on the PCA algorithm after reducing the dimension of the data, and automatically identifies the passengers' boarding and alighting behaviors. The time series analysis algorithm is used to perform pattern recognition and trend prediction of historical data, and the hash value T2 is decoded to obtain the predicted results of the passenger's alighting location, all the stops and the riding time at the bus station, and the vehicle is dispatched to divert the passengers. At the same time, the platform edge computing module adopts a dynamic data compression algorithm based on a neural network, adjusts the data compression rate in real time according to the current connection strength status, gives priority to ensuring the transmission quality of key data, and adopts data redundancy elimination technology to optimize the processing of user data, facilitate data analysis and pattern recognition, thereby optimizing data transmission and storage, accelerating data processing speed, and ensuring the effective integration of real-time and historical data;

[0081] 2. The present invention is provided with a dangerous area demarcation module, which uses a dynamic graph convolutional network model to demarcate the dangerous area of ​​the bus stop area according to historical data and real-time data of the bus stop, and updates the bus stop data in real time through the connection between the OBU module and the RSU module, and feeds back the position information of the dangerous demarcated area in real time to ensure the safety of the bus when entering the bus stop, and updates the position information of the dangerous demarcated area in real time to ensure the standardization of the bus under dual perspectives during the entry process;

[0082] 3. The present invention is provided with a pedestrian and vehicle collision warning module. Based on the dual-view state, the pedestrian and vehicle collision warning module adopts an improved sequence-to-sequence model to automatically identify pedestrians and vehicles in the bus stop and issue collision warnings to ensure safety within the dangerous designated area. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a schematic diagram of the system flow of the present invention;

[0084] Figure 2 It is a schematic diagram of the dangerous area delineation and pedestrian and vehicle collision warning process of the present invention;

[0085] Figure 3 It is a schematic diagram of the passenger data feature detection process of the present invention. DETAILED DESCRIPTION

[0086] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0087] See also Figure 1-Figure 3 The present invention provides a multi-modal edge cloud vehicle-road data collaborative driving decision system based on dual perspectives, including:

[0088] The roadside safety warning system before the bus enters the station is used to issue safety warnings for dangers on the roadside of the platform before the bus enters the platform;

[0089] The roadside safety warning system before the bus enters the station includes:

[0090] Video acquisition module, which is installed at the front of the bus and at the bus station to collect image information;

[0091] OBU module, the OBU module is installed on the bus;

[0092] RSU module: The RSU module is installed at the bus station. The OBU module and the RSU module are used together to trigger the connection 100 meters before the bus enters the bus station area. The roadside safety warning system is triggered before the bus enters the station and transmits image data to the bus.

[0093] The platform edge computing module is installed at the bus station and is used to process and transmit image data;

[0094] Platform voice reminder module, which is installed at the bus station and is used to issue safety warnings to passengers;

[0095] The roadside safety warning system before the bus enters the station also includes a dangerous area demarcation module, which is used to demarcate the dangerous area of ​​the bus station area based on historical data and real-time data of the bus station;

[0096] The dangerous area demarcation module demarcates the dangerous area after the OBU module and the RSU module trigger the connection. The roadside safety warning system before the bus enters the station receives the bus's platform parking position, speed, direction and running route data, and updates the bus platform data in real time through the connection between the OBU module and the RSU module;

[0097] The dangerous area delineation module uses a dynamic graph convolutional network to analyze and delineate dangerous areas at bus stops;

[0098] The model calculation formula of the dynamic graph convolutional network is shown in formulas (1) and (2):

[0099] (1);

[0100] (2);

[0101] Among them, A i (t) represents the output feature of node i at time point t, indicating the danger level;

[0102] Represents the attention weight from node i to node j at time point t;

[0103] W(t) and b(t) represent learnable weights and biases;

[0104] h j (t) represents the feature vector of node j at time point t;

[0105] Leaky ReLU is the activation function, expressed as Leaky ReLU(x)=max(0.01x,x). When the input value x is less than 0, it is multiplied by 0.01, and the rest remains unchanged;

[0106] i represents the i-th bus stop; represents the bus j entering the i-th bus stop;

[0107] exp represents the exponential function, defined as exp(x)=e x ;

[0108] a represents a parameter vector, which is used to learn the relationship between neighbor nodes;

[0109] a T Represents the transpose of the parameter vector a, used for dot product calculation;

[0110] h k (t) represents the feature vector of node k at time point t;

[0111] After the dangerous area demarcation module determines the location information of the dangerous area, it converts the geographic coordinates into image coordinates and displays them in the image taken by the video acquisition module at the bus stop. The specific process is as follows:

[0112] Receiving dangerous area location information: receiving dangerous area location information including geographic coordinates calculated by the dynamic graph convolutional network based on historical data and real-time data of bus stops;

[0113] Coordinate conversion: through the calibration of the video acquisition module, the perspective transformation coordinate conversion formula is used to convert the geographic coordinates into two-dimensional image coordinates;

[0114] Box drawing: Determine the position and size of the box based on the converted image coordinates, dynamically adjust the color and size of the box according to the danger level, and draw it on the real-time video stream or static image;

[0115] Real-time update and display: Dynamically adjust the position and size of the box on the image to reflect the latest dangerous area location information, and display the updated image in real time on the bus interface;

[0116] The present invention is provided with a dangerous area demarcation module, which uses a dynamic graph convolutional network model to demarcate the dangerous area of ​​the bus stop area according to historical data and real-time data of the bus stop, and updates the bus stop data in real time through the connection of the OBU module and the RSU module, and feeds back the position information of the dangerous area in real time to ensure the safety of the bus when entering the bus stop, and updates the position information of the dangerous area in real time to ensure the standardization of the bus under dual perspectives during the entry process;

[0117] The roadside safety warning system before the bus enters the station also includes a pedestrian and vehicle collision warning module, which is used to automatically identify pedestrians and vehicles in the bus station;

[0118] After the OBU module and the RSU module are triggered to connect, the pedestrian and vehicle collision warning module predicts the potential collision risk between the bus and the pedestrian according to the trajectory in the dangerous area defined by the dangerous area demarcation module, and issues an early warning through the platform voice reminder module;

[0119] The pedestrian and vehicle collision warning module uses an improved sequence-to-sequence model for collision warning:

[0120] The calculation formula of the improved sequence-to-sequence model is shown in formulas (3), (4) and (5):

[0121] (3);

[0122] (4);

[0123] (5);

[0124] in, Represents the output of the control gate, regulating the flow of information;

[0125] represents the predicted collision risk index;

[0126] represents the linear transformation of the update gate;

[0127] represents the bias term of the update gate;

[0128] σ represents the sigmoid activation function, which maps the input value to between (0, 1);

[0129] represents the hidden state at the previous time point t-1, and the input at the current time point t The concatenation vector of including the position and speed of pedestrians and vehicles;

[0130] Represents an element-by-element multiplication operation;

[0131] tanh represents the hyperbolic tangent function, which maps the input value to between (-1, 1);

[0132] c t Represents the state of the memory unit at the current time point t;

[0133] Represents the activation value of the forget gate at the current time point t;

[0134] Represents the state of the memory unit at the previous time point t-1;

[0135] Represents the activation value of the input gate at the current time point t;

[0136] Represents the new information at the current time point t;

[0137] The present invention is provided with a pedestrian and vehicle collision warning module. Based on the dual-view state, the pedestrian and vehicle collision warning module uses an improved sequence-to-sequence model to automatically identify pedestrians and vehicles in the bus station and issue collision warnings to ensure safety within the dangerous demarcated area.

[0138] Passenger bus route prediction system, which is used to predict bus routes for passengers based on dual-view visual feature detection and boarding and alighting behaviors, to achieve reasonable allocation of buses;

[0139] The passenger bus route prediction system includes:

[0140] Visual feature detection module, the visual feature detection module is installed at the bus station and in the bus;

[0141] The platform edge computing module is used to calculate visual features in time-sharing. The platform edge computing module performs fast feature matching when the bus enters the bus station. After the platform edge computing module uses the Haar feature classifier to retain the facial feature area, it uses the fast hash matching algorithm based on the PCA algorithm to reduce the dimension of the data and realizes real-time user data encoding, and encodes it into an 8-byte hash value T1 for automatic identification of passengers' boarding and getting on the bus: when the visual feature detection module at the bus station first detects the feature code, and the visual feature detection module in the bus detects and matches the same, it is the passenger's boarding state, and the matching state and data are cleared after the matching is completed; when the visual feature detection module in the bus first detects the feature code, the visual feature detection module at the bus station detects the same feature code and matches the same, it is the passenger's getting off state, and the matching state is cleared after the matching is completed; and the passenger's boarding location, alighting location, all stops, boarding time, and riding duration are recorded, and encoded into an 8-byte hash value T2;

[0142] The cloud system performs data analysis in its spare time. The cloud system uses a time series analysis algorithm to perform pattern recognition and trend prediction of historical data. The visual features of the passenger's mobile phone, the boarding location, and the boarding time are used as input, and the destination location and ride duration are used as output for training. The system predicts the passenger's destination location, all the stations, and ride duration. The visual features and the prediction results are encoded into a 16-byte hash value and sent to the platform edge computing module for storage, where the visual features are the first 8 bytes T1 and the last 8 bytes are the prediction results T3.

[0143] Decode the hash value T2 to get the predicted results of the passenger's getting off location, all the stops and the ride duration at the bus stop. Count the bus stops in the predicted results. When the predicted number of passengers at a certain stop exceeds 2 times the bus capacity, remind the operator to dispatch vehicles to divert passengers.

[0144] The passenger bus route prediction system also includes a data optimization processing module that collaborates with the platform edge computing module and the cloud system. The data optimization processing module uses data redundancy elimination technology to optimize the processing of user data. The specific process is as follows:

[0145] Generate a unique hash value for each data block, and quickly identify and eliminate duplicate data blocks by comparing hash values. The platform edge computing module uses CDC technology to process real-time collected user data, segment data blocks and generate hash values. When the hash value of a data block matches the hash value of an existing historical data block, it is identified as a duplicate data block and is no longer uploaded to the cloud system. For unknown new data blocks, they are uploaded to the cloud system during idle time of the passenger bus route prediction system for further in-depth analysis and pattern recognition, thereby optimizing data transmission and storage.

[0146] After the OBU module and the RSU module trigger the connection, the dangerous area demarcation module uses a dynamic graph convolutional network model to calculate the dynamic data for analyzing and demarcating dangerous areas at bus stops and transmits it to the platform edge computing module. The platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength.

[0147] The dynamic data compression algorithm uses a combination of convolutional neural networks and long short-term memory networks to optimize the compression task. The convolutional neural network is used to extract data features, and the long short-term memory network is used to connect changes in strength conditions and adjust the dynamic data compression rate accordingly.

[0148] The dynamic data compression algorithm dynamically adjusts the dynamic data compression rate by calculating formula (6):

[0149] (6);

[0150] Among them, C t represents the data compression rate at time point t;

[0151] σ represents the sigmoid activation function, which maps the data compression rate to between (0,1);

[0152] W c represents the weight matrix;

[0153] h t-1 represents the hidden state of the long short-term memory network at time point t-1;

[0154] x t A feature vector representing the current network state extracted by the convolutional neural network;

[0155] b c represents the bias term;

[0156] The dynamic data compression algorithm learns the importance and urgency of different types of data, and can prioritize the transmission quality of key data when the OBU module and RSU module trigger the connection. It monitors the network bandwidth and traffic status in real time and dynamically adjusts the compression rate.

[0157] In the present invention, the platform edge computing module performs fast feature matching when the bus enters the bus stop. After the platform edge computing module uses the Haar feature classifier to retain the facial feature area, it uses the fast hash matching algorithm to realize real-time user data encoding after reducing the dimension of the data based on the PCA algorithm, and automatically identifies the passengers' boarding and getting on the bus. The time series analysis algorithm is used to perform pattern recognition and trend prediction of historical data, and the hash value T2 is decoded to obtain the predicted results of the passengers' getting off the bus stop, all the stops they have taken, and the length of time they have been riding. The vehicles are dispatched to divert the passengers. At the same time, the platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength status, give priority to ensuring the transmission quality of key data, and use data redundancy elimination technology to optimize the processing of user data, facilitate data analysis and pattern recognition, thereby optimizing data transmission and storage, accelerating data processing speed, and ensuring the effective integration of real-time and historical data.

[0158] The multimodal edge cloud vehicle-road data collaborative driving decision method based on dual perspectives provided by the present invention is applied to a multimodal edge cloud vehicle-road data collaborative driving decision system based on dual perspectives, and includes the following steps:

[0159] A: When the bus enters the bus station area 100 meters before, the OBU module on the bus and the RSU module at the bus station trigger the connection, the roadside safety warning system before the bus enters the station is triggered, the video acquisition module collects image information, and transmits the collected image information to the platform edge computing module;

[0160] B: Then the dangerous area delineation module updates the bus stop data in real time through the connection between the OBU module and the RSU module, and uses a dynamic graph convolutional network to analyze and delineate the dangerous area of ​​the bus stop. After determining the location information of the dangerous area, the geographic coordinates are converted into image coordinates and displayed in the image taken by the video acquisition module at the bus stop;

[0161] C: The platform voice reminder module issues safety warnings to passengers when the bus enters the bus station, and automatically identifies pedestrians and vehicles in the bus station through the pedestrian and vehicle collision warning module, and predicts the potential collision risk between buses and pedestrians in the dangerous area, and issues warnings through the platform voice reminder module;

[0162] D: The platform edge computing module performs fast feature matching when the bus enters the bus station. After using the Haar feature classifier to retain the facial feature area, the platform edge computing module uses the fast hash matching algorithm to achieve real-time user data encoding after reducing the dimension of the data based on the PCA algorithm, and automatically identifies the passengers' boarding and alighting behaviors, and stores their data in the cloud system;

[0163] E: The cloud system uses a time series analysis algorithm to perform pattern recognition and trend prediction on historical data. It decodes the hash value T2 to obtain the predicted results of the passengers' alighting location, all the stops they will take, and the duration of their ride at the bus stop, and dispatches vehicles to divert passengers. At the same time, the platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength, giving priority to ensuring the transmission quality of key data.

[0164] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-modal edge cloud vehicle-road data collaborative driving decision system based on dual perspectives, characterized in that: include: A roadside safety warning system for buses before they enter a station. The roadside safety warning system for buses before they enter a station is used to issue safety warnings for dangers on the roadside of a platform before the bus enters the platform. A passenger bus route prediction system, which is used to predict bus routes for passengers based on dual-view visual feature detection and boarding and alighting behaviors, so as to achieve reasonable allocation of buses; The roadside safety warning system before the bus enters the station includes: A video acquisition module, which is installed at the front of the bus and at the bus stop and is used to collect image information; An OBU module, wherein the OBU module is installed on a bus; An RSU module, the RSU module is installed at the bus stop, the OBU module and the RSU module are used together to trigger the connection 100 meters before the bus enters the bus stop area, the roadside safety warning system before the bus enters the station is triggered to run, and image data is transmitted to the bus; A platform edge computing module, which is installed at the bus station and is used to process and transmit image data; A platform voice reminder module, which is installed at the bus station and is used to issue safety warnings to passengers; The roadside safety warning system for buses entering the station also includes a dangerous area demarcation module, which is used to demarcate the dangerous area of ​​the bus station area based on historical data and real-time data of the bus station; The dangerous area demarcation module demarcates the dangerous area after the OBU module and the RSU module trigger the connection. The roadside safety warning system before the bus enters the station receives the bus's platform parking position, speed, direction and running route data, and updates the bus station data in real time through the connection between the OBU module and the RSU module; The dangerous area delineation module uses a dynamic graph convolutional network to analyze and delineate dangerous areas at bus stops; The model calculation formula of the dynamic graph convolutional network is shown in formulas (1) and (2): （1）； （2）； Among them, A i (t) represents the output feature of node i at time point t, indicating the danger level; Represents the attention weight from node i to node j at time point t; W(t) and b(t) represent learnable weights and biases; h j (t) represents the feature vector of node j at time point t; Leaky ReLU is the activation function, expressed as Leaky ReLU(x)=max(0.01x,x). When the input value x is less than 0, it is multiplied by 0.01, and the rest remains unchanged; i represents the i-th bus stop; represents the bus j entering the i-th bus stop; exp represents the exponential function, defined as exp(x)=e x ; a represents a parameter vector, which is used to learn the relationship between neighbor nodes; a T Represents the transpose of the parameter vector a, used for dot product calculation; h k (t) represents the feature vector of node k at time point t.

2. The multimodal edge cloud vehicle-road data collaborative driving decision system based on dual-viewing angles according to claim 1 is characterized in that: After the dangerous area demarcation module determines the location information of the dangerous area, the geographical coordinates are converted into image coordinates and displayed in the image taken by the video acquisition module at the bus station. The specific process is as follows: Receiving dangerous area location information: receiving dangerous area location information including geographic coordinates calculated by the dynamic graph convolutional network based on historical data and real-time data of bus stops; Coordinate conversion: through the calibration of the video acquisition module, the perspective transformation coordinate conversion formula is used to convert the geographic coordinates into two-dimensional image coordinates; Box drawing: Determine the position and size of the box based on the converted image coordinates, dynamically adjust the color and size of the box according to the danger level, and draw it on the real-time video stream or static image; Real-time update and display: Dynamically adjust the position and size of the box on the image to reflect the latest danger zone location information, and display the updated image in real time on the bus interface.

3. The multimodal edge cloud vehicle-road data collaborative driving decision system based on dual-viewing angles according to claim 2 is characterized in that: The roadside safety warning system for buses entering the bus station also includes a pedestrian and vehicle collision warning module, which is used to automatically identify pedestrians and vehicles in the bus station; After the OBU module and the RSU module are triggered to connect, the pedestrian and vehicle collision warning module predicts the potential collision risk between the bus and the pedestrian in the dangerous area according to the dangerous area position demarcated by the dangerous area demarcation module, and issues an early warning through the platform voice reminder module; The pedestrian and vehicle collision warning module uses an improved sequence-to-sequence model for collision warning: The calculation formula of the improved sequence-to-sequence model is shown in formulas (3), (4) and (5): （3）； （4）； （5）； in, Represents the output of the control gate, regulating the flow of information; represents the predicted collision risk index; represents the linear transformation of the update gate; represents the bias term of the update gate; σ represents the sigmoid activation function, which maps the input value to between (0, 1); represents the hidden state at the previous time point t-1, and the input at the current time point t The concatenation vector of including the position and speed of pedestrians and vehicles; Represents an element-by-element multiplication operation; tanh represents the hyperbolic tangent function, which maps the input value to between (-1, 1); c t Represents the state of the memory unit at the current time point t; Represents the activation value of the forget gate at the current time point t; Represents the state of the memory unit at the previous time point t-1; Represents the activation value of the input gate at the current time point t; Represents the new information at the current time point t.

4. The multimodal edge cloud vehicle-road data collaborative driving decision system based on dual-viewing angles according to claim 3 is characterized in that: The passenger bus route prediction system comprises: A visual feature detection module installed at a bus stop and inside a bus; The platform edge computing module is used to calculate visual features in time-sharing. The platform edge computing module performs fast feature matching when the bus enters the bus station. After the platform edge computing module uses the Haar feature classifier to retain the facial feature area, it uses the fast hash matching algorithm based on the PCA algorithm to reduce the dimension of the data and realizes real-time user data encoding, and encodes it into an 8-byte hash value T1 for automatic identification of passengers' boarding and getting on the bus: when the visual feature detection module at the bus station first detects the feature code, and the visual feature detection module in the bus detects and matches the same, it is the passenger's boarding state, and the matching state and data are cleared after the matching is completed; when the visual feature detection module in the bus first detects the feature code, the visual feature detection module at the bus station detects the same feature code and matches the same, it is the passenger's getting off state, and the matching state is cleared after the matching is completed; and the passenger's boarding location, alighting location, all stops, boarding time, and riding duration are recorded, and encoded into an 8-byte hash value T2; The cloud system performs data analysis in its spare time. The cloud system uses a time series analysis algorithm to perform pattern recognition and trend prediction of historical data. The visual features of the passenger's mobile phone, the boarding location, and the boarding time are used as input, and the destination location and ride duration are used as output for training. The system predicts the passenger's destination location, all the stations, and ride duration. The visual features and the prediction results are encoded into a 16-byte hash value and sent to the platform edge computing module for storage, where the visual features are the first 8 bytes T1 and the last 8 bytes are the prediction results T3. Decode the hash value T2 to obtain the predicted results of the passengers' getting off location, all the stops they will take and the duration of the ride. The bus stops in the predicted results are counted. When the predicted number of passengers at a certain stop exceeds twice the bus capacity, the operator is reminded to dispatch vehicles to divert the passengers.

5. The multimodal edge cloud vehicle-road data collaborative driving decision system based on dual-viewing angles according to claim 4 is characterized in that: The passenger bus route prediction system also includes a data optimization processing module that cooperates with the platform edge computing module and the cloud system. The data optimization processing module uses data redundancy elimination technology to optimize the processing of user data. The specific process is as follows: A unique hash value is generated for each data block, and duplicate data blocks are quickly identified and eliminated by comparing hash values. The platform edge computing module uses CDC technology to process user data collected in real time, segment data blocks and generate hash values. When the hash value of a data block matches the hash value of an existing historical data block, it is identified as a duplicate data block and is no longer uploaded to the cloud system. For unknown new data blocks, they are uploaded to the cloud system when the passenger bus route prediction system is idle for further in-depth analysis and pattern recognition, thereby optimizing data transmission and storage.

6. The multimodal edge cloud vehicle-road data collaborative driving decision system based on dual-viewing angles according to claim 5 is characterized in that: After the OBU module and the RSU module trigger the connection, the dangerous area demarcation module uses a dynamic graph convolutional network model to calculate the dynamic data for analyzing and demarcating the dangerous area of ​​the bus station and transmits it to the platform edge computing module. The platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength status; The dynamic data compression algorithm uses a combination of a convolutional neural network and a long short-term memory network to optimize the compression task. The convolutional neural network is used to extract data features, and the long short-term memory network is used to connect changes in strength conditions and adjust the dynamic data compression rate accordingly. The dynamic data compression algorithm dynamically adjusts the dynamic data compression rate by calculating formula (6): （6）； Among them, C t represents the data compression rate at time point t; σ represents the sigmoid activation function, which maps the data compression rate to between (0,1); W c represents the weight matrix; h t-1 represents the hidden state of the long short-term memory network at time point t-1; x t A feature vector representing the current network state extracted by the convolutional neural network; b c represents the bias term; The dynamic data compression algorithm learns the importance and urgency of different types of data, and can give priority to ensuring the transmission quality of key data when the OBU module and the RSU module trigger the connection, monitor the network bandwidth and traffic status in real time, and dynamically adjust the compression rate.

7. A method for collaboratively driving decision-making based on multimodal edge cloud vehicle-road data under dual perspectives, applied to the collaboratively driving decision-making system based on multimodal edge cloud vehicle-road data under dual perspectives as described in claim 6, characterized in that: The steps include: A: When the bus enters the bus station area 100 meters before, the OBU module on the bus and the RSU module at the bus station trigger the connection, the roadside safety warning system before the bus enters the station is triggered, the video acquisition module collects image information, and transmits the collected image information to the platform edge computing module; B: Then the dangerous area delineation module updates the bus stop data in real time through the connection between the OBU module and the RSU module, and uses a dynamic graph convolutional network to analyze and delineate the dangerous area of ​​the bus stop. After determining the location information of the dangerous area, the geographic coordinates are converted into image coordinates and displayed in the image taken by the video acquisition module at the bus stop; C: The platform voice reminder module issues safety warnings to passengers when the bus enters the bus station, and automatically identifies pedestrians and vehicles in the bus station through the pedestrian and vehicle collision warning module, and predicts the potential collision risk between buses and pedestrians in the dangerous area, and issues warnings through the platform voice reminder module; D: The platform edge computing module performs fast feature matching when the bus enters the bus station. After using the Haar feature classifier to retain the facial feature area, the platform edge computing module uses the fast hash matching algorithm to achieve real-time user data encoding after reducing the dimension of the data based on the PCA algorithm, and automatically identifies the passengers' boarding and alighting behaviors, and stores their data in the cloud system; E: The cloud system uses a time series analysis algorithm to perform pattern recognition and trend prediction on historical data. It decodes the hash value T2 to obtain the predicted results of the passengers' alighting location, all the stops they will take, and the duration of their ride at the bus stop, and dispatches vehicles to divert passengers. At the same time, the platform edge computing module uses a dynamic data compression algorithm based on a neural network to adjust the data compression rate in real time according to the current connection strength, giving priority to ensuring the transmission quality of key data.

Images