Pedestrian crossing traffic intention prediction method and warning system in bird's-eye view

Abstract

The invention discloses a pedestrian crossing traffic intention prediction method and a warning system at an aerial view angle, and belongs to the technical field of traffic management, and the prediction method comprises the following specific steps: S1, collecting and processing each group of data, and carrying out the cross-camera matching; s2, optimizing the matched comprehensive information graph and extracting feature data; s3, fusing the feature information of each group and predicting a pedestrian crossing traffic intention; the method is helpful to capture the uncertainty in pedestrian behaviors, can dynamically adapt to the change trend of the pedestrian behaviors, provides more elastic and intelligent intention prediction capability, enhances the adaptability in diversified scenes, reduces the probability of misclassification, can reduce the probability of mismatching, is helpful to reduce information loss and error accumulation, and improves the prediction efficiency. The robustness of cross-view information fusion is improved, the input data quality of overall pedestrian intention prediction is optimized, and a more stable pedestrian crossing intention prediction basis is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic management, and in particular to a method and a warning system for predicting the traffic intention of pedestrians crossing the street from a bird's-eye view. Background Art

[0002] With the rapid development of intelligent transportation systems and autonomous driving technologies, accurately predicting pedestrians' intention to cross the street has become an important issue to improve road safety and traffic efficiency. Pedestrian crossing behavior is highly uncertain and is affected by many factors, such as road geometry, traffic conditions, pedestrians' personal characteristics and behavior patterns. In urban traffic environments, especially in pedestrian crossing areas, drivers and autonomous driving systems need to perceive and predict whether pedestrians have the intention to cross the street in a timely manner, so as to make effective countermeasures. Traditional perception technologies based on vision and radar can capture the position and motion state of pedestrians, but the prediction accuracy of pedestrians' intention to cross the street is still limited. At the same time, existing predictions usually calculate the probability of future behavior through historical data. However, the movement of people in traffic is multimodal. Pedestrians walk while looking at the surrounding situation, and the occlusion between vehicles affects the line of sight. Therefore, there is a detour between pedestrians' intention to cross the street and not to cross the street. At this time, it is necessary to consider the intention to cross the street based on the surrounding traffic conditions, that is, the speed of surrounding vehicles, the speed of target vehicles, the distance between vehicles, the width of the road, the gender of pedestrians, and the age stage. That is, most of the various image recognition algorithms consider the pedestrian crossing problem from the perspective of image processing, without fully considering the traffic rules and the traffic characteristics of pedestrians and drivers as traffic participants.

[0003] The existing pedestrian crossing intention prediction method and warning system from a bird's-eye view are not conducive to capturing the uncertainty in pedestrian behavior, cannot dynamically adapt to the changing trend of pedestrian behavior, and have a high probability of mismatching; in addition, the existing pedestrian crossing intention prediction method and warning system from a bird's-eye view have low robustness of cross-perspective information fusion, and the input data quality of the overall pedestrian intention prediction is low, which cannot provide a stable basis for pedestrian crossing intention prediction; for this reason, we propose a pedestrian crossing intention prediction method and warning system from a bird's-eye view. Summary of the invention

[0004] The purpose of the present invention is to solve the defects in the prior art and to propose a method and a warning system for predicting pedestrian crossing traffic intention from a bird's eye view.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The method for predicting pedestrian crossing traffic intention from a bird's-eye view is as follows:

[0007] S1, collect and process each set of data and perform cross-camera matching;

[0008] S2, optimizing the comprehensive information graph after matching and extracting feature data;

[0009] S3, integrating each group of feature information and predicting pedestrians’ traffic intentions when crossing the street;

[0010] S4. Optimize the prediction process and store the collected and predicted data through blockchain;

[0011] S5. Generate real-time warning information based on the prediction results and collect various feedback information;

[0012] S6. Adjust and optimize the system based on feedback information and deploy it for actual application.

[0013] As a further solution of the present invention, the specific steps of cross-camera matching in S1 are as follows:

[0014] Q1.1: Obtain the intrinsic parameters of each camera through the checkerboard calibration method, then collect the relative positions and orientations of the cameras through the calibration board, and construct a set of image pyramids with the original images collected by each camera, and select a pixel in each layer of image data;

[0015] Q1.2: Construct a circle with the selected pixel as the center according to the preset radius, and calculate the difference between each pixel on the circumference and the selected pixel. If there are multiple groups of pixels whose values ​​are higher or lower than the preset threshold than the selected pixel, the selected pixel is marked as a corner point, and non-maximum suppression is performed on each group of marked corner points.

[0016] Q1.3: Use the grayscale centroid method to calculate the direction of the feature point, and generate the BRIEF descriptor by performing binary comparison on the pixel pairs in the neighborhood of the feature point. The pixel points whose pixel values ​​in the neighborhood of each feature point are smaller than the feature point are marked as 1, otherwise they are marked as 0. Use the FLANN matcher to find the feature descriptor closest to the corresponding descriptor in another image through approximate nearest neighbor search to obtain multiple sets of matching point pairs and construct a set of matching point pairs.

[0017] Q1.4: Randomly select a group of samples from the set of matching point pairs, and calculate the parameters of the geometric transformation model based on the selected random samples. Output all the remaining matching points to the geometric transformation model, and calculate the error between the model's predicted value and the actual value of each matching point. If there is a pair of matching points whose error is less than the preset threshold, it will be considered a successful match, otherwise it will be considered a failed match. Repeat multiple iterations to remove misaligned matching points.

[0018] Q1.5: Spatially transform the coordinates of feature points from different perspectives to align them in a unified coordinate system, and integrate the pedestrian and vehicle information captured by different cameras into the same perspective to form a comprehensive information map.

[0019] As a further solution of the present invention, the specific steps of fusing each group of feature information in S3 are as follows:

[0020] Q2.1: Align the image data and radar point cloud data in the comprehensive information map to the same coordinate system through spatial calibration, extract the feature information of the image data and radar point cloud data through the ResNet network and the point cloud network respectively, then serialize each group of feature information and add position coding to each group of feature information;

[0021] Q2.2: Input the processed feature information into the Transformer model, divide the input features into multiple subspaces through the multi-head self-attention mechanism, and independently calculate the attention on each subspace through the self-attention mechanism. Then, through the Transformer model fusion layer, the image features and point cloud features processed by the multi-head self-attention are weighted fused;

[0022] Q2.3: The fused features are aligned with the global features generated by the encoder through the cross-attention mechanism of the Transformer decoder. The feature data at each position is processed nonlinearly through the fully connected layer in the feedforward neural network, and the input features are added to the output processed by the self-attention and feedforward networks using residual connections.

[0023] Q2.4: The summed results are standardized, and the geometric characteristics of the road are identified based on the processed data. The location, gender, and age characteristics of the surrounding pedestrians, as well as the speed and location of the vehicle are marked.

[0024] As a further solution of the present invention, the specific steps of predicting the pedestrian crossing traffic intention in S3 are as follows:

[0025] S3.1: The classifier receives each set of feature data output by the Transformer model, and linearly transforms the feature vector and the weight vector of each category of pedestrian crossing intention to map the input features to each category space and obtain the unnormalized score of each category;

[0026] S3.2: Pass The unnormalized scores of each category are converted into probability distribution, and the category with the highest probability is selected as the final prediction result according to the probability distribution result, which is used as the output of the pedestrian's crossing intention, where P(y=c|h) represents the probability that the pedestrian belongs to category c, h represents the feature data output from the Transformer model, c represents a specific category label, c′ represents the index of all categories, and z c represents the unnormalized score corresponding to category c; z c′ represents the unnormalized scores corresponding to all categories c′, represents the index value of the current category c, represents the sum of index values ​​for all categories;

[0027] S3.3: The loss value between the output result and the actual value is calculated through the cross entropy loss function. Then, starting from the classifier output layer, the calculated loss value is back-propagated to update the classifier parameters. The updated parameter combination is used as the root node, and it is randomly perturbed to generate multiple groups of different parameter combinations, and it is used as a child node to build an exploration tree.

[0028] S3.4: Starting from the root node, the child node with the highest UCB1 value is selected as the next node through the UCB1 strategy, and the node selection is continued until a leaf node is reached. If the leaf node is not in the final state, the parameter combination represented by the child node is expanded, and the expansion result is added to the tree as a new child node;

[0029] S3.5: Perform simulation on the expanded nodes through random strategies and calculate the loss value of each child node until the preset simulation time is reached and then stop. The simulation results are then traced back to all nodes on the path, and the information of each node is updated. The selection, expansion, simulation and backtracking steps are fed back and executed until the change in the loss value reaches the preset threshold and then stops.

[0030] S3.6: Based on the loss value results of each child node in the final exploration tree, select the parameter combination with the smallest loss value as the optimal parameter combination output of the classifier, and replace it with the original parameter combination, and train and verify the classifier at the same time.

[0031] The pedestrian crossing traffic intention warning system from a bird's eye view includes a collection and processing module, a data matching module, an optimization and adjustment module, an extraction and fusion module, an intention prediction module, a warning generation module, a real-time monitoring module, a storage management module, and a feedback optimization module;

[0032] The acquisition and processing module is used to obtain video images, environmental point cloud data, and location information of people and vehicles from cameras, laser radars, and GPS devices;

[0033] The data matching module is used to match pedestrian and vehicle information captured by different cameras to generate a comprehensive information map containing multiple camera perspectives;

[0034] The optimization and adjustment module is used to optimize cross-camera matching to generate a comprehensive information graph;

[0035] The extraction and fusion module is used to extract key features from the comprehensive information graph and preprocessed data through the Transformer model for fusion;

[0036] The intention prediction module is used to predict whether the pedestrian intends to cross the street and the possibility thereof based on the fused features;

[0037] The warning generation module is used to generate warning information according to the prediction results and provide it to pedestrians and drivers through multiple transmission channels;

[0038] The real-time monitoring module is used to monitor the operating status of the system and the effectiveness of the warning information in real time;

[0039] The storage management module is used to use blockchain technology to decentralize the storage of collected data, feature extraction results and prediction information;

[0040] The feedback optimization module is used to optimize the algorithms and parameters of the system according to the data and feedback in actual operation.

[0041] As a further solution of the present invention, the optimization and adjustment module optimizes cross-camera matching to generate a comprehensive information graph in the following specific steps:

[0042] S4.1: Initialize the number of adjustment bodies, pheromone concentration, heuristic factor, and pheromone volatility, initialize the pheromone concentration matrix on each matching pair in the comprehensive information graph, and then use the feature points of the same object captured by every two different cameras as nodes, and randomly distribute the positions of the nodes of each adjustment body;

[0043] S4.2: Each adjustment body starts from the starting node and calculates its corresponding selection probability according to the pheromone concentration and heuristic information of the remaining nodes. Then, based on each selection probability, it selects the next node to move by random selection, and repeats the selection and movement until a complete matching path is constructed;

[0044] S4.3: Evaluate the quality of the path constructed by each adjustment body, and update the pheromone concentration of the path according to the path fitness of the adjustment body, repeatedly iterate the path construction and update the pheromone concentration until the pheromone concentration change value reaches the preset threshold value;

[0045] S4.4: Compare the pheromone concentrations of each group of constructed paths, and take the path with the highest pheromone concentration as the current optimal matching solution. At the same time, update the matching pairs in the comprehensive information graph generated by cross-camera matching according to the output optimal matching solution.

[0046] As a further solution of the present invention, the multiple transmission channels of the warning generation module specifically include voice prompts, vehicle network broadcasts, and large screens to inform pedestrians of upcoming vehicle information and warnings.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The pedestrian crossing intention prediction method from a bird's-eye view receives each set of feature data output by the Transformer model through a classifier, and linearly transforms the feature vector and the weight vector of each category of pedestrian crossing intention to map the input feature to each category space, and obtain the unnormalized score of each category, and converts the unnormalized score of each category into a probability distribution, and selects the category with the highest probability as the final prediction result according to the probability distribution result, as the output of the pedestrian's crossing intention, and calculates the loss value between the output result and the actual value through the cross entropy loss function, and then starts from the classifier output layer, and back-propagates the calculated loss value to update the classifier parameters, and takes the updated parameter combination as the root node, and randomly perturbs it to generate multiple groups of different parameter combinations, and uses it as a child node to construct an exploration tree, starting from the root node, and selects the child node with the highest UCB1 value as the next node through the UCB1 strategy, and continuously Node selection is performed until a leaf node is reached. If the leaf node is not a final state, the parameter combination represented by the child node is expanded, and the expansion result is added to the tree as a new child node. Simulation is performed on the expanded node through a random strategy, and the loss value of each child node is calculated until the preset simulation time is reached and the simulation result is traced back to all nodes on the path. At the same time, the information of each node is updated, and the selection, expansion, simulation and backtracking steps are fed back until the change in the loss value reaches the preset threshold and stops. According to the loss value results of each child node in the final exploration tree, the parameter combination with the smallest loss value is selected as the optimal parameter combination output of the classifier, and it replaces the original parameter combination. At the same time, the classifier is trained and verified, which helps to capture the uncertainty in pedestrian behavior, can dynamically adapt to the changing trend of pedestrian behavior, provide more flexible and intelligent intention prediction capabilities, enhance adaptability in diverse scenarios, and reduce the probability of misclassification.

[0049] 2. The present invention initializes the number of adjustment bodies, pheromone concentration, heuristic factor and pheromone volatility, initializes the pheromone concentration matrix on the matching pairs in each pair of comprehensive information graph, and then takes the feature points of the same object captured by every two different cameras as nodes, randomly distributes the node positions of each adjustment body, and each adjustment body starts from the starting node, and calculates its corresponding selection probability according to the pheromone concentration and heuristic information of the remaining nodes. Then, based on each selection probability, the next node is selected by random selection to move, and the selection movement is repeated until a complete matching path is constructed, and the path quality constructed by each adjustment body is evaluated, and according to The path fitness of the adjustment body is updated to update the pheromone concentration of the path. The path is constructed and the pheromone concentration is updated repeatedly until the pheromone concentration change value reaches the preset threshold. The pheromone concentrations of each group of constructed paths are compared, and the path with the highest pheromone concentration is used as the current optimal matching solution. At the same time, the matching pairs in the comprehensive information graph generated by cross-camera matching are updated according to the output optimal matching solution. This can reduce the probability of mismatching, help reduce information loss and error accumulation, improve the robustness of cross-view information fusion, optimize the input data quality of the overall pedestrian intention prediction, and provide a more stable basis for pedestrian crossing intention prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0051] Figure 1 A flowchart of a method for predicting pedestrian crossing traffic intention from a bird's-eye view proposed by the present invention;

[0052] Figure 2 This is a system block diagram of the pedestrian crossing traffic intention warning system from a bird's-eye view proposed by the present invention. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0054] Example 1

[0055] Reference Figure 1 This embodiment discloses a method for predicting pedestrian crossing traffic intention from a bird's-eye view. The specific steps of the prediction method are as follows:

[0056] Collect and process each set of data and perform cross-camera matching.

[0057] Specifically, the intrinsic parameters of each camera are obtained through the checkerboard calibration method, and then the relative positions and orientations of the cameras are collected through the calibration plate, and the original images collected by each camera are respectively constructed into a group of image pyramids, and a pixel is selected in each layer of image data, and a circle is constructed with the selected pixel as the center according to the preset radius, and the difference between each pixel point on the circumference and the selected pixel is calculated. If there are multiple groups of pixel points whose values ​​are higher or lower than the preset threshold than the selected pixel point, the selected pixel point is marked as a corner point, and non-maximum suppression is performed on each group of marked corner points, and the grayscale centroid method is used to calculate the direction of the feature point, and the BRIEF descriptor is generated by performing binary comparison on the pixel pairs in the neighborhood of the feature point, so that the pixel point with a pixel value less than the feature point in the field of each feature point is marked as 1, otherwise it is marked as 0, and the feature point is used to generate a BRIEF descriptor. The FLANN matcher is used to find the feature descriptor that is closest to the corresponding descriptor in another image through approximate nearest neighbor search to obtain multiple sets of matching point pairs and construct a set of matching point pairs. A set of samples is randomly selected from the set of matching point pairs, and the parameters of the geometric transformation model are calculated based on the selected random samples. All the remaining matching points are output to the geometric transformation model, and the error between the predicted value and the actual value of each matching point is calculated. If there is a pair of matching points with an error less than a preset threshold, it will be judged as a successful match, otherwise it will be regarded as a failed match. With multiple iterations, the misplaced matching points are eliminated, the coordinates of the feature points under different perspectives are spatially transformed to align them in a unified coordinate system, and the pedestrian and vehicle information captured by different cameras is integrated into the same perspective to form a comprehensive information map.

[0058] Optimize the matching of the completed comprehensive information graph and extract feature data;

[0059] Integrate each group of feature information and predict pedestrian crossing intention.

[0060] Specifically, the image data and radar point cloud data in the comprehensive information map are aligned to the same coordinate system through spatial calibration, and the feature information of the image data and the radar point cloud data are extracted through the ResNet network and the point cloud network respectively. Then, each group of feature information is serialized, and position encoding is added to each group of feature information. The processed feature information is input into the Transformer model, and the input features are divided into multiple subspaces through the multi-head self-attention mechanism, and the attention is independently calculated on each subspace through the self-attention mechanism. Then, the image features and point cloud features processed by the multi-head self-attention are weighted fused through the Transformer model fusion layer. The fused features are aligned with the global features generated by the encoder through the cross attention mechanism of the Transformer decoder. The feature data of each position is nonlinearly processed through the fully connected layer in the feedforward neural network, and the input features are added to the output processed by the self-attention and feedforward networks using residual connections. The addition result is standardized, and the geometric characteristics of the road are identified based on the processed data, and the positions, genders, and age characteristics of the surrounding pedestrians, as well as the speed and position of the vehicle are marked.

[0061] Specifically, the classifier receives each set of feature data output by the Transformer model, and linearly transforms the feature vector and the weight vector of each category of pedestrian crossing intention to map the input feature to each category space and obtain the unnormalized score of each category. The unnormalized scores of each category are converted into probability distribution, and the category with the highest probability is selected as the final prediction result according to the probability distribution result, which is used as the output of the pedestrian's crossing intention, where P(y=c|h) represents the probability that the pedestrian belongs to category c, h represents the feature data output from the Transformer model, c represents a specific category label, c′ represents the index of all categories, and z c represents the unnormalized score corresponding to category c; z c′ represents the unnormalized scores corresponding to all categories c′, represents the index value of the current category c, The sum of the exponential values ​​representing all categories is used to calculate the loss value between the output result and the actual value through the cross entropy loss function. Then, starting from the output layer of the classifier, the calculated loss value is back-propagated to update the classifier parameters. The updated parameter combination is used as the root node, and it is randomly perturbed to generate multiple sets of different parameter combinations. At the same time, it is used as a child node to build an exploration tree. Starting from the root node, the child node with the highest UCB1 value is selected as the next node through the UCB1 strategy. Node selection is continuously performed until a leaf node is reached. If the leaf node is not the final state, the parameter represented by the child node is changed to the next node. The combination is expanded and the expanded result is added to the tree as a new child node. The simulation is performed on the expanded node through a random strategy, and the loss value of each child node is calculated until the preset simulation time is reached and stopped. The simulation results are traced back to all nodes on the path, and the information of each node is updated at the same time. The selection, expansion, simulation and backtracking steps are fed back and stopped until the change in the loss value reaches the preset threshold. According to the loss value results of each child node in the final exploration tree, the parameter combination with the smallest loss value is selected as the optimal parameter combination output of the classifier, and it replaces the original parameter combination, and the classifier is trained and verified.

[0062] Optimize the forecasting process and store collected and forecasted data through blockchain.

[0063] Generate real-time warning information based on the prediction results and collect various feedback information.

[0064] Adjust and optimize the system based on feedback information and deploy it to actual applications.

[0065] Example 2

[0066] Reference Figure 2 , this embodiment discloses a pedestrian crossing traffic intention warning system from a bird's eye view perspective, including a collection and processing module, a data matching module, an optimization and adjustment module, an extraction and fusion module, an intention prediction module, a warning generation module, a real-time monitoring module, a storage management module, and a feedback optimization module;

[0067] The acquisition and processing module is used to obtain video images, environmental point cloud data, and location information of people and vehicles from cameras, lidars, and GPS devices; the data matching module is used to match the pedestrian and vehicle information captured by different cameras to generate a comprehensive information map containing multiple camera perspectives; the optimization and adjustment module is used to optimize cross-camera matching to generate a comprehensive information map.

[0068] Specifically, the number of adjustment bodies, pheromone concentration, heuristic factor and pheromone volatility are initialized, and the pheromone concentration matrix is ​​initialized on the matching pairs in each pair of comprehensive information graphs. Then, the feature points of the same object captured by every two different cameras are used as nodes, and the node positions of each adjustment body are randomly distributed. Each adjustment body starts from the starting node, and its corresponding selection probability is calculated according to the pheromone concentration and heuristic information of the remaining nodes. Then, based on each selection probability, the next node is selected by random selection to move, and the selection and movement are repeated until a complete matching path is constructed, the path quality constructed by each adjustment body is evaluated, and the pheromone concentration of the path is updated according to the path fitness of the adjustment body. The path is constructed and the pheromone concentration is updated repeatedly until the pheromone concentration change value reaches a preset threshold and stops. The pheromone concentrations of each group of paths constructed are compared, and the path with the highest pheromone concentration is used as the current optimal matching solution. At the same time, the matching pairs in the comprehensive information graph generated by cross-camera matching are updated according to the output optimal matching solution.

[0069] The extraction and fusion module is used to extract key features from the comprehensive information graph and preprocessed data through the Transformer model for fusion; the intention prediction module is used to predict whether the pedestrian has the intention to cross the street and the possibility based on the fused features; the warning generation module is used to generate warning information based on the prediction results and provide it to pedestrians and drivers through multiple transmission channels.

[0070] In this embodiment, the various transmission channels of the warning generation module specifically include voice prompts, Internet of Vehicles broadcasts, and large screens to inform pedestrians of upcoming vehicle information and warnings.

[0071] The real-time monitoring module is used to monitor the operating status of the system and the effectiveness of warning information in real time; the storage management module is used to use blockchain technology to decentralize the storage of collected data, feature extraction results and prediction information; the feedback optimization module is used to optimize the system's algorithms and parameters based on actual operating data and feedback.

Claims

1. A method for predicting pedestrian crossing intention from a bird's eye view, characterized in that: The specific steps of the prediction method are as follows: S1, collect and process each set of data and perform cross-camera matching; S2, optimizing the comprehensive information graph after matching and extracting feature data; S3, integrating each group of feature information and predicting pedestrians’ traffic intentions when crossing the street; S4. Optimize the prediction process and store the collected and predicted data through blockchain; S5. Generate real-time warning information based on the prediction results and collect various feedback information; S6. Adjust and optimize the system based on feedback information and deploy it for actual application.

2. The method for predicting pedestrian crossing intention from a bird's eye view according to claim 1 is characterized in that: The specific steps of cross-camera matching in S1 are as follows: Q1.1: Obtain the intrinsic parameters of each camera through the checkerboard calibration method, then collect the relative positions and orientations of the cameras through the calibration board, and construct a set of image pyramids with the original images collected by each camera, and select a pixel in each layer of image data; Q1.2: Construct a circle with the selected pixel as the center according to the preset radius, and calculate the difference between each pixel on the circumference and the selected pixel. If there are multiple groups of pixels whose values ​​are higher or lower than the preset threshold than the selected pixel, the selected pixel is marked as a corner point, and non-maximum suppression is performed on each group of marked corner points. Q1.3: Use the grayscale centroid method to calculate the direction of the feature point, and generate the BRIEF descriptor by performing binary comparison on the pixel pairs in the neighborhood of the feature point. The pixel points whose pixel values ​​in the neighborhood of each feature point are smaller than the feature point are marked as 1, otherwise they are marked as 0. Use the FLANN matcher to find the feature descriptor closest to the corresponding descriptor in another image through approximate nearest neighbor search to obtain multiple sets of matching point pairs and construct a set of matching point pairs. Q1.4: Randomly select a group of samples from the set of matching point pairs, and calculate the parameters of the geometric transformation model based on the selected random samples. Output all the remaining matching points to the geometric transformation model, and calculate the error between the model's predicted value and the actual value of each matching point. If there is a pair of matching points whose error is less than the preset threshold, it will be considered a successful match, otherwise it will be considered a failed match. Repeat multiple iterations to remove misaligned matching points. Q1.5: Spatially transform the coordinates of feature points from different perspectives to align them in a unified coordinate system, and integrate the pedestrian and vehicle information captured by different cameras into the same perspective to form a comprehensive information map.

3. The method for predicting pedestrian crossing intention from a bird's eye view according to claim 2 is characterized in that: The specific steps of fusing each set of feature information in S3 are as follows: Q2.1: Align the image data and radar point cloud data in the comprehensive information map to the same coordinate system through spatial calibration, extract the feature information of the image data and radar point cloud data through the ResNet network and the point cloud network respectively, then serialize each group of feature information and add position coding to each group of feature information; Q2.2: Input the processed feature information into the Transformer model, divide the input features into multiple subspaces through the multi-head self-attention mechanism, and independently calculate the attention on each subspace through the self-attention mechanism. Then, through the Transformer model fusion layer, the image features and point cloud features processed by the multi-head self-attention are weighted fused; Q2.3: The fused features are aligned with the global features generated by the encoder through the cross-attention mechanism of the Transformer decoder. The feature data at each position is processed nonlinearly through the fully connected layer in the feedforward neural network, and the input features are added to the output processed by the self-attention and feedforward networks using residual connections. Q2.4: The summed results are standardized, and the geometric characteristics of the road are identified based on the processed data. The location, gender, and age characteristics of the surrounding pedestrians, as well as the speed and location of the vehicle are marked.

4. The method for predicting pedestrian crossing intention from a bird's eye view according to claim 3 is characterized in that: The specific steps of predicting pedestrian crossing traffic intention in S3 are as follows: S3.1: The classifier receives each set of feature data output by the Transformer model, and linearly transforms the feature vector and the weight vector of each category of pedestrian crossing intention to map the input features to each category space and obtain the unnormalized score of each category; S3.2: Pass The unnormalized scores of each category are converted into probability distribution, and the category with the highest probability is selected as the final prediction result according to the probability distribution result, which is used as the output of the pedestrian's crossing intention, where P(y=c|h) represents the probability that the pedestrian belongs to category c, h represents the feature data output from the Transformer model, c represents a specific category label, c′ represents the index of all categories, and z c represents the unnormalized score corresponding to category c; z c′ represents the unnormalized scores corresponding to all categories c′, represents the index value of the current category c, represents the sum of index values ​​for all categories; S3.3: The loss value between the output result and the actual value is calculated through the cross entropy loss function. Then, starting from the classifier output layer, the calculated loss value is back-propagated to update the classifier parameters. The updated parameter combination is used as the root node, and it is randomly perturbed to generate multiple groups of different parameter combinations, and it is used as a child node to build an exploration tree. S3.4: Starting from the root node, the child node with the highest UCB1 value is selected as the next node through the UCB1 strategy, and the node selection is continued until a leaf node is reached. If the leaf node is not in the final state, the parameter combination represented by the child node is expanded, and the expansion result is added to the tree as a new child node; S3.5: Perform simulation on the expanded nodes through random strategies and calculate the loss value of each child node until the preset simulation time is reached and then stop. The simulation results are then traced back to all nodes on the path, and the information of each node is updated. The selection, expansion, simulation and backtracking steps are fed back and executed until the change in the loss value reaches the preset threshold and then stops. S3.6: Based on the loss value results of each child node in the final exploration tree, select the parameter combination with the smallest loss value as the optimal parameter combination output of the classifier, and replace it with the original parameter combination, and train and verify the classifier at the same time.

5. A pedestrian crossing traffic intention warning system from a bird's-eye view, used to implement the pedestrian crossing traffic intention prediction method from a bird's-eye view as described in any one of claims 1 to 4, characterized in that: It includes acquisition and processing module, data matching module, optimization and adjustment module, extraction and fusion module, intention prediction module, warning generation module, real-time monitoring module, storage management module and feedback optimization module; The acquisition and processing module is used to obtain video images, environmental point cloud data, and location information of people and vehicles from cameras, laser radars, and GPS devices; The data matching module is used to match pedestrian and vehicle information captured by different cameras to generate a comprehensive information map containing multiple camera perspectives; The optimization and adjustment module is used to optimize cross-camera matching to generate a comprehensive information graph; The extraction and fusion module is used to extract key features from the comprehensive information graph and preprocessed data through the Transformer model for fusion; The intention prediction module is used to predict whether the pedestrian intends to cross the street and the possibility thereof based on the fused features; The warning generation module is used to generate warning information according to the prediction results and provide it to pedestrians and drivers through multiple transmission channels; The real-time monitoring module is used to monitor the operating status of the system and the effectiveness of the warning information in real time; The storage management module is used to use blockchain technology to decentralize the storage of collected data, feature extraction results and prediction information; The feedback optimization module is used to optimize the algorithms and parameters of the system according to the data and feedback in actual operation.

6. The pedestrian crossing traffic intention warning system from a bird's eye view according to claim 5 is characterized in that: The specific steps of the optimization and adjustment module to optimize cross-camera matching and generate a comprehensive information graph are as follows: S4.1: Initialize the number of adjustment bodies, pheromone concentration, heuristic factor, and pheromone volatility, initialize the pheromone concentration matrix on each matching pair in the comprehensive information graph, and then use the feature points of the same object captured by every two different cameras as nodes, and randomly distribute the positions of the nodes of each adjustment body; S4.2: Each adjustment body starts from the starting node and calculates its corresponding selection probability according to the pheromone concentration and heuristic information of the remaining nodes. Then, based on each selection probability, it selects the next node to move by random selection, and repeats the selection and movement until a complete matching path is constructed; S4.3: Evaluate the quality of the path constructed by each adjustment body, and update the pheromone concentration of the path according to the path fitness of the adjustment body, repeatedly iterate the path construction and update the pheromone concentration until the pheromone concentration change value reaches the preset threshold value; S4.4: Compare the pheromone concentrations of each group of constructed paths, and take the path with the highest pheromone concentration as the current optimal matching solution. At the same time, update the matching pairs in the comprehensive information graph generated by cross-camera matching according to the output optimal matching solution.

7. The pedestrian crossing traffic intention warning system from a bird's eye view according to claim 5 is characterized in that: The multiple transmission channels of the warning generation module specifically include voice prompts, vehicle network broadcasts, and large screens to inform pedestrians of upcoming vehicle information and warnings.

Images