A scene modeling method and system for intelligent transportation based on digital twins

Abstract

The present invention relates to the technical field of traffic scene modeling, and discloses a scene modeling method for realizing smart transportation based on digital twins. The method comprises: collecting environmental point clouds of a traffic modeling area, splicing the point clouds to obtain a spliced ​​point cloud to construct an environmental model of the traffic scene, and constructing a traffic simulation network; performing coordinate unification on the environmental model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network; fusing the coordinate unified model and the coordinate unified network to obtain an initial traffic scene modeling model; collecting a traffic map of the traffic modeling area; performing image visual positioning on the traffic map to obtain a positioned traffic map to fill the initial traffic scene modeling model to obtain a filled scene modeling model; collecting traffic information of the traffic modeling area; and using the traffic information to update the filled scene modeling model with traffic information to obtain a target traffic modeling model. The present invention can improve the analysis accuracy of traffic modeling.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic scene modeling, and in particular to a scene modeling method and system for realizing smart transportation based on digital twins. Background Art

[0002] Scenario modeling in smart transportation refers to the process of using various technical means and methods to describe, simulate, and analyze elements of the traffic environment, such as vehicles, road networks, and pedestrians, in order to better understand and optimize traffic conditions. Scenario modeling provides data support for intelligent transportation systems. It can be used to predict traffic flows, optimize signal control and intersection design, and improve road efficiency. It can also evaluate the effectiveness of different traffic design options and optimize traffic planning and design based on simulation results. This enables intelligent scheduling, route planning, congestion relief, and other functions, improving the efficiency and safety of the entire transportation system. Therefore, scenario modeling is of great significance to modern transportation.

[0003] At present, scene modeling under smart transportation is generally achieved through sensor data analysis technology. Various sensors (such as cameras, radars, lidars, etc.) are used to collect data, and traffic scene models are established through data processing and analysis, such as vehicle flow, speed distribution, congestion, etc. However, when using this method, due to the large amount of data, the data collected by the sensors may be interfered with or damaged, requiring quality control and calibration, and it is impossible to achieve the effect of real-time monitoring of traffic flow and conducting traffic analysis. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a scenario modeling method for smart transportation based on digital twins, which can improve the analysis accuracy of traffic modeling.

[0005] In a first aspect, the present invention provides a method for implementing scene modeling in smart transportation based on digital twins, comprising:

[0006] An IoT device is deployed in the traffic modeling area, and after establishing a communication connection with a preconfigured digital twin system using the IoT device, an environmental point cloud of the traffic modeling area is collected, the environmental point cloud is filtered to obtain a filtered point cloud, and point cloud stitching is performed on the filtered point cloud to obtain a stitched point cloud;

[0007] Constructing an environmental model of the traffic scene using the spliced ​​point cloud, and constructing a traffic simulation network of the traffic modeling area using the preconfigured digital twin system, performing coordinate unification on the environmental model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network, and fusing the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model;

[0008] Collecting a traffic map of the traffic modeling area, extracting a multi-layer feature map of the traffic map, performing feature fusion on multi-scale features of the multi-layer feature map to obtain a fused feature, constructing a feature descriptor of the fused feature, and performing image visual positioning on the traffic map based on the feature descriptor to obtain a positioned traffic map;

[0009] The initial traffic scene modeling model is filled with traffic scenes using the positioning traffic map to obtain a filled scene modeling model, the traffic information of the traffic modeling area is collected in real time using the Internet of Things device, and the traffic information is used to update the filled scene modeling model to obtain a target traffic modeling model.

[0010] In a possible implementation of the first aspect, filtering the environment point cloud to obtain a filtered point cloud includes:

[0011] Dividing the environment point cloud into a three-dimensional voxel grid;

[0012] Identifying the centroid of the three-dimensional voxel grid;

[0013] Based on the centroid point, the three-dimensional voxel grid is de-densified to obtain a filtered point cloud.

[0014] In a possible implementation of the first aspect, performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud includes:

[0015] Extracting adjacent point clouds from the filtered point cloud;

[0016] Creating a rotation matrix of the filtered point cloud using the adjacent point clouds;

[0017] The rotation matrix can be expressed in the following form:

[0018]

[0019] Among them, α represents the rotation matrix, r represents the cross vector of adjacent point clouds, Zr x , Z y , Z z Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, θ represents the intersection angle of adjacent point clouds, and e represents the unit vector of adjacent point clouds;

[0020] After the ground normal vector angles of the filtered point clouds are rotated to be consistent using the rotation matrix, the filtered point clouds are spliced ​​to obtain a spliced ​​point cloud.

[0021] In a possible implementation of the first aspect, constructing the environment model of the traffic scene using the spliced ​​point cloud includes:

[0022] performing object detection on the traffic scene according to the spliced ​​point cloud;

[0023] Performing target segmentation on the spliced ​​point cloud based on the result of the object detection to obtain a segmented point cloud;

[0024] The traffic scene is three-dimensionally modeled based on the segmented point cloud to obtain an environment model.

[0025] In a possible implementation of the first aspect, coordinate unification of the environment model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network includes:

[0026] The first coordinate transformation matrix of the environment model is created using the following formula:

[0027]

[0028] Among them, δ(1) represents the first coordinate transformation matrix, Indicates the angle between the three-dimensional model of the traffic scene in the environment model and the plane;

[0029] The second network coordinate transformation matrix of the traffic simulation network is created using the following formula:

[0030]

[0031] Among them, δ(2) represents the second coordinate transformation matrix, Indicates the angle between the network model and the plane in the traffic simulation network;

[0032] The coordinates of the environment model and the traffic simulation network are adjusted based on the first coordinate transformation matrix and the second network coordinate transformation matrix to obtain a coordinate unified model and a coordinate unified network.

[0033] In a possible implementation of the first aspect, coordinate unification of the environment model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network includes:

[0034] The first coordinate transformation matrix of the environment model is created using the following formula:

[0035]

[0036] Among them, δ(1) represents the first coordinate transformation matrix, Indicates the angle between the three-dimensional model of the traffic scene in the environment model and the plane;

[0037] The second network coordinate transformation matrix of the traffic simulation network is created using the following formula:

[0038]

[0039] Among them, δ(2) represents the second coordinate transformation matrix, Indicates the angle between the network model and the plane in the traffic simulation network;

[0040] The coordinates of the environment model and the traffic simulation network are adjusted based on the first coordinate transformation matrix and the second network coordinate transformation matrix to obtain a coordinate unified model and a coordinate unified network.

[0041] In a possible implementation of the first aspect, extracting a multi-layer feature map of the traffic map includes:

[0042] Inputting the traffic map into a preconfigured residual network;

[0043] Performing horizontal pooling and vertical pooling on the traffic map using the residual network to obtain a cascade feature map;

[0044] Convolving the cascaded feature map to obtain a convolution feature map;

[0045] Regularization is performed on the convolutional feature map to obtain a multi-layer feature map.

[0046] In a possible implementation of the first aspect, the performing feature fusion on the multi-scale features of the multi-layer feature map to obtain the fused features includes:

[0047] aligning the multi-scale features to obtain aligned features;

[0048] Performing weight assignment on the alignment features to obtain feature weights;

[0049] The multi-scale features are linearly weighted fused based on the feature weights to obtain fused features.

[0050] In a possible implementation of the first aspect, performing image visual positioning on the traffic map based on the feature descriptor to obtain the positioned traffic map includes:

[0051] Constructing a reference vector of the traffic map;

[0052] The feature descriptor is converted into a feature vector, and the cosine similarity between the reference vector and the feature vector is calculated using the following formula:

[0053]

[0054] Among them, D represents the cosine similarity, M represents the vector dimension of the reference vector, g i represents the i-th vector in the reference vector, T irepresents the i-th vector in the eigenvector;

[0055] The traffic map is visually positioned according to the cosine distance to obtain a positioned traffic map.

[0056] In a second aspect, the present invention provides a scene modeling system for smart transportation based on digital twins, the system comprising:

[0057] A point cloud stitching module is configured to deploy an IoT device in a traffic modeling area, establish a communication connection between the IoT device and a preconfigured digital twin system, collect an environmental point cloud of the traffic modeling area, filter the environmental point cloud to obtain a filtered point cloud, and stitch the filtered point cloud to obtain a stitched point cloud.

[0058] an initial scene modeling module, configured to construct an environmental model of the traffic scene using the spliced ​​point cloud, and to construct a traffic simulation network of the traffic modeling area using the preconfigured digital twin system, to coordinately unify the environmental model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network, and to fuse the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model;

[0059] An image visual positioning module is used to collect a traffic map of the traffic modeling area, extract a multi-layer feature map of the traffic map, perform feature fusion on multi-scale features of the multi-layer feature map to obtain a fused feature, construct a feature descriptor of the fused feature, and perform image visual positioning on the traffic map based on the feature descriptor to obtain a positioned traffic map;

[0060] The traffic scene modeling module is used to use the positioning traffic map to fill the traffic scene of the initial traffic scene modeling model to obtain a filled scene modeling model, use the Internet of Things device to collect traffic information of the traffic modeling area in real time, and use the traffic information to update the filled scene modeling model to obtain a target traffic modeling model.

[0061] Compared with the existing technology, the technical principle and beneficial effects of this solution are:

[0062] The embodiment of the present invention can monitor traffic flow, vehicle speed and density in real time by deploying IoT devices in the traffic modeling area, helping traffic management departments to better plan roads, traffic lights and traffic flows. After establishing a communication connection with the preconfigured digital twin system using the IoT device, the environmental point cloud collected in the traffic modeling area can be used to establish environmental maps, perform target detection and tracking, autonomous navigation and other applications. Furthermore, the embodiment of the present invention can help users analyze potential risk factors in the traffic scene and evaluate the effectiveness of various traffic safety measures by using the spliced ​​point cloud to construct an environmental model of the traffic scene. This helps to formulate and optimize traffic safety policies, design safer roads and traffic facilities, and by unifying the coordinates of the environmental model and the traffic simulation network respectively, obtaining a unified coordinate model and a unified coordinate network can ensure that the positions of each element in the environmental model on the network are accurate, so that spatial consistency can be maintained, making the correspondence between the model and the network more accurate. Furthermore, the embodiment of the present invention can provide basic data support for traffic simulation and simulation by collecting the traffic map of the traffic modeling area, which is very useful for evaluating the effects of different traffic strategies, predicting future traffic flow, and testing the robustness of the traffic system. In addition, the initial traffic scene modeling model is filled with traffic scenes using the positioning traffic map, and the filled scene modeling model can be used as the input of environmental perception data, thereby helping users to perform environmental perception and decision-making of autonomous driving systems or intelligent transportation systems, and can evaluate and improve the behavior and decision-making of the system by simulating different traffic scenes. The embodiment of the present invention proposes a scene modeling method and system based on digital twins to realize smart transportation, which can improve the analysis accuracy of traffic modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0065] Figure 1 A flowchart of a method for implementing scene modeling in smart transportation based on digital twins provided by one embodiment of the present invention;

[0066] Figure 2 A schematic diagram of a module for a scene modeling system for smart transportation based on digital twins, provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0068] An embodiment of the present invention provides a method for scene modeling in smart transportation based on digital twins. The execution subject of the method for scene modeling in smart transportation based on digital twins includes but is not limited to at least one of electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the method for scene modeling in smart transportation based on digital twins can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0069] See Figure 1 FIG2 is a flow chart of a method for implementing scene modeling in smart transportation based on digital twins according to an embodiment of the present invention. Figure 1 A scenario modeling method for realizing smart transportation based on digital twins described in [1] includes:

[0070] S1. Deploy an IoT device in the traffic modeling area, establish a communication connection with a preconfigured digital twin system using the IoT device, collect the environmental point cloud of the traffic modeling area, filter the environmental point cloud to obtain a filtered point cloud, and perform point cloud stitching on the filtered point cloud to obtain a stitched point cloud.

[0071] The embodiments of the present invention can monitor traffic flow, vehicle speed and density in real time by deploying IoT devices in the traffic modeling area, helping traffic management departments to better plan roads, traffic lights and traffic flows. The traffic modeling area refers to the scope of research and modeling of a specific geographical area in traffic planning and traffic flow analysis. This area can be a city or a certain area of ​​a city, such as the main road traffic area of ​​a city. The IoT devices refer to devices or device components connected to the IoT that can collect data, exchange information, and perform specific tasks. The IoT devices typically include sensors, controllers, and network connection functions, enabling them to communicate with other devices and realize functions such as remote monitoring, automation, and data collection.

[0072] Furthermore, in embodiments of the present invention, after establishing a communication connection between the IoT device and a preconfigured digital twin system, the collected environmental point cloud of the traffic modeling area can be used to create environmental maps, perform target detection and tracking, and perform autonomous navigation, among other applications. The environmental point cloud refers to a large amount of point data acquired in three-dimensional space through laser scanning or other sensors, and the digital twin system refers to a technology that uses digital models of physical entities or processes to reflect and simulate their state, behavior, and performance in real time. It can interconnect with physical entities, capture data through sensors and feed it back into the digital model, while also applying the digital model's predictions to the control and optimization of the physical entity.

[0073] The embodiment of the present invention performs filtering processing on the environmental point cloud to obtain a filtered point cloud, which can improve data quality, reduce computational complexity, extract key features, and provide a more accurate and reliable data basis for various applications.

[0074] As an embodiment of the present invention, the filtering processing of the environmental point cloud to obtain a filtered point cloud includes: dividing the environmental point cloud into three-dimensional voxel grids, identifying the center of gravity of the three-dimensional voxel grids, and based on the center of gravity, de-densifying the three-dimensional voxel grids to obtain a filtered point cloud.

[0075] Optionally, the dividing of the environmental point cloud into three-dimensional voxel grids is achieved through a voxel grid method, and the process of identifying the center of gravity of the three-dimensional voxel grid is as follows: traversing the number of voxel points in the three-dimensional voxel grid and calculating the center of the voxel points; based on the center of gravity, the three-dimensional voxel grid is de-densified to obtain a filtered point cloud, which is achieved through a voxel filtering function.

[0076] The embodiment of the present invention performs point cloud stitching on the filtered point cloud to obtain a stitched point cloud, which can stitch multiple point clouds together to generate a map of a larger range, which is crucial for establishing accurate geographic information system (GIS) data, navigation systems, and traffic modeling.

[0077] As an embodiment of the present invention, performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud includes: extracting adjacent point clouds from the filtered point cloud, and using the adjacent point clouds to create a rotation matrix of the filtered point cloud, wherein the rotation matrix can be expressed in the following form:

[0078]

[0079] Among them, α represents the rotation matrix, r represents the cross vector of adjacent point clouds, Zr x , Z y , Zz Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, θ represents the intersection angle of adjacent point clouds, and e represents the unit vector of adjacent point clouds;

[0080] After the ground normal vector angles of the filtered point clouds are rotated to be consistent using the rotation matrix, the filtered point clouds are spliced ​​to obtain a spliced ​​point cloud.

[0081] S2. Use the spliced ​​point cloud to construct an environmental model of the traffic scene, and use the preconfigured digital twin system to construct a traffic simulation network of the traffic modeling area, and unify the coordinates of the environmental model and the traffic simulation network respectively to obtain a coordinate unified model and a coordinate unified network, and fuse the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model.

[0082] By constructing an environmental model of a traffic scene using the spliced ​​point cloud, embodiments of the present invention can help users analyze potential hazards in traffic scenes and evaluate the effectiveness of various traffic safety measures. This helps formulate and optimize traffic safety policies and design safer roads and transportation facilities.

[0083] As an embodiment of the present invention, the use of the spliced ​​point cloud to construct an environmental model of the traffic scene includes: performing object detection on the traffic scene according to the spliced ​​point cloud, and performing target segmentation on the spliced ​​point cloud based on the result of the object detection to obtain a segmented point cloud, and performing three-dimensional modeling of the traffic scene based on the segmented point cloud to obtain an environmental model.

[0084] Optionally, the object detection on the traffic scene based on the stitched point cloud refers to identifying the scene conditions in the stitched point cloud, such as pedestrians, vehicles and roads, which can be identified by a target detection algorithm. The target segmentation on the stitched point cloud based on the result of the object detection to obtain the segmented point cloud is as follows: obtaining the target bounding box information of each stitched point cloud, and extracting a point cloud subset corresponding to the target segmentation from the stitched point cloud according to the target bounding box information to obtain a segmented point cloud. The three-dimensional modeling of the traffic scene based on the segmented point cloud is performed to obtain an environment model created by a 3ds tool.

[0085] Furthermore, the embodiment of the present invention can simulate the impact of traffic accidents on the traffic system by using the preconfigured digital twin system to construct a traffic simulation network in the traffic modeling area, thereby helping emergency management departments to better formulate traffic accident handling plans.

[0086] The embodiment of the present invention can ensure that the positions of various elements in the environmental model on the network are accurate by respectively unifying the coordinates of the environmental model and the traffic simulation network to obtain a unified coordinate model and a unified coordinate network, thereby maintaining spatial consistency and making the correspondence between the model and the network more accurate.

[0087] As an embodiment of the present invention, the coordinate unification of the environment model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network includes:

[0088] The first coordinate transformation matrix of the environment model is created using the following formula:

[0089]

[0090] Among them, δ(1) represents the first coordinate transformation matrix, Indicates the angle between the three-dimensional model of the traffic scene in the environment model and the plane;

[0091] The second network coordinate transformation matrix of the traffic simulation network is created using the following formula:

[0092]

[0093] Among them, δ(2) represents the second coordinate transformation matrix, Indicates the angle between the network model and the plane in the traffic simulation network;

[0094] The coordinates of the environment model and the traffic simulation network are adjusted based on the first coordinate transformation matrix and the second network coordinate transformation matrix to obtain a coordinate unified model and a coordinate unified network.

[0095] The embodiment of the present invention fuses the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model, which can be used to virtually display the traffic scene, thereby achieving the initial effect of digital twins.

[0096] As an embodiment of the present invention, the coordinate unified model is fused with the coordinate unified network to obtain an initial traffic scene modeling model, including: identifying the relative distance between the coordinate origin of the coordinate unified model and the coordinate origin of the coordinate unified network, performing coordinate scaling on the three-dimensional coordinates of the coordinate unified model and the three-dimensional coordinates of the coordinate unified network based on the relative distance to obtain a first matching model and a second matching model, and fusing the first matching model with the second matching model to obtain an initial traffic scene modeling model.

[0097] Optionally, the relative distance between the coordinate origin of the coordinate unified model and the coordinate origin of the coordinate unified network is identified by aligning the coordinate origin of the coordinate unified model with the coordinate origin of the coordinate unified network, and then calculating the three-axis distance between the coordinates of the coordinate unified model and the coordinates of the coordinate unified network. The three-dimensional coordinates of the coordinate unified model and the three-dimensional coordinates of the coordinate unified network are scaled based on the relative distance by taking the three-dimensional coordinates of the coordinate unified network as a reference and scaling the three-dimensional coordinates of the coordinate unified network according to the relative distance.

[0098] S3. Collect a traffic map of the traffic modeling area, extract a multi-layer feature map of the traffic map, perform feature fusion on multi-scale features of the multi-layer feature map to obtain a fused feature, construct a feature descriptor of the fused feature, and perform image visual positioning of the traffic map based on the feature descriptor to obtain a positioned traffic map.

[0099] The embodiment of the present invention can provide basic data support for traffic simulation and simulation by collecting the traffic map of the traffic modeling area, which is very useful for evaluating the effects of different traffic strategies, predicting future traffic flow, testing the robustness of the traffic system, etc. The traffic map can be obtained through sensors or satellite images.

[0100] The embodiment of the present invention can identify the connection relationship and road network between different areas by extracting the multi-layer feature map of the traffic map, which is helpful for tasks such as route planning, traffic congestion prediction and optimization.

[0101] As an embodiment of the present invention, the extraction of a multi-layer feature map of the traffic map includes: inputting the traffic map into a preconfigured residual network, using the residual network to perform horizontal pooling and vertical pooling on the traffic map to obtain a cascaded feature map, convolving the cascaded feature map to obtain a convolutional feature map, and regularizing the convolutional feature map to obtain a multi-layer feature map.

[0102] Optionally, the residual network is used to perform horizontal pooling and vertical pooling on the traffic map to obtain a cascaded feature map, which is obtained by processing the pooling layer in the preconfigured residual network; the cascaded feature map is convolved to obtain a convolutional feature map, which is obtained by processing the convolutional layer in the preconfigured residual network; and the convolutional feature map is regularized to obtain a multi-layer feature map through Lasso regularization.

[0103] Furthermore, the embodiment of the present invention performs feature fusion on the multi-scale features of the multi-layer feature map to obtain fused features that can capture information at different levels and angles, thereby improving the expressive power of features and providing a more comprehensive and rich data representation.

[0104] As an embodiment of the present invention, the feature fusing of the multi-scale features of the multi-layer feature map to obtain the fused features includes: scaling the multi-scale features to obtain alignment features, weighting the alignment features to obtain feature weights, and linearly weighted fusing the multi-scale features based on the feature weights to obtain the fused features.

[0105] Optionally, the scaling of the multi-scale features to obtain alignment features refers to aligning the different spatial resolutions of the multi-scale feature maps to a unified scale, which can be adjusted by interpolation or convolution and other methods. The weighting of the aligned features to obtain feature weights refers to a method of integrating the results of multiple data sources or models by assigning different weights to each data source or model. The linear weighted fusion of the multi-scale features based on the feature weights to obtain fused features can be obtained by linear weighted fusion using a weight fusion method.

[0106] The embodiment of the present invention extracts key, distinguishable information from the fused features by constructing the feature descriptors of the fused features, and represents the information in a structured manner to help users better understand and interpret the data.

[0107] Optionally, the feature descriptor may be constructed by a script generated by Java language.

[0108] The embodiment of the present invention performs image visual positioning of the traffic map based on the feature descriptors to obtain a positioned traffic map, which can provide valuable data support for urban planning and traffic system optimization. By analyzing the traffic hotspots and traffic flow distribution in the traffic map, decisions such as road expansion, public transportation optimization, and traffic light scheduling can be made.

[0109] As an embodiment of the present invention, performing image visual positioning on the traffic map based on the feature descriptor to obtain the positioned traffic map includes: constructing a reference vector of the traffic map, converting the feature descriptor into a feature vector, and calculating the cosine similarity between the reference vector and the feature vector using the following formula:

[0110]

[0111] Among them, D represents the cosine similarity, M represents the vector dimension of the reference vector, g i represents the i-th vector in the reference vector, T i represents the i-th vector in the eigenvector;

[0112] The traffic map is visually positioned according to the cosine distance to obtain a positioned traffic map.

[0113] Optionally, the traffic map is visually positioned according to the cosine distance to obtain a positioned traffic map. A reference image that is most similar to the test image is selected based on the result of cosine similarity, and the geographical location associated with the reference image is used as the final position of the test image. The cosine similarity is 0.95 and can also be set according to the actual scenario.

[0114] S4. Use the positioning traffic map to fill the initial traffic scene modeling model with traffic scenes to obtain a filled scene modeling model. Use the Internet of Things device to collect traffic information of the traffic modeling area in real time, and use the traffic information to update the filled scene modeling model with traffic information to obtain a target traffic modeling model.

[0115] The embodiment of the present invention uses the positioning traffic map to fill the initial traffic scene modeling model with traffic scenes, and the obtained filled scene modeling model can be used as input for environmental perception data, thereby helping users to perform environmental perception and decision-making for autonomous driving systems or intelligent transportation systems, and can evaluate and improve the system's behavior and decision-making by simulating different traffic scenarios.

[0116] Furthermore, the embodiment of the present invention can help users or systems perform traffic safety monitoring and accident warnings by using the Internet of Things device to collect traffic information of the traffic modeling area in real time through real-time traffic information, and can use vehicle speed, acceleration and other data to detect and warn of abnormal driving behavior, thereby reducing the occurrence of traffic accidents and improving road safety.

[0117] In the embodiment of the present invention, after the traffic information is updated on the filling scene modeling model using the traffic information, the target traffic modeling model is obtained and the latest traffic information can be analyzed in real time through the updated model, thereby performing more accurate data analysis and decision support.

[0118] As an embodiment of the present invention, the method of using the traffic information to update the filling scene modeling model to obtain the target traffic modeling model includes: using the traffic information to perform data twinning on the filling scene modeling model to obtain twin information, and calculating the multiple similarity loss of the twin information using the following formula:

[0119]

[0120] Among them, MS represents multiple similarity loss, N represents the maximum spatial distance between traffic information and twin information in the corresponding vector space, ρ, σ, τ are hyperparameters, S1 represents the first similarity between traffic information and twin information, S2 represents the second similarity between traffic information and twin information, A represents traffic information, B represents twin information, and U represents the same part of the traffic information and the twin information;

[0121] When the multiple similarity losses meet a preset loss value, a target traffic modeling model is obtained.

[0122] Optionally, when the multiple similarity losses meet the preset loss value, obtaining the target traffic modeling model means that in the step, the traffic information is used to perform data twinning on the filling scene modeling model, and when the twin information is obtained, the information loss meets the requirements, and the preset loss value is 0.1.

[0123] like Figure 2 The figure shows a functional module diagram of a scene modeling system for smart transportation based on digital twins according to the present invention.

[0124] The scene modeling system 200 for implementing smart transportation based on digital twins described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the scene modeling system 200 for implementing smart transportation based on digital twins can include a point cloud splicing module 201, an initial scene modeling module 202, an image visual positioning module 203, and a traffic scene modeling module 204. The modules described in the present invention, also referred to as units, refer to a series of computer program segments that can be executed by an electronic device processor and perform fixed functions, and are stored in the memory of the electronic device.

[0125] In the embodiment of the present invention, the functions of each module / unit are as follows:

[0126] The point cloud stitching module 201 is used to deploy an Internet of Things device in the traffic modeling area, establish a communication connection between the Internet of Things device and a preconfigured digital twin system, collect an environmental point cloud of the traffic modeling area, filter the environmental point cloud to obtain a filtered point cloud, and stitch the filtered point cloud to obtain a stitched point cloud.

[0127] The initial scene modeling module 202 is used to construct an environmental model of the traffic scene using the spliced ​​point cloud, and to construct a traffic simulation network of the traffic modeling area using the preconfigured digital twin system, and to unify the coordinates of the environmental model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network, and to fuse the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model;

[0128] The image visual positioning module 203 is used to collect a traffic map of the traffic modeling area, extract a multi-layer feature map of the traffic map, perform feature fusion on multi-scale features of the multi-layer feature map to obtain a fused feature, construct a feature descriptor of the fused feature, and perform image visual positioning on the traffic map based on the feature descriptor to obtain a positioned traffic map;

[0129] The traffic scene modeling module 204 is used to use the positioning traffic map to fill the initial traffic scene modeling model with traffic scenes to obtain a filled scene modeling model, use the Internet of Things device to collect traffic information of the traffic modeling area in real time, and use the traffic information to update the filled scene modeling model with traffic information to obtain a target traffic modeling model.

[0130] In detail, the modules in the scene modeling system 200 for realizing smart transportation based on digital twins in the embodiment of the present invention are used in the same manner as above. Figure 1 The technical means described in the article are similar to the scene modeling method based on digital twins for smart transportation, and can produce the same technical effects, so I will not go into details here.

[0131] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and actual implementation may employ other division methods.

[0132] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0133] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0134] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0135] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0136] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0137] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A scenario modeling method for realizing smart transportation based on digital twins, characterized in that: The method comprises: An Internet of Things device is deployed in a traffic modeling area. After establishing a communication connection with a preconfigured digital twin system using the Internet of Things device, an environmental point cloud of the traffic modeling area is collected, the environmental point cloud is filtered to obtain a filtered point cloud, and the filtered point cloud is stitched to obtain a stitched point cloud. The stitching of the filtered point cloud to obtain the stitched point cloud includes: extracting adjacent point clouds from the filtered point cloud, and using the adjacent point clouds to create a rotation matrix of the filtered point cloud, wherein the rotation matrix is ​​expressed in the following form: ； in, represents the rotation matrix, Represents the intersection vector of adjacent point clouds, Z x , , Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, represents the intersection angle of adjacent point clouds, Representing unit vectors of adjacent point clouds, after rotating the ground normal vectors of the filtered point clouds to the same angle using the rotation matrix, the filtered point clouds are stitched together to obtain a stitched point cloud; Using the spliced ​​point cloud to construct an environmental model of the traffic scene, and using the preconfigured digital twin system to construct a traffic simulation network of the traffic modeling area, respectively unifying the coordinates of the environmental model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network, and fusing the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model; Collecting a traffic map of the traffic modeling area, extracting a multi-layer feature map of the traffic map, performing feature fusion on multi-scale features of the multi-layer feature map to obtain a fused feature, constructing a feature descriptor of the fused feature, and performing image visual positioning on the traffic map based on the feature descriptor to obtain a positioned traffic map; The initial traffic scene modeling model is filled with traffic scenes using the positioning traffic map to obtain a filled scene modeling model, the traffic information of the traffic modeling area is collected in real time using the Internet of Things device, and the traffic information is used to update the filled scene modeling model to obtain a target traffic modeling model, wherein the method of using the traffic information to update the filled scene modeling model to obtain the target traffic modeling model includes: using the traffic information to perform data twinning on the filled scene modeling model to obtain twin information, and using the following formula to calculate the multiple similarity loss of the twin information: ； in, represents multiple similarity loss, N represents the maximum spatial distance between traffic information and twin information in the corresponding vector space, , , is a hyperparameter, represents the first similarity between traffic information and twin information, Indicates the second similarity between traffic information and twin information, A represents traffic information, B represents twin information, It represents the same part of the traffic information and the twin information, and when the multiple similarity losses meet the preset loss value, the target traffic modeling model is obtained.

2. The method according to claim 1, characterized in that The filtering process on the environment point cloud to obtain a filtered point cloud includes: Dividing the environment point cloud into a three-dimensional voxel grid; Identifying the centroid of the three-dimensional voxel grid; Based on the centroid point, the three-dimensional voxel grid is de-densified to obtain a filtered point cloud.

3. The method according to claim 1, characterized in that The method of constructing an environmental model of a traffic scene by using the spliced ​​point cloud comprises: performing object detection on the traffic scene according to the spliced ​​point cloud; Performing target segmentation on the spliced ​​point cloud based on the result of the object detection to obtain a segmented point cloud; The traffic scene is three-dimensionally modeled based on the segmented point cloud to obtain an environment model.

4. The method according to claim 1, wherein The coordinate unification of the environment model and the traffic simulation network is performed to obtain a coordinate unified model and a coordinate unified network, including: The first coordinate transformation matrix of the environment model is created using the following formula: ； in, represents the first coordinate transformation matrix, Indicates the angle between the three-dimensional model of the traffic scene in the environment model and the plane; The second network coordinate transformation matrix of the traffic simulation network is created using the following formula: ； in, represents the second coordinate transformation matrix, Indicates the angle between the network model and the plane in the traffic simulation network; The coordinates of the environment model and the traffic simulation network are adjusted based on the first coordinate transformation matrix and the second network coordinate transformation matrix to obtain a coordinate unified model and a coordinate unified network.

5. The method according to claim 1, characterized in that The step of fusing the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model includes: Identifying the relative distance between the coordinate origin of the coordinate unified model and the coordinate origin of the coordinate unified network; Based on the relative distance, coordinate scaling is performed on the three-dimensional coordinates of the coordinate unified model and the three-dimensional coordinates of the coordinate unified network to obtain a first matching model and a second matching model. The first matching model and the second matching model are fused to obtain an initial traffic scene modeling model.

6. The method according to claim 1, characterized in that The step of extracting a multi-layer feature map of the traffic map includes: Inputting the traffic map into a preconfigured residual network; Performing horizontal pooling and vertical pooling on the traffic map using the residual network to obtain a cascade feature map; Convolving the cascaded feature map to obtain a convolution feature map; Regularization is performed on the convolutional feature map to obtain a multi-layer feature map.

7. The method according to claim 1, characterized in that The step of fusing the multi-scale features of the multi-layer feature map to obtain fused features includes: aligning the multi-scale features to obtain aligned features; Performing weight assignment on the alignment features to obtain feature weights; The multi-scale features are linearly weighted fused based on the feature weights to obtain fused features.

8. The method according to claim 1, characterized in that The performing image visual positioning on the traffic map based on the feature descriptor to obtain a positioned traffic map includes: Constructing a reference vector of the traffic map; The feature descriptor is converted into a feature vector, and the cosine similarity between the reference vector and the feature vector is calculated using the following formula: ； in, represents the cosine similarity, M represents the vector dimension of the reference vector, represents the i-th vector in the base vector, represents the i-th vector in the eigenvector; The traffic map is visually positioned according to the cosine similarity to obtain a positioned traffic map.

9. A scene modeling system for smart transportation based on digital twins, characterized by: The system comprises: The point cloud stitching module is used to deploy an Internet of Things device in the traffic modeling area, establish a communication connection with the preconfigured digital twin system using the Internet of Things device, collect the environmental point cloud of the traffic modeling area, filter the environmental point cloud to obtain a filtered point cloud, and stitch the filtered point cloud to obtain a stitched point cloud. The stitching of the filtered point cloud to obtain the stitched point cloud includes: extracting adjacent point clouds from the filtered point cloud, and using the adjacent point clouds to create a rotation matrix of the filtered point cloud. The rotation matrix is ​​expressed in the following form: ； in, represents the rotation matrix, Represents the intersection vector of adjacent point clouds, Z x , , Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, represents the intersection angle of adjacent point clouds, Representing unit vectors of adjacent point clouds, after rotating the ground normal vectors of the filtered point clouds to the same angle using the rotation matrix, the filtered point clouds are stitched together to obtain a stitched point cloud; an initial scene modeling module, configured to construct an environmental model of the traffic scene using the spliced ​​point cloud, and to construct a traffic simulation network of the traffic modeling area using the preconfigured digital twin system, and to unify the coordinates of the environmental model and the traffic simulation network to obtain a coordinate unified model and a coordinate unified network, and to fuse the coordinate unified model with the coordinate unified network to obtain an initial traffic scene modeling model; An image visual positioning module is used to collect a traffic map of the traffic modeling area, extract a multi-layer feature map of the traffic map, perform feature fusion on multi-scale features of the multi-layer feature map to obtain a fused feature, construct a feature descriptor of the fused feature, and perform image visual positioning on the traffic map based on the feature descriptor to obtain a positioned traffic map; The traffic scene modeling module is used to use the positioning traffic map to fill the initial traffic scene modeling model with traffic scenes to obtain a filled scene modeling model, use the Internet of Things device to collect traffic information of the traffic modeling area in real time, and use the traffic information to update the filled scene modeling model with traffic information to obtain a target traffic modeling model, wherein the method of using the traffic information to update the filled scene modeling model with traffic information to obtain the target traffic modeling model includes: using the traffic information to perform data twinning on the filled scene modeling model to obtain twin information, and using the following formula to calculate the multiple similarity loss of the twin information: ； in, represents multiple similarity loss, N represents the maximum spatial distance between traffic information and twin information in the corresponding vector space, , , is a hyperparameter, represents the first similarity between traffic information and twin information, Indicates the second similarity between traffic information and twin information, A represents traffic information, B represents twin information, It represents the same part of the traffic information and the twin information, and when the multiple similarity losses meet the preset loss value, the target traffic modeling model is obtained.

Images