Urban slow traffic potential safety hazard checking method and system based on multimode data

Abstract

The invention discloses an urban slow traffic potential safety hazard checking method and system based on multimode data, and the method comprises the steps: obtaining traffic accident information, carrying out the data preprocessing, and obtaining a structured traffic accident information table; based on the structured traffic accident information table, black spot identification is carried out on an accident site through a Gaussian kernel function, and an accident high-frequency site street scene picture is obtained; performing semantic segmentation on the street scene picture of the accident high-frequency site through a DeeplabV3 + model to obtain a semantic segmentation result of the urban slow traffic accident; and carrying out slow traffic potential safety hazard analysis and visualization processing on the semantic segmentation result of the urban slow traffic accident to obtain an urban slow traffic potential safety hazard troubleshooting result. According to the invention, the efficiency and accuracy of slow traffic potential safety hazard checking can be improved. The urban slow traffic potential safety hazard checking method and system based on multimode data can be widely applied to the technical field of traffic safety.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety, and particularly to a method and system for detecting potential safety hazards in urban slow traffic based on multi-modal data. Background Art

[0002] With the increasing prominence of urban traffic congestion problems, the safety issues of urban slow traffic systems have gradually become the focus of people's attention. However, at present, there are problems in the detection of potential safety hazards in urban slow traffic systems, such as unclear data sources, difficult hazard identification, and high labor consumption, and there is a lack of an intelligent decision support system. Using artificial intelligence methods to obtain black spots of road network accidents and efficiently and accurately identify potential safety hazards in slow traffic can help relevant management departments improve the slow traffic environment, enhance road safety, and reduce the accident frequency. At present, the detection of potential safety hazards mainly relies on experts and scholars to conduct on-site manual inspections in person, which has high labor costs, low efficiency, and strong subjectivity. Although domestic and foreign scholars have carried out a large number of studies in the fields of accident mode analysis, hazard identification algorithms, etc., they mainly focus on motor vehicle accidents and ignore the slow traffic system. When applied to non-motor vehicle driving scenarios, there are still problems such as insufficient multi-source data fusion analysis capabilities and low intelligent level of hazard assessment. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for detecting potential safety hazards in urban slow traffic based on multi-modal data, which can improve the efficiency and accuracy of detecting potential safety hazards in slow traffic.

[0004] The first technical solution adopted by the present invention is: a method for detecting potential safety hazards in urban slow traffic based on multi-modal data, including the following steps:

[0005] Obtain traffic accident information and perform data preprocessing to obtain a structured traffic accident information table;

[0006] Based on the structured traffic accident information table, identify black spots at accident locations through a Gaussian kernel function to obtain street view pictures of accident high-frequency locations;

[0007] Perform semantic segmentation on the street view pictures of accident high-frequency locations through the deeplabV3+ model to obtain the semantic segmentation results of urban slow traffic accidents;

[0008] Perform analysis and visualization processing on the semantic segmentation results of urban slow traffic accidents to obtain the results of detecting potential safety hazards in urban slow traffic.

[0009] Furthermore, the step of obtaining traffic accident information and performing data preprocessing to obtain a structured traffic accident information table specifically includes:

[0010] Collect and process traffic accident data through a multi-source heterogeneous data crawling framework to obtain traffic accident information, where the traffic accident information includes structured text data and unstructured video data;

[0011] Extract target keywords from the structured text data to construct a text accident database, where the target keywords include accident time, geographical coordinates, accident type, and accident liability cause;

[0012] Perform key frame data interception processing on the unstructured video data to construct a video accident database, where the key frame data includes video shooting time, GPS coordinates, and user annotation information;

[0013] Perform spatio-temporal alignment and data fusion processing on the text accident database and the video accident database, and eliminate duplicate data to obtain a standardized slow traffic accident data set;

[0014] Perform data cleaning and unification processing on the standardized slow traffic accident data set to obtain a structured traffic accident information table.

[0015] Furthermore, the step of identifying black spots at accident locations through the Gaussian kernel function based on the structured traffic accident information table to obtain street view pictures of high-frequency accident locations specifically includes:

[0016] Obtain the traffic accident text address according to the structured traffic accident information table and perform longitude and latitude conversion processing to obtain traffic accident point coordinate information;

[0017] Perform spatial density interpolation processing on the traffic accident point coordinate information through the Gaussian kernel function to obtain continuous kernel density raster data;

[0018] Obtain the administrative division vector map of the target city and perform spatial superposition with the continuous kernel density raster data to obtain the fused target city raster map;

[0019] Perform density value grading on the fused target city raster map through the natural breakpoint method, generate a multi-level gradient heat map and add marked elements to obtain street view pictures of high-frequency accident locations.

[0020] Furthermore, the expression of the Gaussian kernel function is specifically as follows:

[0021]

[0022] In the above formula, f(x) represents the spatial density interpolation of the Gaussian kernel function, K represents the kernel function, h represents the fixed search radius, d(x,x i ) represents the distance between point x and the accident point x i , and b represents the number of accident locations.

[0023] Further, the step of performing semantic segmentation on the street view images of accident high-frequency locations through the DeeplabV3+ model to obtain the semantic segmentation results of urban slow traffic accidents specifically includes:

[0024] Obtain the Cityscapes dataset and several street view images and perform data mixing to construct a mixed dataset;

[0025] Adopt the semi-supervised learning method with pseudo-labels and iteratively train the DeeplabV3+ model through the mixed dataset to construct the trained DeeplabV3+ model;

[0026] Based on the encoder and decoder in the trained DeeplabV3+ model, perform semantic segmentation on the street view images of accident high-frequency locations to obtain the semantic segmentation results of urban slow traffic accidents.

[0027] Further, the step of adopting the semi-supervised learning method with pseudo-labels and iteratively training the DeeplabV3+ model through the mixed dataset to construct the trained DeeplabV3+ model specifically includes:

[0028] Input the mixed dataset into the DeeplabV3+ model to obtain the output results of semantic segmentation;

[0029] Perform Softmax normalization on the output results of semantic segmentation to obtain class probabilities;

[0030] Based on the class probabilities, return the class with the highest probability to generate pseudo-labels;

[0031] Filter the pseudo-labels, retain the classes with high confidence, and adjust the parameters of the DeeplabV3+ model to construct the trained DeeplabV3+ model.

[0032] Further, the step of performing slow traffic safety hazard analysis and visualization processing on the semantic segmentation results of urban slow traffic accidents to obtain the slow traffic safety hazard investigation results of the city specifically includes:

[0033] Obtain knowledge graph data, define the entity relationships between the knowledge graph data, and perform named entity recognition through natural language processing tools to construct a knowledge graph. The knowledge graph data includes hazard data, strategy data, and EPC project process data;

[0034] Perform slow traffic safety hazard analysis on the semantic segmentation results of urban slow traffic accidents through the knowledge graph to obtain the slow traffic accident safety hazard analysis results of the city;

[0035] Perform visualization processing on the slow traffic accident safety hazard analysis results to obtain the slow traffic safety hazard investigation results of the city.

[0036] The second technical solution adopted by the present invention is: an urban slow traffic safety hazard investigation system based on multi-modal data, including:

[0037] The first module is used to obtain traffic accident information and perform data preprocessing to obtain a structured traffic accident information table;

[0038] The second module is used to identify black spots at accident locations based on the structured traffic accident information table through a Gaussian kernel function to obtain street view pictures of accident high-frequency locations;

[0039] The third module is used to perform semantic segmentation on the street view pictures of accident high-frequency locations through the deeplabV3+ model to obtain the semantic segmentation results of urban slow traffic accidents;

[0040] The fourth module is used to analyze and visualize the hidden dangers of slow traffic safety for the semantic segmentation results of urban slow traffic accidents to obtain the results of the investigation of hidden dangers of urban slow traffic safety.

[0041] The beneficial effects of the method and system of the present invention are as follows: By obtaining traffic accident information and performing data preprocessing, the present invention can integrate accident text data, accident video data, and geospatial shp data, establish a multi-modal dataset, utilize artificial intelligence technology, reduce manual consumption, and improve the accuracy of hidden danger investigation. Further, based on the structured traffic accident information table, black spots are identified at accident locations through a Gaussian kernel function, and the existing accident data is used to investigate the hidden dangers of slow traffic safety, improving the efficiency of the investigation of hidden dangers of slow traffic safety. Potential risk factors within the research area can be identified, providing a reference basis for traffic accident prevention and control. Furthermore, semantic segmentation is performed on the street view pictures of accident high-frequency locations through the deeplabV3+ model, and finally, the semantic segmentation results of urban slow traffic accidents are analyzed and visualized for hidden dangers of slow traffic safety, and a knowledge graph is used for precise intelligent matching of slow traffic hidden danger problems and improvement strategies, improving traffic management efficiency and safety. Brief Description of the Drawings

[0042] Figure 1 is the flowchart of the steps of the method for investigating hidden dangers of urban slow traffic safety based on multi-modal data of the present invention;

[0043] Figure 2 is the structural block diagram of the system for investigating hidden dangers of urban slow traffic safety based on multi-modal data of the present invention;

[0044] Figure 3 is the schematic framework diagram of the investigation of hidden dangers of urban slow traffic safety provided by a specific embodiment of the present invention;

[0045] Figure 4It is a schematic diagram for determining black spots of road network accidents based on kernel density estimation provided by a specific embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of the DeeplabV3+ model and semi-supervised learning fine-tuning parameters provided by a specific embodiment of the present invention;

[0047] Figure 6 It is a schematic diagram of the knowledge graph matching process provided by a specific embodiment of the present invention;

[0048] Figure 7 It is a schematic diagram of the structure of the DeeplabV3+ model provided by a specific embodiment of the present invention. Specific Embodiments

[0049] The following further elaborates on the present invention in detail with reference to the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0050] Refer to Figure 1 The present invention provides a method for investigating potential safety hazards of urban slow traffic based on multi-modal data, and the method includes the following steps:

[0051] S100. Obtain traffic accident information and perform data preprocessing to obtain a structured traffic accident information table;

[0052] Specifically, traffic accident data is collected and processed through a multi-source heterogeneous data crawling framework to obtain traffic accident information, where the traffic accident information includes structured text data and unstructured video data; target keywords are extracted from the structured text data to construct a text accident database, and the target keywords include accident time, geographical coordinates, accident type, and accident liability cause; key frame data is intercepted from the unstructured video data to construct a video accident database, and the key frame data includes video shooting time, GPS coordinates, and user annotation information; the text accident database and the video accident database are subjected to spatio-temporal alignment and data fusion processing, and duplicate data is removed to obtain a standardized slow traffic accident data set; the standardized slow traffic accident data set is subjected to data cleaning and unification processing to obtain a structured traffic accident information table.

[0053] In this embodiment, accident information is obtained from media such as Weibo, official accounts, and news using web crawler technology, including: road network accident data, geographical location information, road environment data, surveillance videos, etc.; and the obtained data is preprocessed, including: classifying the obtained data by accident type and recording the time, location, and cause to form an accident case data set.

[0054] It should also be noted that in some specific embodiments, first, a multi-source heterogeneous data crawling framework is constructed. For the public databases of traffic management departments, social media platforms, and video sharing websites, targeted crawler algorithms are designed respectively. The crawler algorithms are based on the Scrapy framework to implement dynamic web page parsing, bypass anti-crawling mechanisms by simulating user requests, and real-time capture accident data related to non-motor vehicles and pedestrians, including structured text data and unstructured video data; further, keyword extraction is performed on the text data respectively to extract four core fields: accident time, geographical coordinates, accident type, and responsibility cause, and a text accident database TT is established, and data extraction is performed on the video data, key frames of the accident video are intercepted, and the video shooting time, GPS coordinates, and user annotation information are extracted to analyze the dynamic process of the accident and generate a video accident database VV; then, spatio-temporal alignment and data fusion are performed on the text accident database TT and the video accident database VV, the text and video records of the same accident are associated, duplicate data is removed, and finally a standardized slow traffic accident dataset SS is formed; finally, data cleaning and standardization are performed on the dataset SS, including coordinate deviation correction, time format unification, and accident type coding, and a structured accident information table is output.

[0055] S200. Based on the structured traffic accident information table, black spot identification is performed on the accident location through a Gaussian kernel function to obtain street view pictures of high-frequency accident locations;

[0056] Specifically, the traffic accident text address is obtained according to the structured traffic accident information table and converted into longitude and latitude coordinates to obtain traffic accident point coordinate information; spatial density interpolation processing is performed on the traffic accident point coordinate information through a Gaussian kernel function to obtain continuous kernel density raster data; the administrative division vector map of the target city is obtained and spatially overlaid with the continuous kernel density raster data to obtain a fused target city raster map; the density values of the fused target city raster map are classified by the natural break method, a multi-level gradient heat map is generated and marking elements are added to obtain street view pictures of high-frequency accident locations.

[0057] In this embodiment, as Figure 4As shown in the figure, to obtain the accident black spots on the urban road network, the kernel density estimation of ArcGIS is mainly used to identify the black spots at the accident locations. First, the text addresses of non-motor vehicle accidents in a certain city crawled from the network are processed for conversion to longitude and latitude. The HTML code is used to call a certain map Web service API to batch complete the conversion of text addresses to longitude and latitude: send a structured address request to the API, parse the returned JSON data to extract coordinate information, and further perform kernel density estimation through spatial interpolation. Spatial interpolation processing is performed on the longitude and latitude data of non-motor vehicle accident points in a certain city after geocoding conversion (i.e., accident point coordinate information) to obtain continuous density raster data, which is used to reflect the spatial aggregation characteristics of non-motor vehicle accidents. Among them, the expression of the Gaussian kernel function is specifically as follows:

[0058]

[0059] In the above formula, f(x) represents the spatial density interpolation of the Gaussian kernel function, K represents the kernel function, h represents the fixed search radius, d(x,x i ) represents the distance between point x and the accident point x i , and b represents the number of accident locations.

[0060] Through the "Kernel Density" tool in ArcGIS, combined with the system-defined cell size, a continuous density raster is generated to reflect the accident spatial aggregation characteristics. Then, the kernel density raster is spatially overlaid with the administrative division vector map of a certain city to achieve the fusion of multiple layers. The natural breakpoint method (Jenks Natural Breaks) is used to classify the density values to generate a multi-level gradient heat map (dark colors represent high-risk areas), and the road network data can be overlaid to enhance the geographical semantic expression. At the same time, elements such as a legend, scale, and north arrow are added to improve the visualization effect. Finally, maps in PDF and JPG formats with a resolution not lower than 300 dpi are output, supporting A4 format printing. By comparing the historical accident data with the distribution characteristics of the heat map, the rationality of the kernel density analysis is verified to ensure that the results conform to the actual risk law.

[0061] S300. Semantic segmentation is performed on the street view images of accident high-frequency locations through the deeplabV3+ model to obtain the semantic segmentation results of urban slow traffic accidents;

[0062] Specifically, the Cityscapes dataset and several street view images are obtained and mixed to construct a mixed dataset; the semi-supervised learning method with pseudo-labels is used to iteratively train the deeplabV3+ model through the mixed dataset to construct the trained deeplabV3+ model; based on the encoder and decoder in the trained deeplabV3+ model, semantic segmentation is performed on the street view images of accident high-frequency locations to obtain the semantic segmentation results of urban slow traffic accidents.

[0063] In this embodiment, as Figure 7 shown, download the Cityscapes dataset and the open-source pre-trained weights of the Deeplabv3+ model, select Resnet as the backbone network, and use the deeplabv3plus_resnet101 model to perform the initial training on the cityscapes dataset to obtain an initial model. Due to the differences in urban street scenes, call a certain map API to collect a thousand street view images, establish a street view map set of a certain city for the annotation of pseudo-labels and model training later, and mix it with the Cityscapes dataset to form a mixed dataset, enhancing the generality and accuracy of the model.

[0064] Among them, for the semi-supervised learning method using pseudo-labels, in the step of iteratively training the deeplabV3+ model through the mixed dataset to construct the trained deeplabV3+ model, it also includes inputting the mixed dataset into the deeplabV3+ model to obtain the output result of semantic segmentation; performing Softmax normalization on the output result of semantic segmentation to obtain class probabilities; returning the class with the highest probability based on the class probabilities to generate pseudo-labels; filtering the pseudo-labels, retaining the classes with high confidence, and adjusting the parameters of the deeplabV3+ model to construct the trained deeplabV3+ model.

[0065] In some specific embodiments, as Figure 5 shown, the semi-supervised learning method using pseudo-labels is used to perform iterative training on the model.

[0066] First, input the labels into the model to obtain an output result of semantic segmentation, and its expression is:

[0067] Z u = f(X u )

[0068] In the above formula, Z u represents the logits(b, c, w, h) output by the model, and X u represents the input unlabeled image.

[0069] Perform Softmax normalization on the output result to obtain class probabilities, and its expression is:

[0070] P u = Softmax(Z u )

[0071]

[0072] In the above formula, P uRepresents the predicted probability distribution, where i represents the batch index, j and k represent the spatial location indices, c represents the class index, and C represents the number of classes.

[0073] Further returns the class with the highest probability, and its expression is:

[0074] Y u = argmax(P u , dim = 1)

[0075] In the above formula, Y u represents the generated pseudo-label, shown as a class.

[0076] Finally, filter the pseudo-labels, retain the classes with high confidence, avoid interference from input noise, and reduce the accuracy of the model. The expression of the process is:

[0077]

[0078] In the above formula, v represents the confidence threshold, taken as 0.9.

[0079] Fuse the pseudo-labels with the Cityscapes dataset, input the model for iterative training to obtain a new model. Fine-tune the parameters of the iterated model, including adjusting the learning rate to avoid overfitting of the model; and expanding the range of light brightness changes in data augmentation after expanding the data input to adapt to the situation where the brightness difference of the street view map obtained by a certain map call is large. Train the model with the adjusted parameters to obtain a model with higher final semantic segmentation accuracy.

[0080] S400. Conduct slow traffic safety hazard analysis and visualization processing on the semantic segmentation results of urban slow traffic accidents to obtain the results of urban slow traffic safety hazard investigation.

[0081] S410. Obtain knowledge graph data, define the entity relationships between the knowledge graph data, perform named entity recognition through natural language processing tools, and construct a knowledge graph. The knowledge graph data includes hazard data, policy data, and EPC project process data;

[0082] S420. Conduct slow traffic safety hazard analysis on the semantic segmentation results of urban slow traffic accidents through the knowledge graph to obtain the results of urban slow traffic accident safety hazard analysis;

[0083] In this embodiment, as Figure 6As shown in the figure, first is data collection, including potential hazard data, obtaining potential hazard information (such as road cracks, signal failures, missing construction enclosures, etc.) from channels such as traffic management departments, accident reports, online platforms, and public feedback. It also includes strategy data: extracting optimization strategies (such as "install additional traffic signals", "optimize construction processes", "deploy intelligent monitoring systems", etc.) from knowledge bases, industry standards, and historical project cases, as well as EPC project process data: referring to EPC project process documents in existing knowledge bases and extracting process links related to potential hazards (such as construction management, acceptance criteria, etc.).

[0084] Secondly is to define entity types, including potential hazard entities such as road design defects, traffic signal failures, construction management loopholes, facility aging, etc., and strategy entities such as engineering and technical measures (such as strengthening roads), management process optimizations (such as improving construction approval processes), and intelligent monitoring systems.

[0085] Furthermore is to define relationship types, including "causes" (such as "construction management loopholes cause road cracks"); "applies to" (such as "installing additional signal lights applies to accident hazards at intersections"); "belongs to" (such as "construction management loopholes belong to problems in the construction stage of the EPC project process").

[0086] Finally, use natural language processing (NLP) tools (such as BERT, SimCSE) to perform named entity recognition (NER) on the entities in the text data, eliminate ambiguities (such as "construction delays" and "construction quality defects"), and unify entity names based on industry glossaries (such as traffic engineering specifications) (such as standardizing "signal light failure" to "traffic signal failure"), and establish associations between entities through rules.

[0087] S430. Visualize the analysis results of potential hazards in urban slow traffic accidents to obtain the results of the investigation of potential hazards in urban slow traffic safety.

[0088] In this embodiment, a web page is made to generate an open-source platform to visually display the results of the investigation of potential hazards in slow traffic safety.

[0089] In some specific embodiments, first is the system architecture design. Based on the B / S mode, construct the overall system architecture, design the hierarchical structures of the front end, back end, and database, determine the interaction methods and data flow processes between each layer, and plan the distribution of system modules to ensure that the system has good scalability and maintainability to meet the needs of future business growth and function expansion. And database design. According to data types and business requirements, design the table structure, fields, data types, indexes, etc. of the database, covering road network accident data, geographical location information, road environment data, surveillance videos, etc.; establish the association relationships between data, optimize the database performance, and ensure the efficient storage and fast query of data.

[0090] Furthermore, page design and layout are carried out. The front-end pages are designed using HTML and CSS technologies, including the main pages such as the home page, map introduction, data sharing page, login / registration, as well as the sub-pages and pop-ups of each page. And responsive design is adopted to ensure good display on different terminal devices (such as computers, tablets, mobile phones). Also, interactive functions are implemented. With the help of JavaScript and its related framework Vue.js, interactive functions such as user login and registration, map zooming and panning, video playback control are realized on the front-end pages, enhancing the user experience and enabling users to conveniently and quickly obtain and operate the required information.

[0091] Then, the server environment is set up. The server operating system and Web server software are selected for the installation and configuration of the server environment to ensure the stable operation of the server, with good performance and security, and capable of bearing the access traffic and data processing tasks of the system.

[0092] The back-end framework is built. The mature back-end development framework Django is used to build the basic framework of the system back-end, providing basic support and unified development specifications for the development of system function modules.

[0093] Data interface development is carried out. According to the front-end requirements and data interaction rules, the back-end data interfaces are developed to implement functions such as data reception, processing, storage, and return. The interface design should follow the RESTful style or other agreed-upon specifications to ensure the accuracy and efficiency of data transmission. At the same time, security protection is carried out on the interfaces, such as parameter verification, identity authentication, prevention of injection attacks, etc., to prevent illegal access and data leakage.

[0094] Business logic implementation is carried out. Based on the system function requirements, various business logics are implemented in the back-end development to ensure that the system can operate according to the predetermined business processes and rules, meeting the users' analysis and application requirements for slow traffic data.

[0095] Finally, system integration and testing are carried out. System integration: The developed front-end and back-end are integrated to ensure correct interface docking between the two, smooth data interaction, and the ability of each function module to work together to form a complete system. At the same time, the system is deployed to the selected server environment, connected and configured with the database, so that the system can be normally started and run in the actual operating environment. Function testing: All functions of the system are comprehensively tested, including user login and registration, data collection and upload, data visualization display, data sharing, etc., to check whether the functions meet the requirements of the requirements specification, whether there are function defects or abnormal situations, and the discovered problems are repaired in a timely manner to ensure the integrity and accuracy of the system functions.

[0096] In summary, as Figure 3As shown in the figure, the embodiment of the present invention uses a web crawler to obtain multi-modal data such as accident data, geographical location information, road environment data, surveillance videos, and street view maps; preprocesses the obtained data, classifies accidents into accidental accidents and illegal acts, classifies accident types into those related to non-motor vehicles or pedestrians, and records the time, location, and cause; uses the kernel density estimation of ArcGIS to identify black spots at accident locations to obtain high-frequency accident locations; uses the deeplabV3+ model to perform semantic segmentation of traffic infrastructure, buildings, and traffic environment on street view pictures of high-frequency accident locations for safety hazard investigation; analyzes the segmentation results, gives detailed existing hazards and solutions, and uses a knowledge graph to match accident sets and countermeasure sets for automated safety hazard analysis and countermeasure generation; makes a web page to visually display the results obtained after processing. The embodiment of the present invention can investigate the intuitive hazards existing in urban roads based on accident black spots, reduce manual consumption, and facilitate the traffic management department to optimize urban traffic.

[0097] Refer to Figure 2 , a system for investigating safety hazards of urban slow traffic based on multi-modal data, includes:

[0098] The first module 201 is used to obtain traffic accident information and perform data preprocessing to obtain a structured traffic accident information table;

[0099] The second module 202 is used to identify black spots at accident locations based on the structured traffic accident information table through a Gaussian kernel function to obtain street view pictures of high-frequency accident locations;

[0100] The third module 203 is used to perform semantic segmentation on street view pictures of high-frequency accident locations through the deeplabV3+ model to obtain a semantic segmentation result of urban slow traffic accidents;

[0101] The fourth module 204 is used to perform slow traffic safety hazard analysis and visualization processing on the semantic segmentation result of urban slow traffic accidents to obtain a result of investigating safety hazards of urban slow traffic.

[0102] The content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented by the system embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0103] The above is a specific description of the preferred embodiments of the present invention. However, the present invention is not limited to the described embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for investigating potential safety hazards of urban slow traffic based on multi-modal data, characterized in that The following steps are involved: Obtain traffic accident information and perform data preprocessing to obtain a structured traffic accident information table; Based on the structured traffic accident information table, the black spots of the accident sites are identified by Gaussian kernel function to obtain street view images of accident high-frequency locations. The deeplabV3+ model is used to perform semantic segmentation on street view images of accident-prone locations, and the semantic segmentation results of urban slow-moving traffic accidents are obtained; The semantic segmentation results of urban slow traffic accidents are used to analyze and visualize the safety hazards of slow traffic, and the results of the urban slow traffic safety hazard investigation are obtained.

2. The method for investigating potential safety hazards of urban slow traffic based on multi-modal data according to claim 1, wherein The step of obtaining traffic accident information and performing data preprocessing to obtain a structured traffic accident information table specifically includes: Traffic accident data is collected and processed through a multi-source heterogeneous data crawling framework to obtain traffic accident information, wherein the traffic accident information includes structured text data and unstructured video data; Extract target keywords from structured text data to build a text accident database, where the target keywords include accident time, geographic coordinates, accident type and accident responsibility cause; Perform key frame data interception processing on unstructured video data to build a video accident database. The key frame data includes video shooting time, GPS coordinates and user annotation information; Perform spatiotemporal alignment and data fusion processing on the text accident database and the video accident database, and remove duplicate data to obtain a standardized slow-moving traffic accident dataset; The standardized slow-moving traffic accident data set is cleaned and unified to obtain a structured traffic accident information table.

3. The method for investigating potential safety hazards of urban slow traffic based on multi-modal data according to claim 2, wherein The step of identifying black spots at the accident site by using a Gaussian kernel function based on the structured traffic accident information table to obtain street view images of accident high-frequency sites specifically includes: Obtain the traffic accident text address according to the structured traffic accident information table, and convert it into longitude and latitude to obtain the coordinate information of the traffic accident point; The coordinate information of traffic accident points is processed by spatial density interpolation through Gaussian kernel function to obtain continuous kernel density grid data; Obtain the administrative division vector map of the target city and spatially overlay it with the continuous kernel density raster data to obtain the fused raster map of the target city; The density value of the fused target city raster map is graded through the natural breakpoint method, a multi-level gradient heat map is generated, and marker elements are added to obtain street view images of accident-high-frequency locations.

4. The method for investigating potential safety hazards of urban slow traffic based on multi-modal data according to claim 3, wherein, The expression of the Gaussian kernel function is specifically as follows: In the above formula, f(x) represents the spatial density interpolation of the Gaussian kernel function, K represents the kernel function, g represents the fixed search radius, d(x, x i ) represents the distance between point x and the accident point x i , and n represents the number of accident locations.

5. The method for investigating potential traffic safety hazards of urban slow traffic based on multi-modal data according to claim 4, characterized in that, The step of performing semantic segmentation on street view images of accident high-frequency locations by using the deeplabV3+ model to obtain semantic segmentation results of urban slow traffic accidents specifically includes: Obtain the Cityscapes dataset and several street view images and mix the data to construct a mixed dataset; The pseudo-labeled semi-supervised learning method is used to iteratively train the deeplabV3+ model through a mixed data set to construct a trained deeplabV3+ model; Based on the encoder and decoder in the trained deeplabV3+ model, semantic segmentation is performed on street view images of accident-frequently occurring locations to obtain the semantic segmentation results of urban slow-moving traffic accidents.

6. The method for investigating potential traffic safety hazards of urban slow traffic based on multi-mode data according to claim 5, wherein The step of using the semi-supervised learning method with pseudo-labels to iteratively train the DeeplabV3+ model through a mixed dataset and construct the trained DeeplabV3+ model specifically includes: Input the mixed dataset into the DeeplabV3+ model to obtain the output result of semantic segmentation; Perform Softmax normalization on the output result of semantic segmentation to obtain class probabilities; Based on the class probabilities, return the class with the highest probability to generate pseudo-labels; Filter the pseudo-labels, retain the classes with high confidence, adjust the parameters of the DeeplabV3+ model, and construct the trained DeeplabV3+ model.

7. The method for investigating potential traffic safety hazards of urban slow traffic based on multi-modal data according to claim 6, wherein, The step of performing slow traffic safety hazard analysis and visualization processing on the semantic segmentation result of urban slow traffic accidents to obtain the urban slow traffic safety hazard investigation result specifically includes: Obtain knowledge graph data, define the entity relationships between the knowledge graph data, perform named entity recognition through natural language processing tools, and construct a knowledge graph. The knowledge graph data includes hazard data, strategy data, and EPC project process data; Perform slow traffic safety hazard analysis on the semantic segmentation result of urban slow traffic accidents through the knowledge graph to obtain the urban slow traffic accident safety hazard analysis result; Perform visualization processing on the urban slow traffic accident safety hazard analysis result to obtain the urban slow traffic safety hazard investigation result.

8. The urban slow traffic safety hazard investigation system based on multi-mode data is characterized in that It includes the following modules: The first module is used to obtain traffic accident information and perform data preprocessing to obtain a structured traffic accident information table; The second module is used to identify black spots at accident locations through the Gaussian kernel function based on the structured traffic accident information table to obtain street view pictures of accident-prone locations; The third module is used to perform semantic segmentation on the street view pictures of accident-prone locations through the DeeplabV3+ model to obtain the semantic segmentation result of urban slow traffic accidents; The fourth module is used to perform slow traffic safety hazard analysis and visualization processing on the semantic segmentation result of urban slow traffic accidents to obtain the urban slow traffic safety hazard investigation result.

Images