Driving information processing method and device based on intelligent helmet

Abstract

The invention relates to the field of driving data processing, in particular to a driving information processing method and device based on an intelligent helmet, and the method comprises the following steps: obtaining a real-time driving process monitoring video and user driving posture data based on the intelligent helmet; performing histogram equalization processing on the real-time driving process monitoring video, and performing global time delay interpolation smoothing processing to obtain a time delay smooth monitoring video; deep road semantic feature segmentation is carried out on the time delay smooth monitoring video, scene space structure modeling is carried out, and a dynamic twin scene model is constructed; current scene driving track prediction is carried out on the user driving posture data, safe passing path planning is carried out, and a scene safe passing path is generated; and marking dynamic vehicles in all scenes based on the dynamic twin scene model. According to the invention, efficient driving information processing is realized, and the driving safety of a driver is improved.

Description

Technical Field

[0001] The present invention relates to the field of driving data processing, and in particular to a driving information processing method and device based on a smart helmet. Background Art

[0002] With the rapid development of intelligent transportation technology, driving safety has become a focal issue of social concern. Traditional driving assistance systems mostly rely on on-board sensors and monitoring equipment, which have defects such as limited information collection range and untimely fault detection, making it difficult to fully perceive the complex and changeable driving environment. In recent years, the driving information processing method based on wearable devices has attracted widespread attention. As an intelligent wearable device, the driving smart helmet can integrate a variety of high-definition cameras, IMU sensors, etc., to collect the driver's physiological state, driving behavior and various indicators of the surrounding environment in real time, providing a strong guarantee for improving driving safety.

[0003] How to effectively process and analyze the massive amount of driving information obtained from smart helmets and to dig out potential safety hazards and warning information has become a key issue that needs to be solved urgently. Traditional information processing methods often rely on manual analysis and experience judgment, which are inefficient and difficult to cope with the ever-changing driving environment and cannot improve the driving safety of drivers. Therefore, it is urgent to establish a driving information processing method based on smart helmets, using artificial intelligence technologies such as computer vision and pattern recognition to achieve real-time perception, dynamic analysis and risk prediction of the driving environment, thereby providing drivers with timely and accurate safety decision support. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a driving information processing method and device based on a smart helmet to solve at least one of the above technical problems.

[0005] To achieve the above object, the present invention provides a method for processing driving information based on a smart helmet, comprising the following steps:

[0006] Step S1: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet; performing histogram equalization processing on the real-time driving process monitoring video, and performing global delay interpolation smoothing processing to obtain a delay smoothed monitoring video;

[0007] Step S2: perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, perform scene spatial structure modeling, and construct a dynamic twin scene model;

[0008] Step S3: predict the current scene driving trajectory based on the user's driving posture data, and plan a safe passage path to generate a scene safe passage path;

[0009] Step S4: Marking the dynamic vehicles in all scenes based on the dynamic twin scene model; predicting the dynamic movement trajectory of the dynamic vehicles in all scenes, thereby generating a multi-period corrected driving prediction trajectory;

[0010] Step S5: Calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, perform safety risk analysis, and mark it as a high-risk vehicle;

[0011] Step S6: Make risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory and generate risk decision results; project the risk decision results onto the smart helmet to complete the driving information processing operation.

[0012] The present invention enhances the contrast and brightness of video images through histogram equalization processing, improves image quality, and makes monitoring videos clearer. Global time delay interpolation smoothing processing reduces jitter and discontinuity in the video, improves user viewing experience, and ensures video fluency. Deep road semantic feature segmentation and scene spatial structure modeling accurately understand road conditions and scene structures, provide accurate scene information for subsequent safe passage path planning, construct a dynamic twin scene model to capture and reflect the dynamic changes of road scenes in real time, improve the driving system's ability to understand and predict scenes, and use driving trajectory prediction and safe passage path planning to predict the user's driving trajectory in advance and plan a safe path, improve driving safety and efficiency, generate a scene safe passage path to avoid potential dangers, and provide users with safe driving. Guidance, dynamic mobile trajectory prediction and multi-period corrected driving prediction trajectory accurately predict the movement trajectory of vehicles in the scene, improve the system's understanding and prediction ability of vehicle behavior, generate corrected driving prediction trajectory to optimize driving path, improve driving efficiency and safety, collision probability calculation and safety risk analysis to identify high-risk vehicles, discover potential collision risks in advance, reduce the occurrence of traffic accidents, mark high-risk vehicles to warn drivers, improve drivers' awareness of potential dangers, enhance safety awareness, risk avoidance decisions and risk decision results are projected onto smart helmets to provide real-time risk prompts and decision support, help drivers make correct decisions, complete driving information processing operations to optimize drivers' driving experience, improve driving safety and comfort, and realize intelligent driving assistance functions.

[0013] Preferably, step S1 comprises the following steps:

[0014] Step S11: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet;

[0015] Step S12: performing brightness deviation detection on the real-time driving process monitoring video to identify the brightness deviation area;

[0016] Step S13: performing histogram equalization processing based on the brightness deviation area to generate a brightness enhanced monitoring video;

[0017] Step S14: performing driving scene edge analysis on the brightness enhanced monitoring video to extract driving scene edge data;

[0018] Step S15: optimizing edge details of the brightness enhanced monitoring video according to the edge data of the travel scene, and constructing an edge optimized monitoring video;

[0019] Step S16: Perform global delay interpolation smoothing processing on the edge optimized monitoring video to obtain a delay smoothed monitoring video.

[0020] The present invention provides basic data for subsequent driving information processing by acquiring driving monitoring videos and user driving posture data in real time. The user driving posture data can be used to analyze user behavior and help improve driving behavior and safety. Brightness deviation detection identifies abnormal brightness in the video and improves image quality. Identifying brightness deviation areas can provide target areas for subsequent image enhancement and optimization. Histogram equalization processing improves image contrast and brightness balance and enhances the visual effect of monitoring videos. Generating brightness enhanced monitoring videos can improve video quality and make details more clearly visible. Driving scene edge analysis identifies important edge information in driving scenes and improves the accuracy of image understanding and analysis. Extracting travel scene edge data can provide important clues for subsequent image processing and optimization. Edge detail optimization can highlight important edge information in the video and improve the clarity and visual effect of the image. Constructing edge optimized monitoring videos can make images more recognizable and ornamental. Global time delay interpolation smoothing processing reduces jitter and discontinuity in the video and improves the viewing experience of the video. Obtaining time delay smoothed monitoring videos can make the video smoother and reduce the discomfort of users when watching.

[0021] Preferably, the specific steps of step S16 are:

[0022] Perform time-series window segmentation on edge-optimized monitoring video and extract the time-series image of each frame;

[0023] Calculating the timestamp of each frame of the time-series image;

[0024] Calculate the time interval between the upper and lower frames according to the timestamp to obtain the inter-frame interval duration of all image frames;

[0025] Fitting the interval duration distribution of all image frames to obtain a global interval distribution map of the video;

[0026] Perform interval discrete unevenness identification on the global interval distribution map of the video and extract the points with uneven interval duration;

[0027] Positioning the time-series image frames based on the unevenly spaced time points, extracting all the unevenly spaced image frames;

[0028] Calculating the inter-frame delay of all non-uniformly spaced image frames, thereby obtaining the delay parameters of the non-uniformly spaced frames;

[0029] The average inter-frame interval duration of all image frames is counted to generate a mean inter-frame interval value;

[0030] Performing delay compensation calculation on the delay parameters of the non-uniformly spaced frames according to the mean inter-frame interval value, thereby obtaining the delay compensation parameters of the non-uniformly spaced frames;

[0031] Performing time-series serialization processing on each frame of the time-series image according to the timestamp to obtain a time-series image frame sequence;

[0032] The delay compensation parameters of non-uniformly spaced frames are used to perform delay interpolation and smoothing processing on the time-series image frame sequence, thereby obtaining a delay-smoothed monitoring video.

[0033] The present invention performs precise time management and analysis of videos by extracting time-series images of each frame and calculating timestamps. Calculating the interval duration between frames and performing distribution fitting can reveal the time relationship between frames in the video, understand the time distribution law between video frames, and identify and extract points with uneven interval durations, which can help discover time anomalies in the video, further optimize the time series of the video, calculate the delay parameters of unevenly spaced frames to understand the changes in the time interval between video frames, and provide a basis for subsequent delay compensation. Generating mean inter-frame interval values ​​and calculating delay compensation parameters can count and adjust the time interval of the video, improve the time fluency and continuity of the video, and time series serialization processing and delay interpolation smoothing can effectively process the delay of unevenly spaced frames, eliminate the screen freeze and discontinuity caused by uneven intervals during video playback, and improve the smoothness of the video.

[0034] Preferably, step S2 specifically comprises the following steps:

[0035] Step S21: performing visual recognition of scene objects on the time-delay smoothed monitoring video, marking objects in the driving scene, wherein the objects in the driving scene include static road signs, road obstacles and dynamic vehicles;

[0036] Step S22: performing deep road semantic feature segmentation on the time-delay smoothed monitoring video to generate road semantic features of the driving scene;

[0037] Step S23: Perform scene spatial structure analysis on static road signs and road obstacles to generate scene static spatial structure data;

[0038] Step S24: performing scene spatial structure modeling on the semantic features of the driving scene road and the scene static spatial structure data to construct a three-dimensional driving scene model;

[0039] Step S25: Perform dynamic movement evolution simulation on the three-dimensional driving scene model according to the dynamic vehicle to construct a dynamic twin scene model.

[0040] The present invention improves driving safety by identifying static road signs, road obstacles and dynamic vehicle drivers' real-time understanding of the driving environment. Marking these objects can enhance the readability of video content and make the information clearer. It generates road semantic features of driving scenes to understand key information such as road conditions and vehicle positions, improves the ability to recognize and analyze driving scenes, performs spatial structural analysis on static road signs and road obstacles to identify potential safety hazards and optimize driving path planning, improves the driver's perception of the driving environment, and constructs a three-dimensional driving scene model to provide a more realistic and accurate environmental simulation for smart helmets, enhance the global cognition and decision-making ability of the driving monitoring system, and simulates the real-time changes of vehicles in driving scenes through dynamic mobile evolution simulation, helping smart helmets to better predict and respond to traffic conditions, and improve the driver's sense of security and driving efficiency.

[0041] Preferably, step S3 specifically comprises the following steps:

[0042] Step S31: Calculate the driving space volume based on the user's driving posture data to generate the user's standard driving volume;

[0043] Step S32: Analyze the user's driving trajectory on the time-delay smoothed monitoring video to extract the user's time-series driving trajectory;

[0044] Step S33: predicting the current scene driving trajectory of the user's time series driving trajectory based on the user's standard driving volume, and generating a predicted driving trajectory in the scene;

[0045] Step S34: Calculate the obstacle area volume of the dynamic twin scene model to generate obstacle area volume parameters;

[0046] Step S35: Plan a safe passage path for the predicted driving trajectory in the scene based on the volume parameters of the obstacle area to generate a safe passage path for the scene.

[0047] The present invention can accurately evaluate the size and space requirements of the user's vehicle through driving space volume calculation, providing a basis for subsequent driving trajectory analysis and path planning. The generated user standard driving volume understands the size and shape of the vehicle, and provides more accurate information for the smart helmet to support driving decisions. Through driving trajectory analysis, the user's trajectory changes during driving can be captured, including the vehicle's moving path and speed changes. The extracted time-series driving trajectory data provides important information for subsequent driving trajectory prediction and safe path planning, helping the smart helmet to better understand the user's driving behavior. The driving trajectory prediction can predict the user's future according to the user's standard driving volume and historical driving trajectory. The smart helmet can identify potential driving conflicts and dangerous situations in advance based on the future driving path and behavior, and generate the predicted driving trajectory in the scene, so as to improve driving safety. The obstacle area volume calculation can help identify the size and spatial range of obstacles in the scene, and provide the key parameters required for safe path planning. The generated obstacle area volume parameters provide a reference for obstacle avoidance and safe passage for driving path planning. The safe passage path planning combines the user's driving trajectory prediction and obstacle area volume parameters to ensure that the smart helmet provides a safe driving path for the user. The generated scene safe passage path can avoid potential collisions and dangerous situations, and improve driving safety and stability.

[0048] Preferably, the specific steps of step S35 are:

[0049] Locate obstacles in the dynamic twin scene model and mark the obstacle coordinates;

[0050] The encounter time point of the predicted driving trajectory in the scene is calculated according to the obstacle position coordinates to obtain the obstacle encounter time point of the predicted trajectory;

[0051] The obstacle area volume parameters are expanded to generate the obstacle safety space bounding volume;

[0052] Based on the obstacle encounter time point of the predicted trajectory, the potential collision area of ​​the obstacle safety space is analyzed to generate the user's potential collision area;

[0053] Perform normal traffic inference on the user's potential collision area and generate user traffic inference results;

[0054] When the user's estimated result is that the vehicle can pass normally, the predicted driving trajectory in the scene is used as the safe passage path of the scene;

[0055] When the user's passage speculation result is that he cannot pass normally, a safe passage path is planned for the user's potential collision area to generate a scene-based safe passage path.

[0056] The present invention can provide accurate basic information for subsequent path planning and conflict avoidance by accurately locating and marking the coordinates of the obstacle position. The smart helmet can timely perceive the surrounding environment by marking the obstacle position in real time, thereby improving driving safety. By calculating the encounter time point of the predicted trajectory and the obstacle, the potential collision risk can be evaluated, and avoidance measures can be taken in advance. The smart helmet can plan the driving strategy at the encounter time point to avoid collision with the obstacle, expand the safe space volume of the obstacle to create a more spacious driving environment, reduce the potential collision risk, and generate a safe space enclosure to provide a more comprehensive collision avoidance solution for the smart helmet to ensure driving safety. Through potential collision area analysis, the collision risk faced by the user can be determined, and corresponding countermeasures can be formulated. The smart helmet can generate a potential collision area to give early warning and avoid potential collision hazards to ensure driving safety. Through normal traffic speculation, the driving situation of the user in the potential collision area can be evaluated to determine whether there is a safe passage path. The traffic speculation result can be generated, and the smart helmet can make real-time decisions to ensure the safe passage of the user. Through safe passage path planning, the driving route can be adjusted according to the actual situation to ensure the safe driving of the user in a complex environment. The smart helmet can adjust the driving strategy in real time according to the safe passage path planning result to minimize the collision risk.

[0057] Preferably, the specific steps of step S4 are:

[0058] Step S41: Perform dynamic vehicle optical flow recognition on the dynamic twin scene model and mark all dynamic vehicles in the scene;

[0059] Step S42: Calculate the optical flow displacement of all dynamic vehicles in the scene one by one to generate the vehicle moving speed;

[0060] Step S43: Analyze the moving directions of all dynamic vehicles in the scene and generate the moving directions of the vehicles;

[0061] Step S44: dynamically predicting the vehicle moving speed and direction, and generating predicted driving trajectories of all vehicles in the scene;

[0062] Step S45: performing road constraint correction on the predicted driving trajectories of all vehicles in the scene based on the scene static spatial structure data to generate a corrected predicted driving trajectory;

[0063] Step S46: Perform multi-period collaborative analysis on the corrected driving prediction trajectory to generate a multi-period corrected driving prediction trajectory.

[0064] The present invention can accurately detect and mark moving vehicles in the scene through dynamic vehicle optical flow recognition, provide basic data for subsequent vehicle movement speed and direction analysis, mark dynamic vehicle smart helmets to better perceive the surrounding traffic conditions, improve the driver's perception of the surrounding environment, one by one optical flow displacement calculation can accurately measure the movement speed of each vehicle, and provide important data for subsequent vehicle behavior analysis and prediction. The generated vehicle movement speed information smart helmet monitors traffic flow in real time and provides corresponding driving suggestions. The vehicle movement direction analysis can reveal the driving direction of each vehicle, helping the smart helmet to better understand the relative motion relationship between vehicles. The generated vehicle movement direction information smart helmet predicts traffic conflicts and takes corresponding avoidance measures. The dynamic movement trajectory prediction is combined with the vehicle The vehicle speed and direction can accurately predict the vehicle's future driving path, helping the smart helmet to make more accurate driving predictions. The generated driving prediction trajectory smart helmet plans a safe path and provides timely driving suggestions to improve the driver's driving safety. Road constraint correction can adjust the driving prediction trajectory according to the road structure information to ensure that the generated trajectory conforms to the actual road rules and conditions. The corrected driving prediction trajectory smart helmet can better adapt to the road environment and improve driving safety and traffic efficiency. Multi-period collaborative analysis can comprehensively consider the driving prediction trajectory in different time periods to provide more comprehensive and robust driving path planning. The generated multi-period corrected driving prediction trajectory smart helmet can better adapt to changes in different traffic conditions and time periods and provide more reliable driving suggestions.

[0065] Preferably, the specific steps of step S5 are:

[0066] Step S51: matching the meeting time of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, and marking the meeting section time point;

[0067] Step S52: cutting the dynamic twin scene model into time window areas according to the time point of the encounter section, and extracting the intersection area model;

[0068] Step S53: Calculate the collision probability of the intersection area model to generate driving collision probability data;

[0069] Step S54: Performing safety risk analysis on the vehicle collision probability data to generate a safety risk value for each vehicle;

[0070] Step S55: risk identification is performed on the safety risk value of each vehicle based on a preset driving safety risk threshold. When the preset driving safety risk threshold is greater than the safety risk value of the vehicle, it is marked as a high-risk vehicle.

[0071] The present invention can identify potential vehicle encounter situations through meeting time matching, help smart helmets predict the behavior of other vehicles during driving, mark the time point of the encounter section, and enable smart helmets to better grasp the relative positions and relationships between vehicles, thereby improving driving safety. Time window area cutting can divide the scene into different time periods and spatial areas, which is convenient for fine processing of the encounter situation between vehicles. The smart helmet that extracts the intersection area model can more accurately capture the area where collisions occur between vehicles, providing a basis for subsequent collision probability calculations. The collision probability calculation can quantify the probability of collisions between vehicles, and provide an objective assessment of driving safety. The generated driving collision probability data can be used by smart helmets to predict potential collision risks and take necessary safety measures in advance. The safety risk analysis can comprehensively consider the collision probability of the vehicle and other factors, and generate a corresponding safety risk value for each vehicle. Through the generation of the safety risk value, the smart helmet can more accurately assess the driving safety status of each vehicle, and take targeted safety measures. Risk identification can quickly mark potential high-risk vehicles by comparing the safety risk value of the vehicle with a preset threshold, and the helmet user can take timely action. Marking high-risk vehicles can remind drivers of dangerous situations, reduce the probability of potential accidents, and enhance driving safety.

[0072] Preferably, the specific steps of step S6 are:

[0073] Step S61: highlighting the risky vehicle trajectory of high-risk vehicles according to the multi-period corrected driving prediction trajectory, and extracting the risky vehicle driving prediction trajectory;

[0074] Step S62: making a risk avoidance decision on the predicted driving trajectory of the risky vehicle and generating a risk decision result;

[0075] Step S63: Project the risk decision result onto the smart helmet to complete the driving information processing operation.

[0076] The present invention can identify high-risk vehicles and highlight their trajectories by analyzing the driving prediction trajectories corrected in multiple time periods, thereby increasing awareness of potential dangers. The intelligent helmets that extract the driving prediction trajectories of risky vehicles can monitor and record the driving paths of these vehicles in real time, providing a basis for subsequent risk avoidance decisions. Risk avoidance decisions made based on the driving prediction trajectories of risky vehicles can quickly respond to potential dangers and take necessary actions to avoid accidents. The generated risk decision results include avoidance strategies and action recommendations, which improve the driver's adaptability and driving safety. The risk decision results are projected to the intelligent helmet in real time, so that the driver can intuitively understand the potential risks and avoidance measures taken, thereby improving the efficiency of dealing with dangerous situations. Completing the driving information processing operation means that the intelligent helmet successfully integrates and presents the risk decision results to the user, providing the user with all-round driving safety protection and assisted driving functions.

[0077] In this specification, a driving information processing device based on a smart helmet is provided, which is used to execute the driving information processing method based on a smart helmet as described above, including:

[0078] The video enhancement module is used to obtain real-time driving process monitoring video and user driving posture data based on the smart helmet; the real-time driving process monitoring video is processed by histogram equalization and global delay interpolation smoothing to obtain a delay smoothed monitoring video;

[0079] The spatial modeling module is used to perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, model the scene spatial structure, and construct a dynamic twin scene model;

[0080] The safe path planning module is used to predict the current scene driving trajectory based on the user's driving posture data, and to plan a safe passage path to generate a scene safe passage path;

[0081] The driving trajectory prediction module is used to mark the dynamic vehicles in all scenes based on the dynamic twin scene model; predict the dynamic movement trajectory of the dynamic vehicles in all scenes, so as to generate multi-period corrected driving prediction trajectories;

[0082] The safety risk analysis module is used to calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, and conduct safety risk analysis and mark high-risk vehicles;

[0083] The risk avoidance decision module makes risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory and generates risk decision results; the risk decision results are projected onto the smart helmet to complete the driving information processing task.

[0084] The present invention enhances the contrast of the video, makes the details in the video clearer, improves the quality and visibility of the monitoring video, reduces the jitter and discontinuity in the video through smoothing, improves the video fluency and coherence, and makes the monitoring video easier to analyze and understand. It also segments the road scene in the video and extracts semantic features to understand the road environment and identify different objects, thus providing a basis for subsequent modeling and path planning. Establishing the spatial structure model of the scene can help the helmet system better understand the surrounding environment and improve the accuracy and safety of driving decisions. By predicting the driving trajectory of the user in the current scene, it can identify potential risks and dangerous situations in advance, provide a basis for safe passage path planning, and generate a scene safe passage path avoidance Avoid collisions and optimize driving paths, improve driving safety and efficiency, mark dynamic vehicles in the scene and predict their movement trajectories. The smart helmet system better understands the behavior of surrounding vehicles and reduces driving risks. Generating a revised driving prediction trajectory can help the system more accurately predict the vehicle's position and trajectory, improve driving safety, and identify high-risk vehicles and take corresponding measures to improve driving safety by calculating the probability of collision and conducting risk analysis. Based on the driving prediction trajectory and risk analysis results, the system can generate corresponding risk avoidance decisions to help drivers avoid potential dangerous situations. The risk decision results are projected onto the smart helmet, allowing the driver to understand the risk situation in real time and improve the timeliness and accuracy of driving decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 A schematic flow chart of the steps of a method for processing driving information based on a smart helmet according to the present invention;

[0086] Figure 2 Detailed implementation flow chart of step S1;

[0087] Figure 3 Detailed implementation flow chart of step S2;

[0088] Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0089] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0090] The example of this application provides a method and device for processing driving information based on a smart helmet. The execution subject of the method and device for processing driving information based on a smart helmet includes but is not limited to the following: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of this application, and the data processing platform includes but is not limited to: at least one of an audio and image management system, an information management system, and a cloud data management system.

[0091] See also Figures 1 to 4 The present invention provides a driving information processing method based on a smart helmet, and the driving information processing method based on a smart helmet comprises the following steps:

[0092] Step S1: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet; performing histogram equalization processing on the real-time driving process monitoring video, and performing global delay interpolation smoothing processing to obtain a delay smoothed monitoring video;

[0093] Step S2: perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, perform scene spatial structure modeling, and construct a dynamic twin scene model;

[0094] Step S3: predict the current scene driving trajectory based on the user's driving posture data, and plan a safe passage path to generate a scene safe passage path;

[0095] Step S4: Marking the dynamic vehicles in all scenes based on the dynamic twin scene model; predicting the dynamic movement trajectory of the dynamic vehicles in all scenes, thereby generating a multi-period corrected driving prediction trajectory;

[0096] Step S5: Calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, perform safety risk analysis, and mark it as a high-risk vehicle;

[0097] Step S6: Make risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory and generate risk decision results; project the risk decision results onto the smart helmet to complete the driving information processing operation.

[0098] The present invention enhances the contrast and brightness of video images through histogram equalization processing, improves image quality, and makes monitoring videos clearer. Global time delay interpolation smoothing processing reduces jitter and discontinuity in the video, improves user viewing experience, and ensures video fluency. Deep road semantic feature segmentation and scene spatial structure modeling accurately understand road conditions and scene structures, provide accurate scene information for subsequent safe passage path planning, construct a dynamic twin scene model to capture and reflect the dynamic changes of road scenes in real time, improve the driving system's ability to understand and predict scenes, and use driving trajectory prediction and safe passage path planning to predict the user's driving trajectory in advance and plan a safe path, improve driving safety and efficiency, generate a scene safe passage path to avoid potential dangers, and provide users with safe driving. Guidance, dynamic mobile trajectory prediction and multi-period corrected driving prediction trajectory accurately predict the movement trajectory of vehicles in the scene, improve the system's understanding and prediction ability of vehicle behavior, generate corrected driving prediction trajectory to optimize driving path, improve driving efficiency and safety, collision probability calculation and safety risk analysis to identify high-risk vehicles, discover potential collision risks in advance, reduce the occurrence of traffic accidents, mark high-risk vehicles to warn drivers, improve drivers' awareness of potential dangers, enhance safety awareness, risk avoidance decisions and risk decision results are projected onto smart helmets to provide real-time risk prompts and decision support, help drivers make correct decisions, complete driving information processing operations to optimize drivers' driving experience, improve driving safety and comfort, and realize intelligent driving assistance functions.

[0099] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of the steps of a method for processing driving information based on a smart helmet of the present invention. In this example, the steps of the method include:

[0100] Step S1: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet; performing histogram equalization processing on the real-time driving process monitoring video, and performing global delay interpolation smoothing processing to obtain a delay smoothed monitoring video;

[0101] In this embodiment, the sensors (such as cameras, accelerometers, gyroscopes, etc.) of the smart helmet are ensured to work properly, and a connection is established with the computing device through a suitable interface (such as Bluetooth or Wi-Fi), and real-time driving process monitoring video is obtained from the camera of the smart helmet to ensure that the video data is smooth and clear. At the same time, the user's driving posture data, including the vehicle's speed, acceleration, steering angle, etc., are collected. These data should be recorded synchronously with the video stream for subsequent analysis. Each frame of the image is extracted from the real-time monitoring video, and a suitable histogram equalization algorithm (such as global equalization or adaptive histogram equalization (CLAHE)) is selected. ) to enhance the contrast of the image, apply the histogram equalization algorithm to each frame of the image, calculate the equalization value of each pixel to improve the brightness and contrast of the image, use image processing libraries such as OpenCV, call related functions to implement equalization processing, ensure that each extracted frame of the image is synchronized with the timestamp of the user's driving posture data, select a suitable interpolation algorithm (such as linear interpolation, spline interpolation or resampling) to deal with the delay problem, for each frame in each time period, interpolate according to the timestamp difference, smooth the delay in the video stream, generate new frames through the interpolation algorithm, and ensure that the video is smooth and there is no obvious lag during playback.

[0102] Step S2: perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, perform scene spatial structure modeling, and construct a dynamic twin scene model;

[0103] In this embodiment, a suitable deep learning semantic segmentation model (such as U-Net, DeepLab, SegNet, etc.) is selected to ensure that the model can effectively handle traffic scenes. If a pre-trained model is used, ensure that an appropriate data set (such as Cityscapes or KITTI) is loaded for fine-tuning. If a custom model is used, it is necessary to prepare training data containing annotations, pre-process each frame in the time-delay smoothed monitoring video, including normalization, resizing, etc., to meet the model input requirements, input each pre-processed image frame, use the deep learning model to perform semantic segmentation, generate a semantic feature map for each frame, convert the output feature map into a label map containing different categories (such as roads, vehicles, pedestrians, traffic signs, etc.), and identify static objects in the scene (such as roads, buildings, etc.) based on the generated semantic feature map. For dynamic objects in the scene (such as moving vehicles), their motion paths and relative positions are recorded instead of static features. Suitable 3D modeling software or libraries (such as Blender, Open3D, Unity, etc.) are selected to build the scene model. According to the extracted spatial features, the 3D structure of the static scene is reconstructed, the geometric shape and spatial position of each object are defined, the motion trajectory of dynamic objects is defined, and corresponding animation effects are added to the model to ensure that the dynamic performance of the model is accurate. The constructed 3D scene model is optimized in detail, including adding textures, lighting effects, and physical properties to enhance the realism of the model. The final dynamic twin scene model is saved in an appropriate file format (such as FBX, OBJ) for subsequent use and display.

[0104] Step S3: predict the current scene driving trajectory based on the user's driving posture data, and plan a safe passage path to generate a scene safe passage path;

[0105] In this embodiment, the user's real-time driving posture data, including speed, acceleration, steering angle, etc., are collected. These data should be combined with the environmental data in the dynamic twin scene model. Extract static and dynamic features of the current environment (such as road structure, obstacle location, traffic signs, etc.) from the dynamic twin scene model. It is suitable for dynamic system state estimation and can handle noise and uncertainty. Recursive neural network (RNN) or long short-term memory network (LSTM) is suitable for processing time series data and can capture complex driving behavior patterns. The user's driving posture data is input into the selected prediction model together with the scene features. The predicted driving trajectory in the current scene is generated by model calculation, and the output is a series of spatial coordinate points (such as (x, y, t)). Determine the constraints for safe passage, including: considering the volume and turning radius of the user's vehicle. Extract the location, shape and safe space of obstacles from the dynamic twin scene model. Consider factors such as driving direction, speed limit and traffic signals. Select a suitable path planning algorithm, for example: suitable for finding the optimal path in a known environment. Suitable for finding the shortest path in a weighted graph. Applicable to path planning in complex and dynamic environments. Path planning execution: The predicted trajectory, scene features, and safe passage constraints are used as input data. The selected algorithm is used to calculate the safe passage path to ensure that the path avoids obstacles and complies with traffic regulations. The generated path is smoothed to reduce sharp turns and acceleration and deceleration to improve driving comfort. Check whether the optimized path meets the safety standards to ensure that there is no potential collision risk.

[0106] Step S4: Marking the dynamic vehicles in all scenes based on the dynamic twin scene model; predicting the dynamic movement trajectory of the dynamic vehicles in all scenes, thereby generating a multi-period corrected driving prediction trajectory;

[0107] In this embodiment, the current dynamic twin scene model is loaded to ensure that it contains real-time traffic information and scene features, a real-time video stream or image sequence containing vehicles is obtained from the monitoring system, and a suitable target detection algorithm (such as YOLO, SSD, Faster R-CNN, etc.) is selected to be able to identify dynamic vehicles in real time. Each frame in the real-time video stream is processed, and the selected target detection algorithm is applied to identify and mark dynamic vehicles, and the position, speed, category, timestamp and other information of each detected dynamic vehicle are recorded, and stored in a data structure for subsequent use. The dynamic vehicle marking data extracted from the first step includes the position, speed and time information of each vehicle, and environmental features (such as road structure, traffic signals, etc.) are extracted from the dynamic twin scene model. Dynamic trajectory prediction model selection: Select a suitable dynamic trajectory prediction model, such as: suitable for dynamic system state estimation, able to process vehicle motion noise, suitable for processing time series data, able to capture the vehicle's driving mode, and predict according to the vehicle's physical characteristics (such as speed, acceleration). The current state of the dynamic vehicle (position, Speed, direction) as input, combined with environmental data, use the selected prediction model to generate the future movement trajectory of each vehicle, and output the coordinates of a series of time points (such as (x, y, t)). The predicted trajectory of the dynamic vehicle is divided into time periods (such as short-term prediction, medium-term prediction and long-term prediction) to ensure that the trajectory of each time period can be analyzed. Constraint definition: Road constraints: Consider the road structure and traffic rules, and make constraint corrections to the predicted trajectory of the vehicle. Check the relative position of the predicted trajectory with other dynamic objects (such as other vehicles, pedestrians, etc.) to ensure that there is no collision risk. Use a path optimization algorithm (such as A* or Dijkstra) to optimize the predicted trajectory of each time period to ensure that it is within the road range and avoids obstacles. Smooth the corrected trajectory to reduce sharp turns and unnecessary acceleration and deceleration, and improve driving comfort.

[0108] Step S5: Calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, perform safety risk analysis, and mark it as a high-risk vehicle;

[0109] In this embodiment, the corrected driving prediction trajectories of all vehicles extracted from the previous steps include the movement information of each vehicle in different time periods, and the safe passage path data in the scene is loaded to ensure that road boundaries, lane lines and other traffic facilities are included. Collision model selection: Select a suitable collision probability calculation model, such as: based on the geometric relationship between the position and shape of the vehicle, calculate the collision probability of the relative position, consider the dynamic model of the vehicle speed and direction, predict the future collision risk, randomly generate multiple scenes, analyze the collision probability, perform intersection detection on the corrected driving prediction trajectory of each pair of vehicles, and determine whether a collision will occur in the future time period. According to the intersection situation and factors such as speed and distance, the collision probability of each pair of vehicles is calculated and output as a probability value. The calculated collision probability is stored in a data structure for subsequent analysis and selection of an appropriate A comprehensive safety risk assessment model is used, such as: calculating the safety risk value based on the collision probability and other factors (such as vehicle type and driving behavior), classifying risks based on historical data and rules, calculating the safety risk value of each vehicle by comprehensively considering factors such as collision probability, vehicle speed, distance, driving environment, etc., setting a safety risk threshold based on historical data, usually based on industry standards or empirical data, comparing the safety risk value of each vehicle, and marking it as a high-risk vehicle if the value is higher than the threshold. The information of all high-risk vehicles, including their location, speed, risk value, etc., is saved for subsequent processing and alarms, and the results of collision probability and high-risk vehicles are visualized, using a graphical interface to display the vehicle location, predicted trajectory and risk level, and generating a risk analysis report containing detailed information on high-risk vehicles, collision probability data and recommended safety measures.

[0110] Step S6: Make risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory and generate risk decision results; project the risk decision results onto the smart helmet to complete the driving information processing operation.

[0111] In this embodiment, the corrected driving prediction trajectories of all vehicles are extracted, including the information of high-risk vehicles, and all vehicles marked as high-risk and their related data (such as location, speed, and driving direction) are collected. For each high-risk vehicle, the selected model is used to analyze the intersection of its current trajectory with its own trajectory, and a corresponding avoidance strategy is generated, such as: lane change: change lanes from the current lane under safe conditions, slow down: slow down when approaching a high-risk vehicle, stop: stop and wait when necessary to avoid conflicts, and store the detailed information of each decision (such as the selected avoidance strategy, execution conditions, etc.) for subsequent use, ensure that the communication interface (such as Bluetooth, Wi-Fi) between the smart helmet and the vehicle works normally and can receive data, and convert the risk decision results into smart The risk decision result is formatted in a format recognizable by the helmet (such as JSON or a specific protocol format), including the decision type, execution time and related conditions. The formatted risk decision result is sent to the smart helmet through the communication interface, and the risk decision result is displayed on the display screen of the smart helmet to ensure that the driver can quickly understand and respond. Visual signals (such as icons, text prompts) can be used for display. If feasible, audio prompts are added to enhance the driver's perception of risks. A feedback mechanism is implemented in the smart helmet to collect the driver's response to the risk decision results (such as confirmation, ignore, execution, etc.), and the driver's feedback and decision results are stored in the system for subsequent analysis and optimization. According to the collected feedback data, the risk avoidance decision model is regularly optimized to improve the accuracy and real-time performance of the decision.

[0112] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0113] Step S11: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet;

[0114] Step S12: performing brightness deviation detection on the real-time driving process monitoring video to identify the brightness deviation area;

[0115] Step S13: performing histogram equalization processing based on the brightness deviation area to generate a brightness enhanced monitoring video;

[0116] Step S14: performing driving scene edge analysis on the brightness enhanced monitoring video to extract driving scene edge data;

[0117] Step S15: optimizing edge details of the brightness enhanced monitoring video according to the edge data of the travel scene, and constructing an edge optimized monitoring video;

[0118] Step S16: Perform global delay interpolation smoothing processing on the edge optimized monitoring video to obtain a delay smoothed monitoring video.

[0119] In this embodiment, a camera on a smart helmet is used to record a video of the driving process in real time, ensuring that the camera's viewing angle can capture the road ahead and the surrounding environment, and using built-in sensors in the helmet (such as an accelerometer and a gyroscope) to collect the user's driving posture data, including the tilt angle and movement state of the head. The video stream and the posture data are synchronized in time so that the video frame can correspond to the user's driving posture in subsequent analysis. A brightness deviation detection is performed on each frame of video, and areas with brightness lower than or higher than the normal range are identified and marked as brightness deviation areas. The position information of the identified brightness deviation areas is recorded to provide a basis for subsequent processing, and a suitable histogram equalization algorithm, such as CLAHE (adaptive histogram equalization), is selected to avoid over-enhancement, and the identified The brightness deviation areas are processed by histogram equalization to improve the contrast and brightness of these areas, and a brightness enhanced monitoring video is generated. A suitable edge detection algorithm is selected, such as Canny edge detection or Sobel operator, and edge analysis is performed on each frame in the brightness enhanced monitoring video to extract edge data of the driving scene. An edge detail optimization algorithm is selected, such as high-pass filtering or sharpening processing. According to the extracted edge data, edge details of the brightness enhanced monitoring video are optimized to increase edge clarity. The processed edge optimized monitoring video is output to ensure that the enhanced edge features are more obvious. Time-delay interpolation smoothing is performed on continuous frames in the edge optimized monitoring video to reduce jitter and blur caused by motion, and a time-delay smoothed monitoring video is generated to ensure that the video is smooth and the picture is clear.

[0120] In this embodiment, the specific steps of step S16 are:

[0121] Perform time-series window segmentation on edge-optimized monitoring video and extract the time-series image of each frame;

[0122] Calculating the timestamp of each frame of the time-series image;

[0123] Calculate the time interval between the upper and lower frames according to the timestamp to obtain the inter-frame interval duration of all image frames;

[0124] Fitting the interval duration distribution of all image frames to obtain a global interval distribution map of the video;

[0125] Perform interval discrete unevenness identification on the global interval distribution map of the video and extract the points with uneven interval duration;

[0126] Positioning the time-series image frames based on the unevenly spaced time points, extracting all the unevenly spaced image frames;

[0127] Calculating the inter-frame delay of all non-uniformly spaced image frames, thereby obtaining the delay parameters of the non-uniformly spaced frames;

[0128] The average inter-frame interval duration of all image frames is counted to generate a mean inter-frame interval value;

[0129] Performing delay compensation calculation on the delay parameters of the non-uniformly spaced frames according to the mean inter-frame interval value, thereby obtaining the delay compensation parameters of the non-uniformly spaced frames;

[0130] Performing time-series serialization processing on each frame of the time-series image according to the timestamp to obtain a time-series image frame sequence;

[0131] The delay compensation parameters of non-uniformly spaced frames are used to perform delay interpolation and smoothing processing on the time-series image frame sequence, thereby obtaining a delay-smoothed monitoring video.

[0132] In this embodiment, the edge optimization monitoring video is loaded from the file system, and a video processing library (such as OpenCV) is used to segment the video according to a set time window (such as per second or per frame). A loop can be used to traverse each frame in the video, and the frames in each window are extracted according to the time interval. The frames in each window are saved in a list or array for subsequent processing. The frame rate (FPS) is obtained through the metadata of the video file. For each frame image, a timestamp is calculated, wherein the frame index increases from 0, and the timestamp of each frame generated is stored in an array and associated with each frame image. For each frame image, the time interval with the previous frame is calculated. All calculated frame interval durations are saved in an array to form a time interval data sequence. The frame interval data is analyzed using statistical tools (such as NumPy and SciPy), the frequency distribution of each time interval is calculated, and a suitable distribution model (such as normal distribution and gamma distribution) is selected for fitting. The fit method in scipy.stats can be used for fitting. Matplotlib or other visualization libraries are used to generate a global interval distribution diagram of the video to display the distribution of each time interval, the standard deviation of the time interval is calculated, and a reasonable threshold is defined to identify which Some time intervals are uneven, all time interval points that exceed the threshold are recorded as uneven duration points, their indexes and corresponding durations are stored, and the corresponding uneven interval image frames are extracted from the original image sequence according to the indexes of the uneven interval duration points extracted in step 5, and the extracted uneven interval image frames are saved in a new data structure for subsequent processing, and the inter-frame delay calculation is performed on all the extracted uneven interval image frames, and the calculated delay parameters of the uneven interval frames are saved in an array, and the average value of the frame interval duration of all image frames is calculated, and the generated average inter-frame interval value is recorded, and the average inter-frame interval value is calculated according to the average inter-frame interval. Interval, delay compensation calculation is performed on the delay parameters of non-uniformly spaced frames, and the time-series images of each frame are serialized according to the calculated timestamp to ensure that the images are arranged in chronological order to form a new time-series image frame sequence, which is convenient for subsequent delay interpolation and smoothing processing. A suitable interpolation algorithm (such as linear interpolation and spline interpolation) is selected to perform delay smoothing processing on the time-series image frame sequence. According to the calculated delay compensation parameters, the time-series image frame sequence is smoothly interpolated to generate a delay smoothing monitoring video, and the final generated delay smoothing monitoring video is saved as a video file (such as MP4 or AVI) for subsequent viewing and use.

[0133] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0134] Step S21: performing visual recognition of scene objects on the time-delay smoothed monitoring video, marking objects in the driving scene, wherein the objects in the driving scene include static road signs, road obstacles and dynamic vehicles;

[0135] Step S22: performing deep road semantic feature segmentation on the time-delay smoothed monitoring video to generate road semantic features of the driving scene;

[0136] Step S23: Perform scene spatial structure analysis on static road signs and road obstacles to generate scene static spatial structure data;

[0137] Step S24: performing scene spatial structure modeling on the semantic features of the driving scene road and the scene static spatial structure data to construct a three-dimensional driving scene model;

[0138] Step S25: Perform dynamic movement evolution simulation on the three-dimensional driving scene model according to the dynamic vehicle to construct a dynamic twin scene model.

[0139] In this embodiment, a suitable object detection algorithm (such as YOLO, Faster R-CNN) is selected to identify objects in the video, each frame of the video is processed, static road signs, road obstacles and dynamic vehicles are identified, and a bounding box and a label are generated for each identified object, and the identification results (including the object position and type) are stored in a data structure for subsequent use, and a suitable deep learning model (such as U-Net, DeepLab) is selected for road semantic segmentation, and each frame of the image is preprocessed (such as normalization, resizing) to adapt to the model input, and the processed frame is input into the semantic segmentation model to generate a road semantic feature map for each frame, and the generated driving scene road semantic feature map is saved, including segmentation information of categories such as roads, pedestrians, and vehicles, based on the identified static road signs and road obstacles, their spatial features (such as position, size and shape) are extracted, and a geometric modeling method (such as point cloud or three-dimensional grid) is used to establish static spatial structure data of the scene to represent the layout of objects in three-dimensional space, and the generated The static spatial structure data of the scene is saved for subsequent modeling. A three-dimensional modeling software or library (such as Blender, Open3D) is used to construct a three-dimensional driving scene model. According to the generated road semantic features and the static spatial structure data generated in step S23, a complete three-dimensional driving scene model is constructed. The model is optimized in detail to ensure the realism and accuracy of the model, including texture mapping and lighting effects. The constructed three-dimensional driving scene model is saved in an appropriate format (such as OBJ, FBX) for subsequent use. A suitable dynamic simulation framework (such as Unity, Gazebo) is selected to implement the dynamic twin scene model. According to the identified dynamic vehicle, its motion behavior (such as speed, acceleration and path) is defined, and the movement of the dynamic vehicle is realized in the three-dimensional driving scene model. Dynamic evolution simulation is performed to simulate the vehicle motion in the real driving scene. The generated dynamic twin scene model is saved for subsequent analysis and visualization.

[0140] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0141] Step S31: Calculate the driving space volume based on the user's driving posture data to generate the user's standard driving volume;

[0142] Step S32: Analyze the user's driving trajectory on the time-delay smoothed monitoring video to extract the user's time-series driving trajectory;

[0143] Step S33: predicting the current scene driving trajectory of the user's time series driving trajectory based on the user's standard driving volume, and generating a predicted driving trajectory in the scene;

[0144] Step S34: Calculate the obstacle area volume of the dynamic twin scene model to generate obstacle area volume parameters;

[0145] Step S35: Plan a safe passage path for the predicted driving trajectory in the scene based on the volume parameters of the obstacle area to generate a safe passage path for the scene.

[0146] In this embodiment, the user's driving posture data is collected, including the size, position and driving state (such as speed and steering angle) of the vehicle. The user's standard driving volume is calculated according to the geometric shape of the vehicle (such as a cuboid or a polygon). The calculated user's standard driving volume is recorded and saved as a data structure. The user's driving trajectory is extracted from the time-delay smoothed monitoring video. The object detection result identified in the previous step is used to extract the user's time-series driving trajectory by analyzing the timestamp and vehicle position in the video and record it as a sequence of coordinate points (such as (x, y, t)). The extracted user's time-series driving trajectory is stored in an array for subsequent use. A machine learning or deep learning model (such as an LSTM or a Kalman filter) is used to predict the user's time-series driving trajectory. Based on the user's standard driving volume and the extracted driving trajectory data, a prediction model is trained so that it can take into account The vehicle's motion characteristics, input the current user's driving trajectory data, generate the future driving prediction trajectory in the scene, record it as a sequence of coordinate points, store the generated driving prediction trajectory for subsequent path planning, and identify the obstacles in the scene according to the object detection results in the dynamic twin scene model. The volume of each identified obstacle is calculated, usually using a simple geometric volume formula (such as a cuboid, cylinder, etc.), and the total obstacle area volume is calculated. Select a suitable path planning algorithm (such as A* algorithm, Dijkstra algorithm or RRT) to plan the user's safe passage path, input the user's standard driving volume and obstacle area volume parameters into the path planning algorithm to ensure that the vehicle's space requirements and obstacle positions are considered during the planning process, and generate a safe passage path based on the predicted driving trajectory and obstacle area, recorded as a sequence of coordinate points.

[0147] In this embodiment, the specific steps of step S35 are:

[0148] Locate obstacles in the dynamic twin scene model and mark the obstacle coordinates;

[0149] The encounter time point of the predicted driving trajectory in the scene is calculated according to the obstacle position coordinates to obtain the obstacle encounter time point of the predicted trajectory;

[0150] The obstacle area volume parameters are expanded to generate the obstacle safety space bounding volume;

[0151] Based on the obstacle encounter time point of the predicted trajectory, the potential collision area of ​​the obstacle safety space is analyzed to generate the user's potential collision area;

[0152] Perform normal traffic inference on the user's potential collision area and generate user traffic inference results;

[0153] When the user's estimated result is that the vehicle can pass normally, the predicted driving trajectory in the scene is used as the safe passage path of the scene;

[0154] When the user's passage speculation result is that he cannot pass normally, a safe passage path is planned for the user's potential collision area to generate a scene-based safe passage path.

[0155] In this embodiment, the identified obstacle data, including the type and location of the obstacle, is extracted from the dynamic twin scene model.

[0156] Generate position coordinates (such as (x, y, z)) for each identified obstacle, record its precise position in three-dimensional space, store the obstacle's position information in a data structure for subsequent use, analyze the user's predicted driving trajectory frame by frame, calculate the relative position of each frame and the obstacle's position coordinates, and use the following formula to calculate the encounter time point with each obstacle: Encounter time point = current time + (obstacle distance / user speed), save all calculated obstacle encounter time points, set the expansion parameters of the obstacle safety space (such as buffer size) according to safety standards and vehicle size, expand the position coordinates of each obstacle, and generate a safe space bounding volume. Geometric bodies (such as spheres or cubes) can be used to represent the safe area. Based on the obstacle encounter time point of the predicted trajectory, analyze the collision area that occurs within the encounter time, analyze the safe space bounding volume of each obstacle, and determine The intersection with the predicted trajectory is recorded as a potential collision area, and the generated user potential collision area is saved for subsequent inference results. According to the size of the potential collision area and the user's driving volume, it is judged whether the user can pass normally, and the user's passage inference result is generated, marked as "normal passage" or "unable to pass normally", and the passage inference result is recorded for subsequent path selection. If the user's passage inference result is "normal passage", the predicted driving trajectory in the scene is directly used as a safe passage path. If the result is "unable to pass normally", it is necessary to plan a safe passage path for the user's potential collision area, and use a suitable path planning algorithm (such as A* or Dijkstra) to generate a new safe passage path. The constraints of the potential collision area and the position of the obstacles are used as input parameters to ensure that the planned path avoids the collision area, and a new scene safe passage path is generated and recorded as a sequence of coordinate points.

[0157] In this embodiment, step S4 includes the following steps:

[0158] Step S41: Perform dynamic vehicle optical flow recognition on the dynamic twin scene model and mark all dynamic vehicles in the scene;

[0159] Step S42: Calculate the optical flow displacement of all dynamic vehicles in the scene one by one to generate the vehicle moving speed;

[0160] Step S43: Analyze the moving directions of all dynamic vehicles in the scene and generate the moving directions of the vehicles;

[0161] Step S44: dynamically predicting the vehicle moving speed and direction, and generating predicted driving trajectories of all vehicles in the scene;

[0162] Step S45: performing road constraint correction on the predicted driving trajectories of all vehicles in the scene based on the scene static spatial structure data to generate a corrected predicted driving trajectory;

[0163] Step S46: Perform multi-period collaborative analysis on the corrected driving prediction trajectory to generate a multi-period corrected driving prediction trajectory.

[0164] In this embodiment, a real-time video stream or image sequence is extracted from the dynamic twin scene model, and an optical flow algorithm (such as the Lucas-Kanade method or the Horn-Schunck method) is used to process the video frames to identify dynamic vehicles in the scene, and the identified dynamic vehicles are marked, and the position coordinates and optical flow vectors of each vehicle are generated. The marking results are stored in a data structure, including the position information and optical flow data of each vehicle, for use in subsequent steps. By analyzing the position changes of dynamic vehicles in consecutive frames, the optical flow displacement of each vehicle is calculated, and the formula: displacement = current frame position - previous frame position is used. The moving speed of the vehicle is calculated according to the optical flow displacement and the time interval (frame rate): speed = displacement / time interval, and the moving speed of each vehicle is stored in an array. The moving direction of the vehicle is analyzed according to the optical flow vector, and the inverse tangent function is usually used to calculate the direction angle: direction = tan-1 (Δx / Δy). The calculated direction angle is standardized to a value between 0 and 360 degrees for subsequent use. The moving directions of all dynamic vehicles are recorded in a data structure, associated with the identification of each vehicle, and a suitable trajectory prediction algorithm (such as The method uses a linear regression, Kalman filter or LSTM) to predict the movement trajectory of the vehicle, takes the movement speed and direction of each vehicle as input parameters, combines the historical trajectory data for model training, uses the model to predict the future position of each vehicle, generates the driving prediction trajectory of all vehicles, stores the predicted driving trajectory as a sequence of coordinate points for use in subsequent steps, extracts road information such as road boundaries, lane lines and traffic signs from the static spatial structure data of the scene, uses a path constraint algorithm (such as graph-based path planning or constraint optimization) to correct the driving prediction trajectory of each vehicle to ensure that the trajectory meets the road constraints, records the corrected trajectory as a new sequence of coordinate points to ensure that it is within the road range, stores the corrected driving prediction trajectory, divides the driving prediction trajectory into time periods (such as short time periods, medium time periods and long time periods), ensures that the trajectory data of each time period can be analyzed, uses a multi-time period analysis algorithm (such as time series analysis or ensemble learning) to analyze the corrected driving prediction trajectory of each time period, identifies the change trend, comprehensively analyzes the results, generates multi-time period corrected driving prediction trajectory, and ensures that the prediction trajectories of each time period are interrelated.

[0165] In this embodiment, step S5 includes the following steps:

[0166] Step S51: matching the meeting time of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, and marking the meeting section time point;

[0167] Step S52: cutting the dynamic twin scene model into time window areas according to the time point of the encounter section, and extracting the intersection area model;

[0168] Step S53: Calculate the collision probability of the intersection area model to generate driving collision probability data;

[0169] Step S54: Performing safety risk analysis on the vehicle collision probability data to generate a safety risk value for each vehicle;

[0170] Step S55: risk identification is performed on the safety risk value of each vehicle based on a preset driving safety risk threshold. When the preset driving safety risk threshold is greater than the safety risk value of the vehicle, it is marked as a high-risk vehicle.

[0171] In this embodiment, the safe passage path data based on the dynamic twin scenario is obtained, the corrected driving prediction trajectory of multiple time periods is loaded from the monitoring system, the driving prediction trajectory is time-matched with the safe passage path, and the time point of the encounter section is found. The following method can be used: the vehicle position and the path are interpolated according to the timestamp to determine the encounter time point, the intersection point of the path is detected using a geometric method, and the time point of the encounter section is recorded. The dynamic twin scenario model is loaded from the system, and the time window area is defined according to the marked encounter section time point. The area should include the spatial range where the meeting occurs. The dynamic twin scenario model is cut using a computational geometry algorithm to extract the intersection area model. Boolean operations (such as intersection) can be used to obtain the intersection part, and the extracted intersection area model is stored for subsequent collision probability calculation. According to the extracted intersection area model, Analyze the positions of vehicles and obstacles in the intersection area, select a suitable collision probability calculation model (such as Monte Carlo simulation or probability map), calculate the collision probability for each pair of intersecting vehicles and obstacles, consider vehicle speed, distance and driving direction, select a suitable risk assessment model (such as decision tree or weighted scoring system) to analyze the driving collision probability data, and calculate the safety risk value for each vehicle based on the collision probability and other relevant factors (such as vehicle type, driving environment). Safety risk value = P (collision) × risk coefficient, set a preset driving safety risk threshold, usually based on historical data and safety standards, compare the safety risk value of each vehicle to determine whether it exceeds the preset threshold. If the safety risk value is greater than the threshold, it will be marked as a high-risk vehicle, and the information of the high-risk vehicle will be saved for subsequent processing and alarm.

[0172] In this embodiment, step S6 includes the following steps:

[0173] Step S61: highlighting the risky vehicle trajectory of high-risk vehicles according to the multi-period corrected driving prediction trajectory, and extracting the risky vehicle driving prediction trajectory;

[0174] Step S62: making a risk avoidance decision on the predicted driving trajectory of the risky vehicle and generating a risk decision result;

[0175] Step S63: Project the risk decision result onto the smart helmet to complete the driving information processing operation.

[0176] In this embodiment, the information of vehicles marked as high-risk is extracted from the previous analysis, including their positions, speeds, and predicted trajectories, and the driving prediction trajectory data corrected in multiple periods is loaded to ensure that the driving information of all relevant vehicles is included. The driving prediction trajectory of high-risk vehicles is screened out, and the trajectory of high-risk vehicles is highlighted using a graphics library (such as Matplotlib or OpenGL), which can be achieved by changing the color, line width, or adding marks. The highlighted driving prediction trajectory of the risky vehicle is saved, and a suitable risk avoidance decision model (such as a decision tree, reinforcement learning, or a rule engine) is selected. The driving prediction trajectory of the risky vehicle, environmental information (such as road conditions, obstacle locations), and vehicle dynamic characteristics (such as speed and acceleration) are used as input parameters, and avoidance decision rules are defined according to safety standards and driving rules. For example, if the predicted trajectory of the risky vehicle intersects with its own driving path, it is necessary to change lanes or slow down. The selected model is used for calculation to generate risk avoidance decision results, such as "change lanes to the left", "slow down" or "stop and wait". Ensure that the communication interface (such as Bluetooth or Wi-Fi) between the smart helmet and the vehicle works normally and can receive external data. Convert the risk decision results into a format recognizable by the smart helmet (such as JSON or a specific protocol format), send the converted risk decision results to the smart helmet through the communication interface, and display the risk decision results on the display screen of the smart helmet to ensure that the driver can quickly understand and respond. Visual signals (such as icons, text prompts) can be used for display. Implement a feedback mechanism in the smart helmet to collect the driver's response to the decision results (such as confirmation, ignore) for subsequent system optimization.

[0177] In this embodiment, a driving information processing device based on a smart helmet is provided, which is used to execute the driving information processing method based on a smart helmet as described above, including:

[0178] The video enhancement module is used to obtain real-time driving process monitoring video and user driving posture data based on the smart helmet; the real-time driving process monitoring video is processed by histogram equalization and global delay interpolation smoothing to obtain a delay smoothed monitoring video;

[0179] The spatial modeling module is used to perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, model the scene spatial structure, and construct a dynamic twin scene model;

[0180] The safe path planning module is used to predict the current scene driving trajectory based on the user's driving posture data, and to plan a safe passage path to generate a scene safe passage path;

[0181] The driving trajectory prediction module is used to mark the dynamic vehicles in all scenes based on the dynamic twin scene model; predict the dynamic movement trajectory of the dynamic vehicles in all scenes, so as to generate multi-period corrected driving prediction trajectories;

[0182] The safety risk analysis module is used to calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, and conduct safety risk analysis and mark high-risk vehicles;

[0183] The risk avoidance decision module makes risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory and generates risk decision results; the risk decision results are projected onto the smart helmet to complete the driving information processing task.

[0184] The present invention enhances the contrast of the video, makes the details in the video clearer, improves the quality and visibility of the monitoring video, reduces the jitter and discontinuity in the video through smoothing, improves the video fluency and coherence, and makes the monitoring video easier to analyze and understand. It also segments the road scene in the video and extracts semantic features to understand the road environment and identify different objects, thus providing a basis for subsequent modeling and path planning. Establishing the spatial structure model of the scene can help the helmet system better understand the surrounding environment and improve the accuracy and safety of driving decisions. By predicting the driving trajectory of the user in the current scene, it can identify potential risks and dangerous situations in advance, provide a basis for safe passage path planning, and generate a scene safe passage path avoidance Avoid collisions and optimize driving paths, improve driving safety and efficiency, mark dynamic vehicles in the scene and predict their movement trajectories. The smart helmet system better understands the behavior of surrounding vehicles and reduces driving risks. Generating a revised driving prediction trajectory can help the system more accurately predict the vehicle's position and trajectory, improve driving safety, and identify high-risk vehicles and take corresponding measures to improve driving safety by calculating the probability of collision and conducting risk analysis. Based on the driving prediction trajectory and risk analysis results, the system can generate corresponding risk avoidance decisions to help drivers avoid potential dangerous situations. The risk decision results are projected onto the smart helmet, allowing the driver to understand the risk situation in real time and improve the timeliness and accuracy of driving decisions.

[0185] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0186] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method and device for processing driving information based on a smart helmet, characterized in that: The following steps are involved: Step S1: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet; performing histogram equalization processing on the real-time driving process monitoring video, and performing global delay interpolation smoothing processing to obtain a delay smoothed monitoring video; Step S2: perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, perform scene spatial structure modeling, and construct a dynamic twin scene model; Step S3: predict the current scene driving trajectory based on the user's driving posture data, and plan a safe passage path to generate a scene safe passage path; Step S4: Marking dynamic vehicles in all scenes based on the dynamic twin scene model; Predict the dynamic movement trajectory of dynamic vehicles in all scenarios, thereby generating multi-period corrected driving prediction trajectories; Step S5: Calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, perform safety risk analysis, and mark it as a high-risk vehicle; Step S6: making risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory, and generating risk decision results; The risk decision results are projected onto the smart helmet to complete the driving information processing task.

2. The method for processing driving information based on a smart helmet according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: obtaining real-time driving process monitoring video and user driving posture data based on the smart helmet; Step S12: performing brightness deviation detection on the real-time driving process monitoring video to identify the brightness deviation area; Step S13: performing histogram equalization processing based on the brightness deviation area to generate a brightness enhanced monitoring video; Step S14: performing driving scene edge analysis on the brightness enhanced monitoring video to extract driving scene edge data; Step S15: optimizing edge details of the brightness enhanced monitoring video according to the edge data of the travel scene, and constructing an edge optimized monitoring video; Step S16: Perform global delay interpolation smoothing processing on the edge optimized monitoring video to obtain a delay smoothed monitoring video.

3. The method for processing driving information based on a smart helmet according to claim 2, characterized in that: The specific steps of step S16 are: Perform time-series window segmentation on edge-optimized monitoring video and extract the time-series image of each frame; Calculating the timestamp of each frame of the time-series image; Calculate the time interval between the upper and lower frames according to the timestamp to obtain the inter-frame interval duration of all image frames; Fitting the interval duration distribution of all image frames to obtain a global interval distribution map of the video; Perform interval discrete unevenness identification on the global interval distribution map of the video and extract the points with uneven interval duration; Positioning the time-series image frames based on the unevenly spaced time points, extracting all the unevenly spaced image frames; Calculating the inter-frame delay of all non-uniformly spaced image frames, thereby obtaining the delay parameters of the non-uniformly spaced frames; The average inter-frame interval duration of all image frames is counted to generate a mean inter-frame interval value; Performing delay compensation calculation on the delay parameters of the non-uniformly spaced frames according to the mean inter-frame interval value, thereby obtaining the delay compensation parameters of the non-uniformly spaced frames; Performing time-series serialization processing on each frame of the time-series image according to the timestamp to obtain a time-series image frame sequence; The delay compensation parameters of non-uniformly spaced frames are used to perform delay interpolation and smoothing processing on the time-series image frame sequence, thereby obtaining a delay-smoothed monitoring video.

4. The method for processing driving information based on a smart helmet according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing visual recognition of scene objects on the time-delay smoothed monitoring video, marking objects in the driving scene, wherein the objects in the driving scene include static road signs, road obstacles and dynamic vehicles; Step S22: performing deep road semantic feature segmentation on the time-delay smoothed monitoring video to generate road semantic features of the driving scene; Step S23: Perform scene spatial structure analysis on static road signs and road obstacles to generate scene static spatial structure data; Step S24: performing scene spatial structure modeling on the semantic features of the driving scene road and the scene static spatial structure data to construct a three-dimensional driving scene model; Step S25: Perform dynamic movement evolution simulation on the three-dimensional driving scene model according to the dynamic vehicle to construct a dynamic twin scene model.

5. The method for processing driving information based on a smart helmet according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: Calculate the driving space volume based on the user's driving posture data to generate the user's standard driving volume; Step S32: Analyze the user's driving trajectory on the time-delay smoothed monitoring video to extract the user's time-series driving trajectory; Step S33: predicting the current scene driving trajectory of the user's time series driving trajectory based on the user's standard driving volume, and generating a predicted driving trajectory in the scene; Step S34: Calculate the obstacle area volume of the dynamic twin scene model to generate obstacle area volume parameters; Step S35: Plan a safe passage path for the predicted driving trajectory in the scene based on the volume parameters of the obstacle area to generate a safe passage path for the scene.

6. The method for processing driving information based on a smart helmet according to claim 5, characterized in that: The specific steps of step S35 are: Locate obstacles in the dynamic twin scene model and mark the obstacle coordinates; The encounter time point of the predicted driving trajectory in the scene is calculated according to the obstacle position coordinates to obtain the obstacle encounter time point of the predicted trajectory; The obstacle area volume parameters are expanded to generate the obstacle safety space bounding volume; Based on the obstacle encounter time point of the predicted trajectory, the potential collision area of ​​the obstacle safety space is analyzed to generate the user's potential collision area; Perform normal traffic inference on the user's potential collision area and generate user traffic inference results; When the user's estimated result is that the vehicle can pass normally, the predicted driving trajectory in the scene is used as the safe passage path of the scene; When the user's passage speculation result is that he cannot pass normally, a safe passage path is planned for the user's potential collision area to generate a scene-based safe passage path.

7. The method for processing driving information based on a smart helmet according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: Perform dynamic vehicle optical flow recognition on the dynamic twin scene model and mark all dynamic vehicles in the scene; Step S42: Calculate the optical flow displacement of all dynamic vehicles in the scene one by one to generate the vehicle moving speed; Step S43: Analyze the moving directions of all dynamic vehicles in the scene and generate the moving directions of the vehicles; Step S44: dynamically predicting the vehicle moving speed and direction, and generating predicted driving trajectories of all vehicles in the scene; Step S45: performing road constraint correction on the predicted driving trajectories of all vehicles in the scene based on the scene static spatial structure data to generate a corrected predicted driving trajectory; Step S46: Perform multi-period collaborative analysis on the corrected driving prediction trajectory to generate a multi-period corrected driving prediction trajectory.

8. The method for processing driving information based on a smart helmet according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: matching the meeting time of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, and marking the meeting section time point; Step S52: cutting the dynamic twin scene model into time window areas according to the time point of the encounter section, and extracting the intersection area model; Step S53: Calculate the collision probability of the intersection area model to generate driving collision probability data; Step S54: Performing safety risk analysis on the vehicle collision probability data to generate a safety risk value for each vehicle; Step S55: risk identification is performed on the safety risk value of each vehicle based on a preset driving safety risk threshold. When the preset driving safety risk threshold is greater than the safety risk value of the vehicle, it is marked as a high-risk vehicle.

9. The method for processing driving information based on a smart helmet according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: highlighting the risky vehicle trajectory of high-risk vehicles according to the multi-period corrected driving prediction trajectory, and extracting the risky vehicle driving prediction trajectory; Step S62: making a risk avoidance decision on the predicted driving trajectory of the risky vehicle and generating a risk decision result; Step S63: Project the risk decision result onto the smart helmet to complete the driving information processing operation.

10. A driving information processing device based on a smart helmet, characterized in that: Used to execute the driving information processing method based on the smart helmet as claimed in claim 1, comprising: The video enhancement module is used to obtain real-time driving process monitoring video and user driving posture data based on the smart helmet; the real-time driving process monitoring video is processed by histogram equalization and global delay interpolation smoothing to obtain a delay smoothed monitoring video; The spatial modeling module is used to perform deep road semantic feature segmentation on the time-delay smoothed monitoring video, model the scene spatial structure, and construct a dynamic twin scene model; The safe path planning module is used to predict the current scene driving trajectory based on the user's driving posture data, and to plan a safe passage path to generate a scene safe passage path; The driving trajectory prediction module is used to mark the dynamic vehicles in all scenes based on the dynamic twin scene model; predict the dynamic movement trajectory of the dynamic vehicles in all scenes, so as to generate multi-period corrected driving prediction trajectories; The safety risk analysis module is used to calculate the collision probability of the multi-period corrected driving prediction trajectory based on the scenario safe passage path, and conduct safety risk analysis and mark high-risk vehicles; The risk avoidance decision module makes risk avoidance decisions for high-risk vehicles based on the multi-period corrected driving prediction trajectory and generates risk decision results; the risk decision results are projected onto the smart helmet to complete the driving information processing task.

Images