Environment perception data processing method and system based on intelligent helmet

Abstract

The invention relates to the field of data processing, in particular to an environment perception data processing method and system based on an intelligent helmet. The method comprises the following steps: acquiring a scene image in front of a user and multi-source environment real-time sensing data; performing scene attribute state analysis on the multi-source environment real-time sensing data, and constructing a real-time scene attribute state diagram; performing contrast dynamic balance adjustment on the scene image in front of the user, performing super-resolution fusion reconstruction, and constructing a super-resolution scene image; performing object visual identification on the super-resolution scene image, and performing object bounding box segmentation processing to obtain a plurality of accurate scene object bounding boxes; and carrying out optical flow motion change analysis on the plurality of accurate scene object bounding boxes, and carrying out space-time scene structure evolution analysis so as to obtain dynamic scene evolution characteristic data. According to the method, the data processing efficiency of the environment perception data is improved, and the environment perception data is visualized more visually.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an environment perception data processing method and system based on a smart helmet. Background Art

[0002] With the continuous development of intelligent transportation technology, driving safety has become a key issue of concern. Traditional driving behavior monitoring and environmental perception methods often rely on vehicle-mounted sensors and monitoring equipment, which often have low data processing efficiency and poor visibility of environmental perception data, making it difficult to meet the safety needs in the increasingly complex traffic environment. In recent years, driving environment perception methods based on wearable devices have attracted widespread attention. As an intelligent wearable device, driving smart helmets can integrate multiple sensors to monitor the driver's physiological state, driving behavior and various indicators of the surrounding environment in real time, providing strong protection for driving safety.

[0003] However, how to effectively process and analyze the massive environmental perception data obtained from smart helmets, and how to convert this data into actionable decision support, has become a key issue that needs to be solved urgently. Traditional data processing methods usually rely on manual intervention, are inefficient, and it is difficult to cope with complex and changing traffic environments. Therefore, there is an urgent need for an intelligent smart helmet environmental perception data processing method. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes an environment perception data processing method and system based on a smart helmet to solve at least one of the above technical problems.

[0005] To achieve the above object, the present invention provides an environment perception data processing method based on a smart helmet, wherein a high-definition camera and a multi-source sensor are embedded inside the smart helmet, and the method comprises the following steps:

[0006] Step S1: obtaining a scene image in front of the user and real-time perception data of a multi-source environment; performing scene attribute state analysis on the real-time perception data of the multi-source environment, and constructing a real-time scene attribute state diagram;

[0007] Step S2: dynamically adjust the contrast of the scene image in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image;

[0008] Step S3: performing object visual recognition on the super-resolution scene image and performing object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0009] Step S4: performing optical flow motion change analysis on multiple precise scene object bounding boxes, and performing spatiotemporal scene structure evolution analysis, thereby obtaining dynamic scene evolution feature data;

[0010] Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, a dynamic structure evolution simulation is performed on the super-resolution scene image, and then environmental attribute rendering is performed to generate a dynamic attribute rendering scene model;

[0011] Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; perform three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

[0012] The present invention can provide the helmet system with an accurate understanding of the state and characteristics of the user's surrounding environment through the construction of a real-time scene attribute state diagram, providing basic data for subsequent processing. The scene attribute state analysis helps the system identify and understand the attributes of the environment, providing key information for the next processing steps. The construction of super-resolution scene images improves the clarity and details of the image, and enhances the user's perception and understanding of the environment. Dynamic contrast equalization adjustment helps to enhance the visual effect of the image, making the details clearer and improving the user experience. The acquisition of precise scene object boundary boxes helps the system accurately identify and locate objects in the environment, providing important information for subsequent scene analysis and processing. Object visual recognition and segmentation processing improves the system's ability to understand objects in the environment, providing a basis for subsequent analysis and simulation. The acquisition of dynamic scene evolution feature data helps the system capture the movement and Spatiotemporal changes provide richer scene information. Spatiotemporal scene structure evolution analysis helps the system understand the dynamic changes of the environment and provides important data support for subsequent simulation and visualization. Dynamic structure evolution simulation and environmental attribute rendering help the system simulate and display the dynamic changes of the environment and provide a vivid scene perception experience. The generated dynamic attribute rendering scene model can present a more realistic and vivid environmental scene to the user and enhance the user's virtual reality experience. Dynamic distortion compensation and three-dimensional scene visualization processing help improve the realism and fidelity of scene rendering and enhance the user's perception and experience of the environment. The scene rendering visualization model can present a more realistic environmental scene to the user, provide an immersive virtual reality experience, enhance user interaction and perception effects, help users mark other objects in the scene, improve the user's visualization of the surrounding scene, and greatly improve the user's driving safety.

[0013] Preferably, step S1 comprises the following steps:

[0014] Step S11: Acquire an image of the scene in front of the user based on a high-definition camera;

[0015] Step S12: Continuously acquiring multi-source environment real-time perception data using multi-source sensors, wherein the multi-source environment real-time perception data includes environmental wind attribute parameters and environmental temperature attribute parameters;

[0016] Step S13: performing an environmental wind trend characteristic analysis on the environmental wind attribute parameters to generate environmental wind trend characteristics;

[0017] Step S14: Calculating the temperature interval change of the ambient temperature attribute parameter to generate a temperature interval change rate;

[0018] Step S15: Perform scene attribute state analysis on the environmental wind trend characteristics and the temperature interval change rate, and construct a real-time scene attribute state diagram.

[0019] The scene images obtained by the high-definition camera of the present invention have higher clarity and detail display, which is helpful to provide users with a more realistic environmental perception experience. Acquiring high-definition images can provide a more accurate data basis for subsequent environmental analysis and processing. The continuous acquisition of multi-source environmental real-time perception data can provide comprehensive environmental information, which is helpful for the system to fully understand the environmental state around the user. Acquiring environmental wind attributes and temperature attribute parameters can provide rich data support for environmental feature analysis and scene construction. The generation of environmental wind trend characteristics helps the system capture the changing trend of environmental wind direction and wind force, and provides environmental dynamic feature information. Analyzing environmental wind attributes can help the system better understand the climate conditions of the environment and provide important references for the simulation of scene states. The calculation of the temperature interval change rate helps the system understand the rate of change of the ambient temperature, and provides temperature dynamic information for scene modeling and simulation. Analyzing temperature changes can help the system evaluate the thermal conditions of the environment and provide users with a more realistic environmental perception experience. The construction of the real-time scene attribute state diagram combines data such as environmental wind trend characteristics and temperature interval change rate to provide the system with comprehensive environmental state information. The scene attribute state analysis helps the system to have an in-depth understanding of environmental characteristics and provides accurate data support for subsequent three-dimensional visualization and environmental simulation.

[0020] Preferably, the specific steps of step S12 are:

[0021] Perform wind intensity average calculation on the ambient wind attribute parameters to generate an average ambient wind intensity;

[0022] Perform multi-time window division processing on real-time perception data of multi-source environment to obtain multiple time period windows;

[0023] Calculate the wind extreme values ​​of environmental wind attribute parameters based on multiple time period windows and extract the wind extreme values ​​of each time period;

[0024] Identify the wind extreme value change amplitude for each time period and generate multi-time period extreme value change amplitude;

[0025] Perform time series fitting on the extreme value variation amplitudes in multiple periods to construct the wind extreme value variation curve;

[0026] Perform wind direction distribution analysis on environmental wind attribute parameters to generate environmental wind direction distribution data;

[0027] The environmental wind trend characteristics are analyzed on the average environmental wind intensity, wind extreme value change curve and environmental wind direction distribution data to generate environmental wind trend characteristics.

[0028] The present invention helps the system grasp the overall level of environmental wind force by calculating the average value of environmental wind strength and provides a benchmark reference value for environmental wind force. The generation of the average value of environmental wind strength can provide important data support for subsequent wind force analysis and environmental feature identification. Dividing the real-time sensed data into multiple time windows helps the system to observe and analyze the dynamic changes of environmental data in more detail. The multi-time window division processing provides data segmentation in more time dimensions, which helps to capture the time series changes of environmental features. Extracting the extreme wind force of each time period helps the system to identify the extreme wind force conditions in the environment and provides important data support for the analysis of wind force change trends. The calculation of the extreme wind force can help the system to understand the fluctuation range and changes of environmental wind force more comprehensively. Identifying the transition amplitude of the extreme wind force helps the system to analyze the fluctuation degree and change rate of wind force and provide key information on wind force changes. It can help the system understand the changing trend of environmental wind force and provide an important basis for subsequent analysis. The construction of wind extreme value change curve through time series fitting can help the system show the overall trend of wind force change and provide an intuitive data display method. The construction of wind extreme value change curve can help users understand the temporal change law of wind force more intuitively and provide a reference for decision-making. The generation of environmental wind direction distribution data can help the system understand the distribution of wind direction in the environment and provide comprehensive wind direction information support. Wind direction distribution analysis can help the system identify the main wind direction trend in the environment and provide data support for the comprehensive analysis of environmental characteristics. The generation of environmental wind trend characteristics combines data such as wind intensity, wind force change curve and wind direction distribution to provide the system with comprehensive environmental wind direction trend information. The analysis of environmental wind trend characteristics can help the system grasp the dynamic change law of environmental wind direction and provide an important basis for the comprehensive evaluation of environmental characteristics.

[0029] Preferably, the specific steps of step S2 are:

[0030] Step S21: performing global contrast deviation detection on the scene image in front of the user to obtain local contrast difference data;

[0031] Step S22: performing pixel deviation traversal calculation on the local contrast difference data to obtain the pixel deviation value of the difference part;

[0032] Step S23: dynamically adjusting the contrast of the scene image in front of the user based on the pixel deviation value of the difference part to generate a contrast balanced scene image;

[0033] Step S24: identifying edge details of the contrast-balanced scene image and extracting a plurality of image edge detail images;

[0034] Step S25: performing image-by-image adaptive filtering optimization on multiple edge detail images to generate multiple filter-optimized edge detail images;

[0035] Step S26: Perform super-resolution fusion reconstruction on multiple filtered optimized edge detail images to construct a super-resolution scene image.

[0036] The present invention uses global contrast deviation detection, and the system can identify local contrast differences in the image, which is helpful to discover important details in the scene. The acquisition of local contrast difference data can provide an important reference basis for subsequent processing, helping the system to better understand the characteristics of the image. Through pixel deviation traversal calculation, the system can quantify the pixel deviation of the difference part in the image, which is helpful to determine the specific area that needs to be adjusted in the image. The acquisition of the pixel deviation value of the difference part can help the system to carry out subsequent processing in a targeted manner, improve the efficiency and accuracy of image processing, and the dynamic contrast equalization adjustment can achieve targeted image contrast adjustment according to the pixel deviation value of the difference part, improve the visual quality of the image, and generate a scene image with balanced contrast, which can make the details in the image clearer and more prominent, and enhance the user's perception and recognition of the environment. The ability to identify and extract image edge details helps the system capture key features in the scene and improve the image's recognition and realism. The extraction of multiple image edge detail images provides more image parts for optimization for subsequent processing, enhancing the image quality and detail performance. Image-by-image adaptive filtering optimization helps improve the clarity and quality of image edge details and enhance the image's visual effects. The generated multiple filtered optimized edge detail images can make the image details more prominent and natural, improving the user's perception and experience of the scene. Through super-resolution fusion reconstruction, the system can synthesize the optimized edge detail images into high-resolution scene images, providing clearer visual effects. The constructed super-resolution scene images can improve the image's clarity and detail performance, allowing users to obtain a more real and realistic scene perception experience.

[0037] Preferably, step S3 specifically comprises the following steps:

[0038] Step S31: performing object visual recognition on the super-resolution scene image and marking the scene image objects;

[0039] Step S32: performing location calculation on the scene image object to obtain object location data;

[0040] Step S33: performing object bounding box segmentation processing on the super-resolution scene image based on the object positioning data to obtain a plurality of scene object image bounding boxes;

[0041] Step S34: performing road semantic feature analysis on the super-resolution scene image to extract the road semantic features in the scene;

[0042] Step S35: performing accurate correction on the frame positioning of the multiple scene object image bounding boxes according to the semantic features of the roads in the scene, to obtain multiple accurate scene object bounding boxes.

[0043] The present invention can identify various objects in the scene through object visual recognition, provide key information for subsequent processing, enhance the understanding and presentation of the scene, mark objects in the scene image so that users can understand the scene content more intuitively, improve users' perception and interactive experience of the scene, object positioning calculation can accurately determine the position of objects in the scene, provide accurate reference data for subsequent processing, and help further analysis and processing of objects. The obtained object positioning data can help the system better understand the scene structure and improve the accuracy and precision of environmental perception. The object bounding box segmentation processing can clearly identify the bounding box of each object, which is helpful for accurately locating objects and extracting object features. The obtained multiple scene object image bounding boxes can provide clear boundary information for subsequent object analysis and processing, and enhance the interpretability of the scene image. Road semantic feature analysis helps to identify the road structure and features in the scene, improve the understanding and perception of the environment, and the extracted road semantic features can provide important clues for subsequent scene understanding and interaction, and enhance the perception ability of the smart helmet to the environment. The frame positioning precise correction can accurately adjust the position of the object bounding box according to the road semantic features, improve the accuracy and precision of object positioning, and the obtained precise scene object bounding box can accurately identify the position of each object in the scene, and provide more refined information for subsequent scene analysis and interaction.

[0044] Preferably, the specific steps of step S4 are:

[0045] Step S41: performing image frame time sequence fitting on the super-resolution scene image to generate a time sequence scene image frame sequence;

[0046] Step S42: performing inter-frame bounding box correspondence matching on a plurality of precise scene object bounding boxes based on a temporal scene image frame sequence, and identifying bounding box matching data between consecutive frames;

[0047] Step S43: performing image frame optical flow tracking on the bounding box matching data between consecutive frames to obtain bounding box inter-frame optical flow tracking data;

[0048] Step S44: performing optical flow motion change analysis on the bounding box inter-frame optical flow tracking data to generate a scene object motion change trajectory;

[0049] Step S45: Performing spatiotemporal scene structure evolution analysis on the scene object motion change trajectory, thereby obtaining dynamic scene evolution feature data.

[0050] The present invention can track the movement and changes of objects between different frames through bounding box correspondence matching, provide continuous object position information, and identify bounding box matching data between consecutive frames to help understand the motion trajectory of the object and provide basic data for subsequent analysis and visualization. Optical flow tracking can more accurately describe the motion trajectory of the object between consecutive frames and provide more detailed motion information. The optical flow tracking data between bounding box frames can reveal the speed and direction changes of the object and provide more details for the understanding and visualization of dynamic scenes. The optical flow motion change analysis can help capture the motion state and change trend of the object and form the object motion trajectory. The generated scene object motion change trajectory can be used to analyze the dynamic behavior and evolution process of the object and provide a deeper understanding of the spatiotemporal characteristics of the scene. The spatiotemporal scene structure evolution analysis helps to understand the dynamic evolution process of the object in the scene and reveals the spatiotemporal relationship and structural changes of the scene. The obtained dynamic scene evolution feature data can provide rich dynamic effects for three-dimensional visualization, allowing users to perceive the changes and development of the scene more deeply.

[0051] Preferably, the specific steps of step S45 are:

[0052] Extracting upper and lower frame images based on a temporal scene image frame sequence;

[0053] Extract the timestamps of the upper and lower frame images, and calculate the time interval between the upper and lower frame images;

[0054] Performing object displacement analysis on the scene object motion change trajectory within a time period according to the time interval between the upper and lower frame images, and generating object displacement data within the time interval;

[0055] Traverse all time-series scene image frame sequences to generate object displacement data within all time intervals;

[0056] The displacement data of the object in all time intervals are sampled on the time axis, thereby generating the displacement trajectory of the object at different times;

[0057] Analyze the scene structure changes within the time interval for the displacement trajectories of objects at different times, and obtain the scene structure change data of multiple time periods;

[0058] Calculate the optical flow change speed of the scene object's motion change trajectory to generate the change speed of the scene object;

[0059] Based on the change speed of scene objects, the scene structure change data of multiple time periods are dynamically evolved to obtain dynamic scene evolution feature data.

[0060] The present invention can calculate the time interval of each frame of image by extracting the timestamps of the upper and lower frame images, providing a time reference for subsequent motion analysis, and analyzing the displacement of scene objects based on the time interval to obtain the motion trajectory and displacement data of objects in different time periods. By traversing all time-series image frames, complete object motion trajectory data can be generated, laying a foundation for dynamic scene modeling. By analyzing the displacement trajectory of objects in different time periods, structural changes in the scene can be found, such as the appearance of new objects, the disappearance of original objects or changes in position, etc. These scene structure change data can be used to describe the evolution process of the dynamic environment and enhance the perception ability of the three-dimensional visualization system to environmental changes. By calculating the optical flow change speed, the real-time motion speed information of the scene objects can be obtained. These speed data can be associated with the scene structure changes for more detailed description of the evolution law of the dynamic environment. By integrating the information such as the object displacement trajectory, the scene structure changes and the object movement speed, the evolution feature data of the dynamic scene can be constructed. These data can be used to drive the three-dimensional visualization system to generate a more realistic and dynamic environment model and enhance the user's immersion and interactivity.

[0061] Preferably, the specific steps of step S5 are:

[0062] Step S51: performing three-dimensional spatial structure analysis on the super-resolution scene image to generate three-dimensional spatial scene structure data;

[0063] Step S52: performing three-dimensional point cloud modeling on the three-dimensional space scene structure data to construct a three-dimensional space scene model;

[0064] Step S53: performing scene dynamic structure change evolution on the three-dimensional space scene structure data based on the dynamic scene evolution feature data, thereby generating scene dynamic structure evolution data;

[0065] Step S54: using the scene dynamic structure evolution data to simulate the dynamic structure evolution of the three-dimensional space scene model to generate a dynamic three-dimensional scene model;

[0066] Step S55: Rendering the dynamic three-dimensional scene model with environmental attributes using the real-time scene attribute state diagram to generate a dynamic attribute rendering scene model.

[0067] The present invention can more accurately understand the position, size and spatial relationship of objects in the scene through three-dimensional spatial structure analysis, and provide basic data for subsequent modeling. The generated three-dimensional spatial scene structure data is helpful to build a scene model with more stereoscopic and depth sense, and improve the three-dimensional visualization effect. Through three-dimensional point cloud modeling, objects and structures in the scene can be presented in the form of point cloud, and more refined scene modeling can be achieved. The constructed three-dimensional spatial scene model can provide a basis for subsequent dynamic structure evolution, and add more elements to the interactivity and realism of the scene. The dynamic structure change evolution in the scene can simulate the dynamic change process of objects in the scene, making the scene more vivid and realistic. The generated scene dynamic structure evolution data provides a basis for subsequent dynamic simulation, and increases the spatiotemporal characteristics and interactivity of the scene. The dynamic structure evolution simulation can make the three-dimensional scene model present object movement and structural changes, increase the dynamicity and visual appeal of the scene, and the generated dynamic three-dimensional scene model can provide a more vivid scene experience, so that users can better perceive the changes and development of the environment. Environmental attribute rendering can add realistic light and shadow effects and environmental characteristics to the dynamic three-dimensional scene model according to the real-time scene attribute state diagram. The generated dynamic attribute rendering scene model can enhance the realism and visual effects of the scene, and improve the user's immersion and experience quality.

[0068] Preferably, the specific steps of step S6 are:

[0069] Step S61: performing dynamic motion detail distortion detection on the dynamic attribute rendering scene model to identify dynamic detail distortion data of object movement;

[0070] Step S62: locating the distorted detail parts based on the dynamic detail distortion data of the object movement, and marking the distorted detail parts of the model;

[0071] Step S63: dynamically compensating the model's distorted details to generate a distortion-compensated rendering model;

[0072] Step S64: performing three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model;

[0073] Step S65: Visualize the scene rendering visualization model scene to the smart helmet.

[0074] The present invention can timely discover the detail distortion that may occur in the model during the movement of the object through dynamic detail distortion detection, improve the accuracy of scene rendering, identify dynamic detail distortion data to help optimize the rendering process, ensure that the scene presentation is more realistic and smooth, locate the distortion detail part to accurately locate the specific part of the model where the distortion occurs, and provide a positioning basis for subsequent distortion compensation, mark the model distortion detail part to help targeted distortion repair, improve the quality and realism of scene rendering, dynamic distortion compensation can targetedly repair the detail distortion that occurs in the model, improve the detail performance and realism of scene rendering, the generated distortion compensation rendering model can eliminate the distortion that may occur during the movement of the object, and improve the rendering effect, through three-dimensional scene visualization processing, the distortion compensated model can be optimized to enhance the visual effect and realism of the scene, the constructed scene rendering visualization model can better display the dynamic movement and detail performance of objects in the environment, and enhance the visual experience of users, presenting the optimized scene rendering visualization model to the smart helmet can enable users to more intuitively perceive the details and dynamic changes of the environment, and the three-dimensional visualization of environmental perception data based on the smart helmet can provide an immersive scene experience and enhance the user's sense of interaction and participation.

[0075] In this specification, a smart helmet-based environment perception data processing system is provided, which is used to execute the smart helmet-based environment perception data processing method as described above, including:

[0076] The scene attribute module obtains the scene image in front of the user and the real-time perception data of the multi-source environment; performs scene attribute status analysis on the real-time perception data of the multi-source environment and constructs a real-time scene attribute status diagram;

[0077] The image reconstruction module dynamically adjusts the contrast of the scene image in front of the user, and performs super-resolution fusion reconstruction to construct a super-resolution scene image;

[0078] The bounding box segmentation module performs object visual recognition on the super-resolution scene image and performs object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0079] The scene structure evolution module performs optical flow motion change analysis on multiple precise scene object bounding boxes and performs spatiotemporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0080] The dynamic attribute rendering module simulates the dynamic structure evolution of the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, and then performs environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0081] The scene visualization module performs dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; performs three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

[0082] The present invention helps to understand the surrounding environment in real time by acquiring the scene image in front of the user and real-time perception data of multiple sources of environment, improves the user's perception ability and safety, and constructs a real-time scene attribute state diagram. By performing state analysis on the real-time perception data of multiple sources of environment, basic data is provided for subsequent processing, thereby enhancing the intelligence and practicality of the system. Dynamically balancing the contrast of the scene image in front of the user and performing super-resolution fusion reconstruction can improve image quality, enhance the display of scene details and recognition accuracy, and construct super-resolution scene images to help provide clearer and more realistic images, and provide better input for subsequent object recognition and scene structure analysis. Object visual recognition and bounding box segmentation processing on super-resolution scene images helps to accurately identify objects in the scene, and provides accurate object position information for subsequent scene structure analysis. Obtaining multiple precise scene object bounding boxes can help the system understand the scene more finely and improve the accuracy of recognition and analysis. Through the analysis of optical flow motion changes and spatiotemporal scene structure evolution, the system can capture the dynamic changes of objects in the scene, which is helpful for real-time monitoring of scene status and behavior. The dynamic scene evolution feature data obtained can provide a basis for subsequent dynamic attribute rendering, making the scene presentation more vivid and realistic. Dynamic structure evolution simulation of super-resolution scene images based on dynamic scene evolution feature data and real-time scene attribute state diagrams helps to simulate the dynamic changes of objects in the scene and enhance the realism and dynamics of the scene. Generating dynamic attribute rendering scene models can provide more vivid and interactive scene presentation, enhance user experience and immersion. Dynamic distortion compensation and three-dimensional scene visualization processing of dynamic attribute rendering scene models can improve the quality and realism of scene rendering, making the scene more realistic and attractive. Building a scene rendering visualization model helps users better understand and perceive the environment and provide a more intuitive and vivid scene display. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 A schematic diagram of the steps of a method for processing environmental perception data based on a smart helmet according to the present invention;

[0084] Figure 2 Detailed implementation flow chart of step S1;

[0085] Figure 3 Detailed implementation flow chart of step S2;

[0086] Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0087] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0088] The present application example provides an environment perception data processing method and system based on a smart helmet. The execution subject of the environment perception data processing method and system based on a smart helmet includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0089] See also Figures 1 to 4 The present invention provides an environment perception data processing method based on a smart helmet, wherein a high-definition camera and a multi-source sensor are embedded inside the smart helmet. The environment perception data processing method based on the smart helmet comprises the following steps:

[0090] Step S1: obtaining a scene image in front of the user and real-time perception data of a multi-source environment; performing scene attribute state analysis on the real-time perception data of the multi-source environment, and constructing a real-time scene attribute state diagram;

[0091] Step S2: dynamically adjust the contrast of the scene image in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image;

[0092] Step S3: performing object visual recognition on the super-resolution scene image and performing object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0093] Step S4: performing optical flow motion change analysis on multiple precise scene object bounding boxes, and performing spatiotemporal scene structure evolution analysis, thereby obtaining dynamic scene evolution feature data;

[0094] Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, a dynamic structure evolution simulation is performed on the super-resolution scene image, and then environmental attribute rendering is performed to generate a dynamic attribute rendering scene model;

[0095] Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; perform three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

[0096] The present invention can provide the helmet system with an accurate understanding of the state and characteristics of the user's surrounding environment through the construction of a real-time scene attribute state diagram, providing basic data for subsequent processing. The scene attribute state analysis helps the system identify and understand the attributes of the environment, providing key information for the next processing steps. The construction of super-resolution scene images improves the clarity and details of the image, and enhances the user's perception and understanding of the environment. Dynamic contrast equalization adjustment helps to enhance the visual effect of the image, making the details clearer and improving the user experience. The acquisition of precise scene object boundary boxes helps the system accurately identify and locate objects in the environment, providing important information for subsequent scene analysis and processing. Object visual recognition and segmentation processing improves the system's ability to understand objects in the environment, providing a basis for subsequent analysis and simulation. The acquisition of dynamic scene evolution feature data helps the system capture the movement and Spatiotemporal changes provide richer scene information. Spatiotemporal scene structure evolution analysis helps the system understand the dynamic changes of the environment and provides important data support for subsequent simulation and visualization. Dynamic structure evolution simulation and environmental attribute rendering help the system simulate and display the dynamic changes of the environment and provide a vivid scene perception experience. The generated dynamic attribute rendering scene model can present a more realistic and vivid environmental scene to the user and enhance the user's virtual reality experience. Dynamic distortion compensation and three-dimensional scene visualization processing help improve the realism and fidelity of scene rendering and enhance the user's perception and experience of the environment. The scene rendering visualization model can present a more realistic environmental scene to the user, provide an immersive virtual reality experience, enhance user interaction and perception effects, help users mark other objects in the scene, improve the user's visualization of the surrounding scene, and greatly improve the user's driving safety.

[0097] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of the steps of a method for processing environmental perception data based on a smart helmet of the present invention. In this example, the steps of the method include:

[0098] Step S1: obtaining a scene image in front of the user and real-time perception data of a multi-source environment; performing scene attribute state analysis on the real-time perception data of the multi-source environment, and constructing a real-time scene attribute state diagram;

[0099] In this embodiment, based on the high-definition camera inside the smart helmet to capture the scene image in front of the user, a mobile device (such as a smart phone, smart glasses) or a fixed camera can be selected, and the scene image is captured in real time using a camera interface (such as OpenCV or GStreamer), and an appropriate frame rate (such as 30fps) is set to ensure the continuity of the image, and the captured image data is converted into a format suitable for subsequent processing (such as RGB or grayscale), and stored in the memory for subsequent use, and a variety of sensors (such as temperature sensors, humidity sensors, laser radar (LiDAR), IMU (inertial measurement unit), etc.) are integrated to obtain multi-dimensional environmental data, and data is read from the sensor using a suitable interface (such as I2C, UART, CAN, etc.) to ensure data The sampling frequency is synchronized with image capture, and a suitable data fusion algorithm (such as Kalman filtering, particle filtering or multi-sensor fusion algorithm) is selected to fuse data from different sensors to extract more accurate environmental status information. According to the extracted attribute status, real-time evaluation is performed to generate scene status indicators (such as safety and comfort), and the structure of the state diagram is designed, including the visualization method of each attribute (such as heat map, bar chart, icon, etc.). The extracted attribute status is displayed in a graphical way using a visualization library (such as Matplotlib, D3.js or OpenGL), ensuring that the graphics are updated synchronously with the real-time data, and setting a real-time update mechanism so that the scene attribute status diagram can instantly reflect the latest status when the data changes, ensuring that users obtain the latest information.

[0100] Step S2: dynamically adjust the contrast of the scene image in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image;

[0101] In this embodiment, a histogram equalization method (such as cv2.equalizeHist in OpenCV) is used to calculate the global contrast, obtain the brightness distribution of the image, and use local contrast calculation (such as local histogram equalization or CLAHE) to perform equalization processing on the local area of ​​the image. This step can enhance the area with rich details in the image, calculate the required contrast gain coefficient, usually based on the comparison of global and local contrast, and use a nonlinear function (such as Gamma correction) for adjustment. The calculated gain coefficient is applied to adjust the original image pixel by pixel to generate an image after contrast equalization, and select a suitable super-resolution reconstruction algorithm. Commonly used ones include SRGAN (Super-Resolution Generative Adversarial Network) based on convolutional neural network (CNN) or EDSR (Enhanced Deep Residual Networks), perform necessary preprocessing on the contrast-equalized image, such as resizing and normalization, to meet the input requirements of the super-resolution model, input the preprocessed image into the super-resolution model, perform the image reconstruction process, the model will generate a high-resolution image, restore details and textures, if there are multiple source images (such as images at different angles or at different time points), use image fusion technology (such as weighted averaging or image stitching) to synthesize multiple super-resolution images into a unified high-resolution scene image, save the generated super-resolution scene image as a new image file (such as PNG or JPEG), and prepare for subsequent processing.

[0102] Step S3: performing object visual recognition on the super-resolution scene image and performing object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0103] In this embodiment, a pre-trained object recognition model is loaded using a deep learning framework (such as TensorFlow or PyTorch), ensuring that the model has been trained on a suitable data set (such as COCO or Pascal VOC), and the pre-processed super-resolution image is input into the object recognition model to obtain the output of the model, which generally includes: identifying the category of the object in the image, indicating the credibility of the model for each recognition result, the bounding box position of each object, generally represented by the coordinates of the upper left corner and the lower right corner (x_min, y_min, x_max, y_max), extracting the bounding box coordinates and the corresponding category information from the output of the object recognition model, setting a threshold according to the confidence score, and filtering out bounding boxes above the threshold to reduce false detections. For example, the threshold is set to 0.5, and only recognition results with a confidence level higher than 50% are retained. In order to process overlapping bounding boxes, a non-maximum suppression algorithm is applied to retain the bounding box with the highest confidence and remove other boxes with high overlap. This process can be achieved by calculating IoU (Intersection over Union).

[0104] Step S4: performing optical flow motion change analysis on multiple precise scene object bounding boxes, and performing spatiotemporal scene structure evolution analysis, thereby obtaining dynamic scene evolution feature data;

[0105] In this embodiment, a suitable optical flow calculation algorithm is selected, such as the Lucas-Kanade method or the Horn-Schunck method, the optical flow field in the image sequence is calculated, a continuous scene image sequence is obtained, and it is ensured that there is a time interval between the images (for example, the previous and next frames), the continuous image frames are processed, the optical flow vector of each pixel is calculated to represent its movement in time, the optical flow features in each bounding box are recorded, including speed, direction and change amplitude, the optical flow information in each bounding box is analyzed, the motion change features of the object, such as acceleration, impact force, etc., are extracted, the optical flow features of the object in different time frames are compared, the dynamic change pattern is identified, the movement trend is recorded, the motion change features obtained from the optical flow analysis are integrated with the time information of the scene, a spatiotemporal data model is established, a suitable spatiotemporal evolution model is selected, such as a state space model or a dynamic Bayesian network, to describe the change of the scene, dynamic scene evolution feature data including the relative position, speed change and interaction of the object are extracted from the trained spatiotemporal evolution model, and the extracted dynamic scene evolution feature data is organized into structured data for subsequent analysis and application.

[0106] Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, a dynamic structure evolution simulation is performed on the super-resolution scene image, and then environmental attribute rendering is performed to generate a dynamic attribute rendering scene model;

[0107] In this embodiment, a real-time scene attribute state diagram is imported to ensure that the environmental attributes in the diagram can be used for rendering, the initial state of the object is set in the simulation environment, the position, speed and state of the object are updated frame by frame according to the dynamic evolution feature data, the continuity and authenticity of the motion are ensured, the interaction and collision between objects are processed, and the objects are ensured to move naturally in a dynamic environment. According to the real-time scene attribute state diagram, rendering parameters are configured to ensure that the real-time scene attribute state diagram can be updated synchronously with the rendering process. For example, when the ambient temperature changes, the lighting and color in the scene are adjusted accordingly. Real-time rendering is performed using GPU acceleration technology to ensure that the dynamic attribute rendering scene model can be presented smoothly, shadows, reflections and other visual effects are processed to enhance the realism of the scene, and the rendered dynamic attribute rendering scene model is saved as a high-quality image or video file (such as PNG, JPEG or MP4) for easy display and analysis. The user interaction function is tested in the generated scene model to ensure that the user can navigate and interact in the dynamic scene, and the reliability and accuracy of the model are verified.

[0108] Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; perform three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

[0109] In this embodiment, the dynamic rendering model is compared with the high-definition reference image, and whether the contour of the object in the model is clearly defined is analyzed. The semantic segmentation technology is applied to extract the dynamic object area from the image. For each extracted dynamic object area, a deep convolutional neural network-based image super-resolution method is used to reconstruct the object contour details, and the high-definition object reconstruction result after super-resolution is fused and replaced with the corresponding area of ​​the original three-dimensional dynamic rendering model to achieve edge detail compensation. For areas with continuous subtle changes (such as faces, fingers, etc.), multi-time frame data is collected for modeling, and the time series model is used to reconstruct the continuous changes in time of these areas. The 3D model with added dynamic details is imported into a virtual scene rendering engine (such as Unreal Engine) for realistic rendering, and the rendering result scene is embodied using augmented reality technology. The virtual scene is synchronously rendered into the real world using a head-mounted device to form a three-dimensional mixed reality effect, and new data is continuously collected to continuously optimize the model to improve the accuracy of detail reconstruction and dynamic restoration.

[0110] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0111] Step S11: Acquire an image of the scene in front of the user based on a high-definition camera;

[0112] Step S12: Continuously acquiring multi-source environment real-time perception data using multi-source sensors, wherein the multi-source environment real-time perception data includes environmental wind attribute parameters and environmental temperature attribute parameters;

[0113] Step S13: performing an environmental wind trend characteristic analysis on the environmental wind attribute parameters to generate environmental wind trend characteristics;

[0114] Step S14: Calculating the temperature interval change of the ambient temperature attribute parameter to generate a temperature interval change rate;

[0115] Step S15: Perform scene attribute state analysis on the environmental wind trend characteristics and the temperature interval change rate, and construct a real-time scene attribute state diagram.

[0116] In this embodiment, based on the high-definition camera on the smart head, the frame rate is set to 30 frames per second, continuous scene images are continuously acquired, the acquired image data is stored in the local storage in JPEG format, the file is named using a timestamp, the temperature sensor and the anemometer are combined to ensure that they can transmit data in real time, the sensor is set to collect data once a second, the ambient wind speed, wind direction and temperature are recorded, the collected wind attribute parameters and temperature parameters are stored in real time in CSV format, the wind attribute data is cleaned, abnormal values ​​exceeding the threshold (such as wind speed greater than 50m / s) are removed, and the moving average method is used to calculate the wind speed and temperature in the past 10 minutes. The wind direction average is used to generate wind speed trend characteristics (such as rising, falling, and stable) and store them in a structured form in the database. The temperature data is organized into a time series format to ensure that the timestamps of the data are continuous. The temperature difference between each pair of adjacent time points is calculated to generate the temperature interval change rate (such as changes per minute). The environmental wind trend characteristics and temperature change rate are integrated into a unified data set. The decision tree algorithm is used to analyze the comprehensive data and determine the current environmental status (such as "strong wind and warm"). Based on the analysis results, a real-time scene attribute status diagram is generated, which is visualized using a chart library (such as Matplotlib) and exported to PNG format.

[0117] In this embodiment, the specific steps of step S12 are:

[0118] Perform wind intensity average calculation on the ambient wind attribute parameters to generate an average ambient wind intensity;

[0119] Perform multi-time window division processing on real-time perception data of multi-source environment to obtain multiple time period windows;

[0120] Calculate the wind extreme values ​​of environmental wind attribute parameters based on multiple time period windows and extract the wind extreme values ​​of each time period;

[0121] Identify the wind extreme value change amplitude for each time period and generate multi-time period extreme value change amplitude;

[0122] Perform time series fitting on the extreme value variation amplitudes in multiple periods to construct the wind extreme value variation curve;

[0123] Perform wind direction distribution analysis on environmental wind attribute parameters to generate environmental wind direction distribution data;

[0124] The environmental wind trend characteristics are analyzed on the average environmental wind intensity, wind extreme value change curve and environmental wind direction distribution data to generate environmental wind trend characteristics.

[0125] In this embodiment, real-time environmental wind speed data is collected through multiple environmental sensing devices (such as wind speed sensors), and the arithmetic average calculation is performed on the multiple collected wind speed data points to obtain the average value of the environmental wind strength. This average wind speed can reflect the overall wind strength of the current environment. The time window length (such as every hour, every minute) is determined, and the start and end time of the time window are set. According to the set time window, the environmental wind attribute parameter data is divided into multiple time period windows, and the maximum and minimum value calculation methods are selected to ensure that the wind extreme value of each time period can be obtained. The data of each time period window is analyzed, the maximum and minimum wind speeds in the time period are extracted, and the wind extreme values ​​of adjacent time periods are calculated. The change amplitude between the two periods is used to generate the extreme value change data of multiple periods, and the extreme value change data is fitted in time series to construct the change curve of the wind extreme value. This analysis can find the change trend and fluctuation characteristics of the environmental wind force, select the appropriate fitting model (such as linear regression, spline interpolation) to fit the extreme value change amplitude of multiple periods, use the fitting algorithm to analyze the change amplitude, generate the wind extreme value change curve, visualize the fitting results, save them as graphic files (such as PNG format), and perform correlation analysis on the environmental wind intensity average value, wind extreme value change curve and wind direction distribution data obtained above. Based on these data, the overall trend characteristics of the environmental wind force are summarized, such as wind strength changes, wind direction changes, etc. This comprehensive analysis result can more comprehensively describe the wind conditions of the current environment.

[0126] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0127] Step S21: performing global contrast deviation detection on the scene image in front of the user to obtain local contrast difference data;

[0128] Step S22: performing pixel deviation traversal calculation on the local contrast difference data to obtain the pixel deviation value of the difference part;

[0129] Step S23: dynamically adjusting the contrast of the scene image in front of the user based on the pixel deviation value of the difference part to generate a contrast balanced scene image;

[0130] Step S24: identifying edge details of the contrast-balanced scene image and extracting a plurality of image edge detail images;

[0131] Step S25: performing image-by-image adaptive filtering optimization on multiple edge detail images to generate multiple filter-optimized edge detail images;

[0132] Step S26: Perform super-resolution fusion reconstruction on multiple filtered optimized edge detail images to construct a super-resolution scene image.

[0133] In this embodiment, the scene image in front of the user is read to ensure that the image format is a format suitable for processing (such as RGB), the global contrast of the image is calculated, and the histogram equalization method is usually used to obtain the global brightness distribution. The contrast of each window is calculated by means of a local window (such as 3x3 or 5x5), and compared with the global contrast to generate local contrast difference data. According to the local contrast difference data, the area with significant contrast difference is identified, the pixel traversal of the identified difference area is performed, the deviation value of the brightness of each pixel and the average brightness of the area is calculated, the deviation value of each pixel is recorded, and the difference part pixel deviation value data is generated. According to the difference part pixel deviation value, the contrast of different areas in the image is adjusted so that the area with low contrast is enhanced to generate a contrast balanced scene image, and the edge is used. An edge detection algorithm (such as Canny edge detection, Sobel operator, etc.) is used to process the contrast-balanced scene image, identify and extract edge details in the image, save the detected edge details as multiple images for subsequent processing, select a suitable adaptive filtering algorithm (such as adaptive median filtering, bilateral filtering, etc.) to reduce noise and retain edge details, apply adaptive filtering to each edge detail image, optimize the edge performance of each image, generate multiple filter-optimized edge detail images, select a suitable super-resolution reconstruction algorithm (such as SRGAN based on deep learning or traditional interpolation methods), input multiple filter-optimized edge detail images into the super-resolution reconstruction algorithm, generate a high-resolution scene image, and save the constructed super-resolution scene image as a high-quality image file (such as TIFF or PNG).

[0134] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0135] Step S31: performing object visual recognition on the super-resolution scene image and marking the scene image objects;

[0136] Step S32: performing location calculation on the scene image object to obtain object location data;

[0137] Step S33: performing object bounding box segmentation processing on the super-resolution scene image based on the object positioning data to obtain a plurality of scene object image bounding boxes;

[0138] Step S34: performing road semantic feature analysis on the super-resolution scene image to extract the road semantic features in the scene;

[0139] Step S35: performing accurate correction on the frame positioning of the multiple scene object image bounding boxes according to the semantic features of the roads in the scene, to obtain multiple accurate scene object bounding boxes.

[0140] In this embodiment, the object detection algorithm using deep learning (such as YOLO, Faster R-CNN, etc.) are used to identify and analyze super-resolution scene images. The algorithm can detect various objects in the image and mark their position coordinates in the image. According to the object positioning data obtained in the previous step, the bounding boxes of each object are drawn on the super-resolution scene image. Using image segmentation technology, the boundary area of ​​each object can be accurately extracted to form an independent object image. The bounding box information of each object is extracted from the object positioning data. According to the bounding box coordinates, multiple scene object images are cropped from the super-resolution scene image to generate independent object images. Each cropped object image is saved as a separate file, and the naming method includes object category and position coordinates. A suitable semantic segmentation model (such as DeepLab, SegNet) is selected to perform road semantic analysis on the super-resolution scene image. The semantic segmentation model is run to extract the road area and its features (such as road type and width) in the image. The extracted road semantic features are stored in a structured form to ensure that detailed information of the road features is recorded. The extracted road semantic features are matched with the object bounding box, and the relative position relationship between the object and the road is analyzed. The bounding box of each object is adjusted using a positioning precision correction algorithm (such as the least squares method) to ensure that it is consistent with the road features.

[0141] In this embodiment, step S4 includes the following steps:

[0142] Step S41: performing image frame time sequence fitting on the super-resolution scene image to generate a time sequence scene image frame sequence;

[0143] Step S42: performing inter-frame bounding box correspondence matching on a plurality of precise scene object bounding boxes based on a temporal scene image frame sequence, and identifying bounding box matching data between consecutive frames;

[0144] Step S43: performing image frame optical flow tracking on the bounding box matching data between consecutive frames to obtain bounding box inter-frame optical flow tracking data;

[0145] Step S44: performing optical flow motion change analysis on the optical flow tracking data between the bounding box frames to generate a motion change trajectory of the scene object;

[0146] Step S45: Performing spatiotemporal scene structure evolution analysis on the scene object motion change trajectory, thereby obtaining dynamic scene evolution feature data.

[0147] In this embodiment, continuous frames are extracted from the super-resolution scene image to ensure that the frame rate is consistent, a suitable time-series fitting method (such as linear interpolation or polynomial fitting) is selected, the image frames are processed, a time-series scene image frame sequence is generated, a smooth transition between each frame is ensured, and the image is saved in a video or image sequence format. Multiple precise scene object bounding box data are extracted from each frame of the image, and a matching algorithm (such as the Hungarian algorithm or IoU-based matching) is used to perform corresponding matching on the bounding boxes between continuous frames, identify and record the bounding box matching data between continuous frames, including the object ID and the corresponding frame information, and select a suitable optical flow algorithm (such as Lucas-Kanade or Farneback) calculates the optical flow within the bounding box, calculates the optical flow in the continuous frames for each matched bounding box, and obtains the optical flow tracking data between the bounding box frames. Combining the aforementioned bounding box matching and optical flow tracking data, the motion change trajectory of each object in the scene can be reconstructed. By analyzing these trajectory data, the object's motion speed, acceleration, motion direction and other features can be extracted. The object's motion change trajectory is correlated with the overall spatiotemporal structure change of the scene. By comparing the scene changes in different time periods, the evolution law of the dynamic scene is discovered, and the dynamic evolution feature data of the scene objects, including motion patterns and change trends, are extracted.

[0148] In this embodiment, the specific steps of step S45 are:

[0149] Extracting upper and lower frame images based on a temporal scene image frame sequence;

[0150] Extract the timestamps of the upper and lower frame images, and calculate the time interval between the upper and lower frame images;

[0151] Performing object displacement analysis on the scene object motion change trajectory within a time period according to the time interval between the upper and lower frame images, and generating object displacement data within the time interval;

[0152] Traverse all time-series scene image frame sequences to generate object displacement data within all time intervals;

[0153] The displacement data of the object in all time intervals are sampled on the time axis, thereby generating the displacement trajectory of the object at different times;

[0154] Analyze the scene structure changes within the time interval for the displacement trajectories of objects at different times, and obtain the scene structure change data of multiple time periods;

[0155] Calculate the optical flow change speed of the scene object's motion change trajectory to generate the change speed of the scene object;

[0156] Based on the change speed of scene objects, the scene structure change data of multiple time periods are dynamically evolved to obtain dynamic scene evolution feature data.

[0157] In this embodiment, two adjacent upper and lower frames of images are extracted from the time-series scene image frame sequence, and the time interval between them is calculated by analyzing the timestamps of the two frames of images. The displacement of each object in the time interval between the upper and lower frames of images is analyzed by using the previously obtained object motion change trajectory data, and the entire time-series image sequence is traversed to generate object displacement data in all time intervals. The object displacement data in all time intervals obtained above are sampled on the time axis, so that the displacement trajectory of each object at different time points can be extracted, and the change of the object displacement trajectory in different time periods can be analyzed to deduce the dynamic change of the overall structure of the scene. For example, if the position of the object changes significantly in a certain time period, it can be inferred that the scene structure has changed significantly in this period. This analysis result can reflect the structural evolution characteristics of the scene in the time dimension. The motion change speed of the scene objects is calculated by using the previously obtained optical flow change data, and the change speed information is associated with the scene structure change data obtained by the previous analysis, so as to deduce the evolution characteristics of the entire dynamic scene. This spatiotemporal correlation analysis can more comprehensively characterize the interaction process between the object movement and the environmental change in the scene.

[0158] In this embodiment, the specific steps of step S5 are:

[0159] Step S51: performing three-dimensional spatial structure analysis on the super-resolution scene image to generate three-dimensional spatial scene structure data;

[0160] Step S52: performing three-dimensional point cloud modeling on the three-dimensional space scene structure data to construct a three-dimensional space scene model;

[0161] Step S53: performing scene dynamic structure change evolution on the three-dimensional space scene structure data based on the dynamic scene evolution feature data, thereby generating scene dynamic structure evolution data;

[0162] Step S54: using the scene dynamic structure evolution data to simulate the dynamic structure evolution of the three-dimensional space scene model to generate a dynamic three-dimensional scene model;

[0163] Step S55: Rendering the dynamic three-dimensional scene model with environmental attributes using the real-time scene attribute state diagram to generate a dynamic attribute rendering scene model.

[0164] In this embodiment, feature points and key areas are extracted from the super-resolution scene image, corner detection is performed using a computer vision library (such as OpenCV) (for example, Harris corner detection or Shi-Tomasi algorithm is used), and SIFT (Scale Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF) algorithm is used to describe and match the feature points to ensure that the feature points under different viewing angles can correspond correctly. Stereo vision technology is used to generate three-dimensional point cloud data through dual-view or multi-view geometric calculations using triangulation. Coordinate transformation is performed through camera intrinsic and extrinsic parameters to obtain three-dimensional space scene structure data. RANSAC (Random Sampling Consensus Algorithm) is used to model the three-dimensional point cloud, extract geometric shapes such as planes and spheres, enhance the quality of the point cloud, and use voxel grid filtering (Voxel Grid Filter) reduces the density of the point cloud, removes noise, retains important features, and stores the processed 3D point cloud data in PLY or OBJ format. These formats support rich geometric information and color information, which is convenient for subsequent 3D modeling and visualization. The previously generated dynamic scene evolution feature data (such as object motion trajectory and change trend) is input into the analysis model. Use physics-based simulation (such as particle system) or machine learning model (such as LSTM or RNN) to analyze the 3D space scene structure data, predict how objects change over time, generate dynamic structure evolution data, record the state changes of objects at different time points, including position, shape, size and other information, configure Unity or Unreal Engine and other 3D development environments, import the constructed 3D space model, use the physics engine (such as NVIDIAPhysX) to perform dynamic simulation according to the dynamic structure evolution data, simulate the movement and interaction effects of objects in the scene, use GPU acceleration technology to update the status of objects in the scene in real time, generate a dynamic 3D scene model, ensure that changes in the scene can be reflected in real time, import the real-time scene attribute state diagram (such as temperature, humidity, lighting, etc.) into the rendering engine, configure material properties (such as reflection, refraction), lighting models (such as Phong or Blinn-Phong) and shadow effects (such as shadow maps) to ensure the authenticity of the rendering effect, use ray tracing technology (Ray Tracing) or rasterization technology (Rasterization) for final rendering, generate a dynamic attribute rendering scene model, and save it as a high-quality image (PNG, JPEG) or video format (MP4).

[0165] In this embodiment, the specific steps of step S6 are:

[0166] Step S61: performing dynamic motion detail distortion detection on the dynamic attribute rendering scene model to identify dynamic detail distortion data of object movement;

[0167] Step S62: locating the distorted detail parts based on the dynamic detail distortion data of the object movement, and marking the distorted detail parts of the model;

[0168] Step S63: dynamically compensating the model's distorted details to generate a distortion-compensated rendering model;

[0169] Step S64: performing three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model;

[0170] Step S65: Visualize the scene rendering visualization model scene to the smart helmet.

[0171] In this embodiment, the optical flow method is used to analyze the dynamic attribute rendering scene model to detect the distortion phenomenon that occurs when the object moves. Specifically, the Lucas-Kanade optical flow algorithm can be used to calculate the motion between adjacent frames, and a distortion metric standard (such as mean square error MSE or structural similarity index SSIM) is defined to evaluate the difference between the moving details and the expected state. By comparing the dynamic attributes of different frames, a significant distortion area is identified, and the detected distortion data is recorded in a data structure, including the position, type and degree of the distortion area. The spatial coordinates and boundary information of all the distortion details are extracted. In the three-dimensional scene model, a visualization tool (such as OpenGL or Unity) is used to mark the distorted details. Different colors or shapes can be used. Mark to highlight the distorted area for subsequent processing, store the marked distorted details in the database to ensure that the information can be quickly accessed during subsequent compensation processing, select a suitable distortion compensation algorithm (such as adaptive filters, deep learning methods or physically based models) to repair the distorted area, consider using a convolutional neural network (CNN) for image compensation, process each marked distorted detail, and generate a distortion compensation rendering model. This process includes interpolating the lost details or filling them with surrounding pixel information. By comparing the models before and after compensation, verify the compensation effect and ensure the coherence and consistency of dynamic details. Use professional 3D visualization tools (such as Blender, Unity or Unreal Engine) to create a 3D rendering model. Engine) processes the distortion compensation rendering model, imports the compensated model into the visualization environment, configures the scene's lighting, material, and camera perspective to ensure the realism and aesthetics of the scene, configures rendering parameters (such as resolution, anti-aliasing, shadow effects, etc.), generates a high-quality 3D scene visualization model, ensures that the scene visualization model is compatible with the interface of the smart helmet, uses SDK or API for connection (such as Oculus SDK or HTC Vive SDK), transmits the constructed scene visualization model to the smart helmet via wireless or wired means, ensures the stability and real-time nature of data transmission, performs real-time rendering in the smart helmet, ensures that users can experience high-quality 3D scenes in a dynamic environment, and realizes interactive functions.

[0172] In this embodiment, a smart helmet-based environment perception data processing system is provided, which is used to execute the smart helmet-based environment perception data processing method as described above, including:

[0173] The scene attribute module obtains the scene image in front of the user and the real-time perception data of the multi-source environment; performs scene attribute status analysis on the real-time perception data of the multi-source environment and constructs a real-time scene attribute status diagram;

[0174] The image reconstruction module dynamically adjusts the contrast of the scene image in front of the user, and performs super-resolution fusion reconstruction to construct a super-resolution scene image;

[0175] The bounding box segmentation module performs object visual recognition on the super-resolution scene image and performs object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0176] The scene structure evolution module performs optical flow motion change analysis on multiple precise scene object bounding boxes and performs spatiotemporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0177] The dynamic attribute rendering module simulates the dynamic structure evolution of the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, and then performs environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0178] The scene visualization module performs dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; performs three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

[0179] The present invention helps to understand the surrounding environment in real time by acquiring the scene image in front of the user and real-time perception data of multiple sources of environment, improves the user's perception ability and safety, and constructs a real-time scene attribute state diagram. By performing state analysis on the real-time perception data of multiple sources of environment, basic data is provided for subsequent processing, thereby enhancing the intelligence and practicality of the system. Dynamically balancing the contrast of the scene image in front of the user and performing super-resolution fusion reconstruction can improve image quality, enhance the display of scene details and recognition accuracy, and construct super-resolution scene images to help provide clearer and more realistic images, and provide better input for subsequent object recognition and scene structure analysis. Object visual recognition and bounding box segmentation processing on super-resolution scene images helps to accurately identify objects in the scene, and provides accurate object position information for subsequent scene structure analysis. Obtaining multiple precise scene object bounding boxes can help the system understand the scene more finely and improve the accuracy of recognition and analysis. Through the analysis of optical flow motion changes and spatiotemporal scene structure evolution, the system can capture the dynamic changes of objects in the scene, which is helpful for real-time monitoring of scene status and behavior. The dynamic scene evolution feature data obtained can provide a basis for subsequent dynamic attribute rendering, making the scene presentation more vivid and realistic. Dynamic structure evolution simulation of super-resolution scene images based on dynamic scene evolution feature data and real-time scene attribute state diagrams helps to simulate the dynamic changes of objects in the scene and enhance the realism and dynamics of the scene. Generating dynamic attribute rendering scene models can provide more vivid and interactive scene presentation, enhance user experience and immersion. Dynamic distortion compensation and three-dimensional scene visualization processing of dynamic attribute rendering scene models can improve the quality and realism of scene rendering, making the scene more realistic and attractive. Building a scene rendering visualization model helps users better understand and perceive the environment and provide a more intuitive and vivid scene display.

[0180] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0181] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for processing environmental perception data based on a smart helmet, characterized in that: The smart helmet is embedded with a high-definition camera and multi-source sensors, including the following steps: Step S1: obtaining a scene image in front of the user and real-time perception data of a multi-source environment; performing scene attribute state analysis on the real-time perception data of the multi-source environment, and constructing a real-time scene attribute state diagram; Step S2: dynamically adjust the contrast of the scene image in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image; Step S3: performing object visual recognition on the super-resolution scene image and performing object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes; Step S4: performing optical flow motion change analysis on multiple precise scene object bounding boxes, and performing spatiotemporal scene structure evolution analysis, thereby obtaining dynamic scene evolution feature data; Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, a dynamic structure evolution simulation is performed on the super-resolution scene image, and then environmental attribute rendering is performed to generate a dynamic attribute rendering scene model; Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; perform three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

2. The method for processing environmental perception data based on a smart helmet according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: Acquire an image of the scene in front of the user based on a high-definition camera; Step S12: Continuously acquiring multi-source environment real-time perception data using multi-source sensors, wherein the multi-source environment real-time perception data includes environmental wind attribute parameters and environmental temperature attribute parameters; Step S13: performing an environmental wind trend characteristic analysis on the environmental wind attribute parameters to generate environmental wind trend characteristics; Step S14: Calculating the temperature interval change of the ambient temperature attribute parameter to generate a temperature interval change rate; Step S15: Perform scene attribute state analysis on the environmental wind trend characteristics and the temperature interval change rate, and construct a real-time scene attribute state diagram.

3. The method for processing environmental perception data based on a smart helmet according to claim 2, characterized in that: The specific steps of step S12 are: Perform wind intensity average calculation on the ambient wind attribute parameters to generate an average ambient wind intensity; Perform multi-time window division processing on real-time perception data of multi-source environment to obtain multiple time period windows; Calculate the wind extreme values ​​of environmental wind attribute parameters based on multiple time period windows and extract the wind extreme values ​​of each time period; Identify the wind extreme value change amplitude for each time period and generate multi-time period extreme value change amplitude; Perform time series fitting on the extreme value variation amplitudes in multiple periods to construct the wind extreme value variation curve; Perform wind direction distribution analysis on environmental wind attribute parameters to generate environmental wind direction distribution data; The environmental wind trend characteristics are analyzed on the average environmental wind intensity, wind extreme value change curve and environmental wind direction distribution data to generate environmental wind trend characteristics.

4. The method for processing environmental perception data based on a smart helmet according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing global contrast deviation detection on the scene image in front of the user to obtain local contrast difference data; Step S22: performing pixel deviation traversal calculation on the local contrast difference data to obtain the pixel deviation value of the difference part; Step S23: dynamically adjusting the contrast of the scene image in front of the user based on the pixel deviation value of the difference part to generate a contrast balanced scene image; Step S24: identifying edge details of the contrast-balanced scene image, and extracting a plurality of image edge detail images; Step S25: performing image-by-image adaptive filtering optimization on multiple edge detail images to generate multiple filter-optimized edge detail images; Step S26: Perform super-resolution fusion reconstruction on multiple filtered optimized edge detail images to construct a super-resolution scene image.

5. The method for processing environmental perception data based on a smart helmet according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: performing object visual recognition on the super-resolution scene image and marking the scene image objects; Step S32: performing location calculation on the scene image object to obtain object location data; Step S33: performing object bounding box segmentation processing on the super-resolution scene image based on the object positioning data to obtain a plurality of scene object image bounding boxes; Step S34: performing road semantic feature analysis on the super-resolution scene image to extract the road semantic features in the scene; Step S35: performing accurate correction on the frame positioning of the multiple scene object image bounding boxes according to the semantic features of the roads in the scene, to obtain multiple accurate scene object bounding boxes.

6. The method for processing environmental perception data based on a smart helmet according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: performing image frame time sequence fitting on the super-resolution scene image to generate a time sequence scene image frame sequence; Step S42: performing inter-frame bounding box correspondence matching on a plurality of precise scene object bounding boxes based on a temporal scene image frame sequence, and identifying bounding box matching data between consecutive frames; Step S43: performing image frame optical flow tracking on the bounding box matching data between consecutive frames to obtain bounding box inter-frame optical flow tracking data; Step S44: performing optical flow motion change analysis on the optical flow tracking data between the bounding box frames to generate a motion change trajectory of the scene object; Step S45: Performing spatiotemporal scene structure evolution analysis on the scene object motion change trajectory, thereby obtaining dynamic scene evolution feature data.

7. The method for processing environmental perception data based on a smart helmet according to claim 6, characterized in that: The specific steps of step S45 are: Extracting upper and lower frame images based on a temporal scene image frame sequence; Extract the timestamps of the upper and lower frame images, and calculate the time interval between the upper and lower frame images; Performing object displacement analysis on the scene object motion change trajectory within a time period according to the time interval between the upper and lower frame images, and generating object displacement data within the time interval; Traverse all time-series scene image frame sequences to generate object displacement data within all time intervals; The displacement data of the object in all time intervals are sampled on the time axis, thereby generating the displacement trajectory of the object at different times; Analyze the scene structure changes within the time interval for the displacement trajectories of objects at different times, and obtain the scene structure change data of multiple time periods; Calculate the optical flow change speed of the scene object's motion change trajectory to generate the change speed of the scene object; Based on the change speed of scene objects, the scene structure change data of multiple time periods are dynamically evolved to obtain dynamic scene evolution feature data.

8. The method for processing environmental perception data based on a smart helmet according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: performing three-dimensional spatial structure analysis on the super-resolution scene image to generate three-dimensional spatial scene structure data; Step S52: performing three-dimensional point cloud modeling on the three-dimensional space scene structure data to construct a three-dimensional space scene model; Step S53: performing scene dynamic structure change evolution on the three-dimensional space scene structure data based on the dynamic scene evolution feature data, thereby generating scene dynamic structure evolution data; Step S54: using the scene dynamic structure evolution data to simulate the dynamic structure evolution of the three-dimensional space scene model to generate a dynamic three-dimensional scene model; Step S55: Rendering the dynamic three-dimensional scene model with environmental attributes using the real-time scene attribute state diagram to generate a dynamic attribute rendering scene model.

9. The method for processing environmental perception data based on a smart helmet according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: performing dynamic motion detail distortion detection on the dynamic attribute rendering scene model to identify dynamic detail distortion data of object movement; Step S62: locating the distorted detail parts based on the dynamic detail distortion data of the object movement, and marking the distorted detail parts of the model; Step S63: dynamically compensating the model's distorted details to generate a distortion-compensated rendering model; Step S64: performing three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model; Step S65: Visualize the scene rendering visualization model scene to the smart helmet.

10. An environment perception data processing system based on a smart helmet, characterized in that: The method for processing environmental perception data based on a smart helmet according to claim 1 comprises: The scene attribute module obtains the scene image in front of the user and the real-time perception data of the multi-source environment; performs scene attribute status analysis on the real-time perception data of the multi-source environment and constructs a real-time scene attribute status diagram; The image reconstruction module dynamically adjusts the contrast of the scene image in front of the user, and performs super-resolution fusion reconstruction to construct a super-resolution scene image; The bounding box segmentation module performs object visual recognition on the super-resolution scene image and performs object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes; The scene structure evolution module performs optical flow motion change analysis on multiple precise scene object bounding boxes and performs spatiotemporal scene structure evolution analysis to obtain dynamic scene evolution feature data; The dynamic attribute rendering module simulates the dynamic structure evolution of the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, and then performs environmental attribute rendering to generate a dynamic attribute rendering scene model; The scene visualization module performs dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensated rendering model; performs three-dimensional scene visualization processing on the distortion compensated rendering model to construct a scene rendering visualization model.

Images