Environmental perception data processing method and system based on intelligent helmet

Abstract

The present invention relates to the field of data processing, and in particular, to an environmental perception data processing method and system based on an intelligent helmet. The method includes the following steps: obtaining a scene image in front of the user and multi-source real-time environmental perception data; analyzing the scene attribute states of the multi-source real-time environmental perception data to construct a real-time scene attribute state graph; performing contrast dynamic equalization adjustment on the scene image in front of the user, and performing super-resolution fusion reconstruction to construct a super-resolution scene image; performing object visual recognition on the super-resolution scene image, and performing object bounding box segmentation processing to obtain a plurality of precise scene object bounding boxes; performing optical flow motion change analysis on the plurality of precise scene object bounding boxes, and performing spatio-temporal scene structure evolution analysis, so as to obtain dynamic scene evolution feature data. The present invention improves the data processing efficiency of environmental perception data and more intuitively visualizes the environmental perception data.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and particularly to an environmental perception data processing method and system based on an intelligent helmet. Background Art

[0002] With the continuous development of intelligent transportation technology, driving safety has become a key issue that people focus on. Traditional driving behavior monitoring and environmental perception methods often rely on in-vehicle sensors and monitoring devices, and there are often problems such as low data processing efficiency, poor visibility of environmental perception data, and difficulty in meeting the safety requirements in an increasingly complex traffic environment. In recent years, driving environment perception methods based on wearable devices have attracted wide attention. As an intelligent wearable device, a driving intelligent helmet can integrate multiple sensors to real-time monitor the physiological state of the driver, driving behavior, and various indicators of the surrounding environment, providing a strong guarantee for driving safety.

[0003] However, how to effectively process and analyze the massive environmental perception data obtained from the intelligent helmet, and how to convert these data into actionable decision support have become key issues to be solved urgently. Traditional data processing methods usually rely on manual intervention, with low efficiency and difficulty in coping with complex and changeable traffic environments. Therefore, there is an urgent need for an intelligent environmental perception data processing method for intelligent helmets. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes an environmental perception data processing method and system based on an intelligent helmet to solve at least one of the above technical problems.

[0005] To achieve the above object, the present invention provides an environmental perception data processing method based on an intelligent helmet. A high-definition camera and multi-source sensors are embedded inside the intelligent helmet, and the method includes the following steps:

[0006] Step S1: Obtain the image of the scene in front of the user and multi-source real-time environmental perception data; analyze the scene attribute state of the multi-source real-time environmental perception data to construct a real-time scene attribute state diagram;

[0007] Step S2: Perform contrast dynamic equalization adjustment on the image of the scene in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image;

[0008] Step S3: Perform object visual recognition on the super-resolution scene image, and perform object bounding box segmentation processing to obtain a plurality of accurate scene object bounding boxes;

[0009] Step S4: Perform optical flow motion change analysis on the plurality of accurate scene object bounding boxes, and perform spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0010] Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute status map, perform dynamic structure evolution simulation on the super-resolution scene image, and then perform environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0011] Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensation rendering model; perform three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model.

[0012] Through the construction of the real-time scene attribute status map, the present invention can provide the helmet system with an accurate understanding of the state and characteristics of the user's surrounding environment, providing basic data for subsequent processing. The scene attribute status analysis helps the system identify and understand the attributes of the environment, providing key information for the next processing steps. The construction of the super-resolution scene image improves the clarity and details of the image, enhancing the user's perception and understanding of the environment. The contrast dynamic equalization adjustment helps to enhance the visual effect of the image, making the details clearer and improving the user experience. The acquisition of the precise scene object bounding box helps the system accurately identify and locate objects in the environment, providing important information for subsequent scene analysis and processing. The object visual recognition and segmentation processing improves the system's understanding of objects in the environment, providing a basis for subsequent analysis and simulation. The acquisition of the dynamic scene evolution feature data helps the system capture the movement and spatio-temporal changes of objects in the environment, providing richer scene information. The spatio-temporal scene structure evolution analysis helps the system understand the dynamic changes of the environment, providing important data support for subsequent simulation and visualization. The dynamic structure evolution simulation and environmental attribute rendering help the system simulate and display the dynamic changes of the environment, providing a vivid scene perception experience. The generated dynamic attribute rendering scene model can present a more real and vivid environment scene for the user, enhancing the user's virtual reality experience. The dynamic distortion compensation and three-dimensional scene visualization processing help to improve the realism and fidelity of the scene rendering, enhancing the user's perception and experience of the environment. The scene rendering visualization model can present a more realistic environment scene for the user, providing an immersive virtual reality experience, enhancing the user interaction and perception effect, helping the user mark other objects in the scene, improving the visualization degree of the surrounding scene of the user, and greatly improving the driving safety of the user.

[0013] Preferably, step S1 includes the following steps:

[0014] Step S11: Obtain the user's front scene image based on a high-definition camera;

[0015] Step S12: Continuously obtain multi-source environment real-time perception data by using multi-source sensors, and the multi-source environment real-time perception data includes environmental wind attribute parameters and environmental temperature attribute parameters;

[0016] Step S13: Analyze the environmental wind trend characteristics of the environmental wind attribute parameters to generate environmental wind trend characteristics;

[0017] Step S14: Calculate the temperature interval change of the environmental temperature attribute parameters to generate a temperature interval change rate;

[0018] Step S15: Analyze the scenario attribute status of the environmental wind trend characteristics and the temperature interval change rate, and construct a real-time scenario attribute status map.

[0019] The scenario image obtained by the high-definition camera in the present invention has higher clarity and detail display, which helps to provide users with a more real environmental perception experience. Obtaining high-definition images can provide a more accurate data basis for subsequent environmental analysis and processing. The continuous acquisition of multi-source environmental real-time perception data can provide comprehensive environmental information, which helps the system to comprehensively understand the environmental status around the user. Obtaining environmental wind attribute and temperature attribute parameters can provide rich data support for environmental feature analysis and scenario construction. The generation of environmental wind trend characteristics helps the system to capture the change trends of environmental wind direction and wind force, and provide environmental dynamic feature information. Analyzing environmental wind attributes can help the system better understand the climate conditions of the environment and provide an important reference for the simulation of scenario states. The calculation of the temperature interval change rate helps the system to understand the change rate of environmental temperature and provide temperature dynamic information for scenario modeling and simulation. Analyzing temperature changes can help the system evaluate the thermal conditions of the environment and provide users with a more realistic environmental perception experience. The construction of the real-time scenario attribute status map combines data such as environmental wind trend characteristics and temperature interval change rate, provides comprehensive environmental status information for the system, and scenario attribute status analysis helps the system to deeply understand environmental characteristics and provide accurate data support for subsequent three-dimensional visualization and environmental simulation.

[0020] Preferably, the specific steps of Step S12 are:

[0021] Perform an average calculation of the wind force intensity of the environmental wind attribute parameters to generate an average environmental wind intensity;

[0022] Perform a multi-time window partitioning process on the multi-source environmental real-time perception data to obtain multiple time period windows;

[0023] Based on the multiple time period windows, perform a wind force extreme value calculation on the environmental wind attribute parameters to extract the wind force extreme value of each time period;

[0024] Identify the change amplitude of the wind force extreme value for the wind force extreme value of each time period to generate a multi-period extreme value change amplitude;

[0025] Perform a time series fitting on the multi-period extreme value change amplitude to construct a wind force extreme value change curve;

[0026] Perform wind direction distribution analysis on the environmental wind attribute parameters to generate environmental wind direction distribution data;

[0027] Perform environmental wind trend feature analysis on the average value of environmental wind intensity, the change curve of wind force extreme values, and the environmental wind direction distribution data to generate environmental wind trend features.

[0028] The present invention helps the system grasp the overall level of environmental wind force by calculating the average value of environmental wind force intensity, provides a reference value for the environmental wind force, and the generation of the average value of environmental wind intensity can provide important data support for subsequent wind force analysis and environmental feature recognition. Dividing the real-time perception data into multiple time windows helps the system observe and analyze the dynamic changes of environmental data more carefully. The multi-time window division processing provides more data subdivision in the time dimension, which helps to capture the temporal changes of environmental features. Extracting the extreme wind force values in each time period helps the system identify the extreme wind force conditions in the environment and provides important data support for the analysis of wind force change trends. The calculation of extreme wind force values can help the system understand the fluctuation range and change situation of environmental wind force more comprehensively. Identifying the change amplitude of extreme wind force values helps the system analyze the fluctuation degree and change rate of wind force and provides key information on wind force changes. The generation of the change amplitude of extreme values in multiple time periods can help the system understand the change trend of environmental wind force and provide an important basis for subsequent analysis. Constructing a change curve of extreme wind force values through time series fitting helps the system show the overall trend of wind force changes and provides an intuitive data display method. The construction of the change curve of extreme wind force values can help users understand the temporal change law of wind force more intuitively and provide a reference basis for decision-making. The generation of environmental wind direction distribution data helps the system understand the distribution of wind directions in the environment and provides comprehensive wind direction information support. Wind direction distribution analysis can help the system identify the main wind direction trends in the environment and provide data support for the comprehensive analysis of environmental features. The generation of environmental wind trend features combines data such as wind force intensity, wind force change curve, and wind direction distribution, providing comprehensive environmental wind direction trend information for the system. The analysis of environmental wind trend features helps the system grasp the dynamic change law of environmental wind direction and provides an important basis for the comprehensive evaluation of environmental features.

[0029] Preferably, the specific steps of step S2 are as follows:

[0030] Step S21: Perform global contrast deviation detection on the scene image in front of the user to obtain local contrast difference data;

[0031] Step S22: Perform pixel deviation traversal calculation on the local contrast difference data to obtain the pixel deviation values of the different parts;

[0032] Step S23: Perform dynamic contrast equalization adjustment on the scene image in front of the user based on the pixel deviation values of the different parts to generate a contrast-equalized scene image;

[0033] Step S24: Identify the edge details of the contrast - balanced scene image and extract multiple edge - detail images of the image;

[0034] Step S25: Perform per - image adaptive filtering optimization on the multiple edge - detail images of the image to generate multiple filtered and optimized edge - detail images;

[0035] Step S26: Perform super - resolution fusion reconstruction on the multiple filtered and optimized edge - detail images to construct a super - resolution scene image.

[0036] Through global contrast deviation detection, the system of the present invention can identify local contrast differences in the image, which helps to discover important detail parts in the scene. The acquisition of local contrast difference data can provide an important reference basis for subsequent processing, helping the system better understand the characteristics of the image. Through pixel deviation traversal calculation, the system can quantify the pixel deviation of the different parts in the image, which helps to determine the specific areas in the image that need to be adjusted. The acquisition of the pixel deviation values of the different parts can help the system perform subsequent processing in a targeted manner, improving the efficiency and accuracy of image processing. Contrast dynamic equalization adjustment can achieve targeted image contrast adjustment according to the pixel deviation values of the different parts, improving the visual quality of the image. Generating a contrast - balanced scene image can make the details in the image clearer and more prominent, enhancing the user's perception and recognition ability of the environment. Identifying and extracting the edge details of the image helps the system capture the key features in the scene, improving the recognition rate and realism of the image. The extraction of multiple edge - detail images of the image provides more image parts for subsequent optimization, enhancing the quality and detail performance of the image. Per - image adaptive filtering optimization helps to improve the clarity and quality of the edge details of the image, enhancing the visual effect of the image. The generated multiple filtered and optimized edge - detail images can make the details of the image more prominent and natural, improving the user's perception and experience of the scene. Through super - resolution fusion reconstruction, the system can synthesize the optimized edge - detail images into a high - resolution scene image, providing a clearer visual effect. The constructed super - resolution scene image can improve the clarity and detail display of the image, enabling the user to obtain a more real and vivid scene perception experience.

[0037] Preferably, the specific steps of step S3 are as follows:

[0038] Step S31: Perform object visual recognition on the super - resolution scene image and mark the objects in the scene image;

[0039] Step S32: Perform positioning calculation on the objects in the scene image to obtain object positioning data;

[0040] Step S33: Based on the object positioning data, perform object bounding - box segmentation processing on the super - resolution scene image to obtain multiple scene - object image bounding boxes;

[0041] Step S34: Perform road semantic feature analysis on the super-resolution scene image to extract road semantic features within the scene;

[0042] Step S35: According to the road semantic features within the scene, perform precise correction on the bounding boxes of multiple scene object images to obtain multiple precise scene object bounding boxes.

[0043] Through object visual recognition, the present invention can recognize various objects in the scene, provide key information for subsequent processing, enhance the understanding and presentation of the scene. Marking the objects in the scene image enables users to more intuitively understand the scene content, improve the user's perception and interaction experience of the scene. Object positioning calculation can accurately determine the positions of objects in the scene, provide accurate reference data for subsequent processing, contribute to the further analysis and processing of objects. The obtained object positioning data can help the system better understand the scene structure, improve the accuracy and precision of environmental perception. Object bounding box segmentation processing can clearly identify the bounding boxes of each object, contribute to accurately positioning the object and extracting object features. The obtained multiple scene object image bounding boxes can provide clear boundary information for subsequent object analysis and processing, enhance the interpretability of the scene image. Road semantic feature analysis helps to identify the road structure and features in the scene, improve the understanding and perception of the environment. The extracted road semantic features can provide important clues for subsequent scene understanding and interaction, enhance the environmental perception ability of the intelligent helmet. Precise correction of the bounding box positioning can accurately adjust the positions of object bounding boxes according to road semantic features, improve the accuracy and precision of object positioning. The obtained precise scene object bounding boxes can accurately identify the positions of each object in the scene, provide more refined information for subsequent scene analysis and interaction.

[0044] Preferably, the specific steps of step S4 are as follows:

[0045] Step S41: Perform image frame temporal sequence fitting on the super-resolution scene image to generate a temporal sequence of scene image frames;

[0046] Step S42: Based on the temporal sequence of scene image frames, perform inter-frame bounding box correspondence matching on multiple precise scene object bounding boxes to identify the bounding box matching data between consecutive frames;

[0047] Step S43: Perform image frame optical flow tracking on the bounding box matching data between consecutive frames to obtain the inter-frame optical flow tracking data of the bounding boxes;

[0048] Step S44: Perform optical flow motion change analysis on the inter-frame optical flow tracking data of the bounding boxes to generate the motion change trajectory of the scene objects;

[0049] Step S45: Perform spatio-temporal scene structure evolution analysis on the motion change trajectory of the scene objects to obtain the dynamic scene evolution feature data.

[0050] Through the corresponding matching of bounding boxes, the present invention can track the movement and changes of objects between different frames, provide continuous object position information. Identifying the bounding box matching data between consecutive frames helps to understand the movement trajectory of the object and provides basic data for subsequent analysis and visualization. Optical flow tracking can more precisely describe the movement trajectory of the object between consecutive frames and provide more detailed motion information. The optical flow tracking data of bounding boxes between frames can reveal the changes in the speed and direction of the object and provide more details for the understanding and visualization of dynamic scenes. The analysis of optical flow motion changes can help capture the motion state and change trend of the object, form the object motion trajectory. The generated motion change trajectory of scene objects can be used to analyze the dynamic behavior and evolution process of the object and provide a deeper understanding of the spatio-temporal characteristics of the scene. The analysis of spatio-temporal scene structure evolution helps to understand the dynamic evolution process of objects in the scene, reveals the spatio-temporal relationship and structural changes of the scene. The obtained dynamic scene evolution feature data can provide rich dynamic effects for 3D visualization, enabling users to more deeply perceive the changes and development of the scene.

[0051] Preferably, the specific steps of step S45 are as follows:

[0052] Extract the upper and lower frame images based on the sequential scene image frame sequence;

[0053] Extract the timestamps of the upper and lower frame images and calculate the time interval between the upper and lower frame images;

[0054] Perform object displacement analysis within the time period on the scene object motion change trajectory according to the time interval between the upper and lower frame images, and generate object displacement data within the time interval;

[0055] Traverse all sequential scene image frame sequences and generate object displacement data within all time intervals;

[0056] Perform time-axis sampling on the object displacement data within all time intervals to generate object displacement trajectories at different times;

[0057] Perform scene structure change analysis within the time interval on the object displacement trajectories at different times to obtain scene structure change data for multiple time periods;

[0058] Calculate the optical flow change speed of the scene object motion change trajectory to generate the change speed of the scene object;

[0059] Perform dynamic scene structure evolution on the scene structure change data for multiple time periods based on the change speed of the scene object to obtain dynamic scene evolution feature data.

[0060] By extracting the timestamps of the upper and lower frame images, the present invention can calculate the time interval of each frame image, providing a time reference for subsequent motion analysis. Based on the time interval, the displacement of scene objects is analyzed, and the motion trajectory and displacement data of objects within different time periods can be obtained. By traversing all sequential image frames, complete object motion trajectory data can be generated, laying a foundation for dynamic scene modeling. By analyzing the object displacement trajectories in different time periods, structural changes in the scene can be discovered, such as the appearance of new objects, the disappearance of original objects, or changes in their positions. These scene structure change data can be used to describe the evolution process of the dynamic environment, enhancing the perception ability of the 3D visualization system to environmental changes. Through the calculation of the optical flow change speed, real-time motion speed information of scene objects can be obtained. These speed data can be correlated with the scene structure changes for a more refined description of the evolution law of the dynamic environment. By integrating information such as object displacement trajectories, scene structure changes, and object motion speeds, evolution feature data of the dynamic scene can be constructed. These data can be used to drive the 3D visualization system to generate a more realistic and dynamic environmental model, enhancing the user's immersion and interactivity.

[0061] Preferably, the specific steps of step S5 are as follows:

[0062] Step S51: Conduct a three-dimensional spatial structure analysis on the super-resolution scene image to generate three-dimensional spatial scene structure data;

[0063] Step S52: Perform three-dimensional point cloud modeling on the three-dimensional spatial scene structure data to construct a three-dimensional spatial scene model;

[0064] Step S53: Based on the dynamic scene evolution feature data, conduct in-scene dynamic structure change evolution on the three-dimensional spatial scene structure data to generate scene dynamic structure evolution data;

[0065] Step S54: Use the scene dynamic structure evolution data to conduct dynamic structure evolution simulation on the three-dimensional spatial scene model to generate a dynamic three-dimensional scene model;

[0066] Step S55: Render the environmental attributes of the dynamic three-dimensional scene model with the real-time scene attribute state diagram to generate a dynamically attribute-rendered scene model.

[0067] Through three-dimensional space structure analysis, the present invention can more accurately understand the position, size, and spatial relationship of objects in a scene, providing basic data for subsequent modeling. The generated three-dimensional space scene structure data helps to construct a more three-dimensional and depth-sense scene model, enhancing the three-dimensional visualization effect. Through three-dimensional point cloud modeling, the objects and structures in the scene can be presented in the form of point clouds, achieving more refined scene modeling. The constructed three-dimensional space scene model can provide a basis for subsequent dynamic structure evolution, adding more elements to the interactivity and realism of the scene. The dynamic structure change and evolution within the scene can simulate the dynamic change process of objects in the scene, making the scene more vivid and realistic. The generated scene dynamic structure evolution data provides a basis for subsequent dynamic simulation, increasing the spatio-temporal characteristics and interactivity of the scene. The dynamic structure evolution simulation can make the three-dimensional scene model present the movement and structure change of objects, increasing the dynamicity and visual attractiveness of the scene. The generated dynamic three-dimensional scene model can provide a more vivid scene experience, enabling users to better perceive the changes and development of the environment. Environment attribute rendering can add realistic lighting effects and environmental features to the dynamic three-dimensional scene model according to the real-time scene attribute status map. The generated dynamic attribute rendering scene model can enhance the realism and visual effect of the scene, improving the user's immersion and experience quality.

[0068] Preferably, the specific steps of step S6 are as follows:

[0069] Step S61: Perform dynamic movement detail distortion detection on the dynamic attribute rendering scene model to identify the dynamic detail distortion data of object movement;

[0070] Step S62: Locate the distorted detail parts based on the dynamic detail distortion data of object movement and mark the distorted detail parts of the model;

[0071] Step S63: Perform dynamic distortion compensation on the distorted detail parts of the model to generate a distortion compensation rendering model;

[0072] Step S64: Perform three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model;

[0073] Step S65: Visually present the scene rendering visualization model to the smart helmet.

[0074] Through dynamic detail distortion detection, the present invention can timely detect possible detail distortions in the model during the movement of objects, improve the accuracy of scene rendering. Identifying dynamic detail distortion data helps optimize the rendering process, ensuring that the scene is presented more realistically and smoothly. The positioning of distorted detail parts can accurately locate the specific parts where distortion occurs in the model, providing a basis for subsequent distortion compensation. Marking the distorted detail parts of the model helps to repair the distortion targeted, improving the quality and realism of scene rendering. Dynamic distortion compensation can specifically repair the detail distortions in the model, enhancing the detail performance and authenticity of scene rendering. The generated distortion compensation rendering model can eliminate the possible distortion phenomena during the movement of objects, improving the rendering effect. Through three-dimensional scene visualization processing, the model after distortion compensation can be optimized to enhance the visual effect and realism of the scene. The constructed scene rendering visualization model can better display the dynamic movement and detail performance of objects in the environment, enhancing the user's visual experience. Presenting the optimized scene rendering visualization model to the smart helmet allows users to more intuitively perceive the details and dynamic changes of the environment. The three-dimensional visualization of the environmental perception data based on the smart helmet can provide an immersive scene experience, enhancing the user's sense of interaction and participation.

[0075] In this specification, a system for processing environmental perception data based on a smart helmet is provided, which is used to execute the method for processing environmental perception data based on a smart helmet as described above, including:

[0076] A scene attribute module, which acquires the scene image in front of the user and multi-source real-time environmental perception data; analyzes the scene attribute status of the multi-source real-time environmental perception data, and constructs a real-time scene attribute status graph;

[0077] An image reconstruction module, which performs contrast dynamic equalization adjustment on the scene image in front of the user, and performs super-resolution fusion reconstruction to construct a super-resolution scene image;

[0078] A bounding box segmentation module, which performs object visual recognition on the super-resolution scene image, and performs object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0079] A scene structure evolution module, which analyzes the optical flow motion changes of multiple accurate scene object bounding boxes, and performs spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0080] A dynamic attribute rendering module, which performs dynamic structure evolution simulation on the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute status graph, and then performs environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0081] The scene visualization module performs dynamic distortion compensation on the dynamically rendered scene model with dynamic attributes to generate a distortion-compensated rendered model, and performs three-dimensional scene visualization processing on the distortion-compensated rendered model to construct a scene rendering visualization model.

[0082] The present invention helps to understand the surrounding environment in real time, improve the user's perception ability and safety by obtaining the scene image in front of the user and multi-source environment real-time perception data. Constructing a real-time scene attribute status map through state analysis of multi-source environment real-time perception data provides basic data for subsequent processing, enhances the intelligence and practicability of the system. Adjusting the contrast dynamic balance of the scene image in front of the user and performing super-resolution fusion reconstruction can improve the image quality, enhance the display of scene details and recognition accuracy. Constructing a super-resolution scene image helps to provide a clearer and more realistic image, providing better input for subsequent object recognition and scene structure analysis. Performing object visual recognition and bounding box segmentation processing on the super-resolution scene image helps to accurately identify the objects in the scene, providing accurate object position information for subsequent scene structure analysis. Obtaining multiple precise scene object bounding boxes can help the system understand the scene more finely and improve the accuracy of recognition and analysis. Through optical flow motion change analysis and spatio-temporal scene structure evolution analysis, the system can capture the dynamic changes of objects in the scene, helping to monitor the scene state and behavior in real time. Obtaining dynamic scene evolution feature data can provide a basis for subsequent dynamic attribute rendering, making the scene presentation more vivid and realistic. Based on the dynamic scene evolution feature data and the real-time scene attribute status map, performing dynamic structure evolution simulation on the super-resolution scene image helps to simulate the dynamic changes of objects in the scene, enhancing the realism and dynamics of the scene. Generating a dynamically rendered scene model with dynamic attributes can provide a more vivid and interactive scene presentation, enhancing the user experience and immersion. Performing dynamic distortion compensation and three-dimensional scene visualization processing on the dynamically rendered scene model with dynamic attributes can improve the quality and realism of scene rendering, making the scene more vivid and attractive. Constructing a scene rendering visualization model helps the user better understand and perceive the environment, providing a more intuitive and vivid scene display. Brief Description of the Drawings

[0083] Figure 1 It is a schematic flow chart of the steps of a method for processing environmental perception data based on an intelligent helmet according to the present invention;

[0084] Figure 2 It is a schematic detailed implementation step flow chart of step S1;

[0085] Figure 3 It is a schematic detailed implementation step flow chart of step S2;

[0086] Figure 4 It is a schematic detailed implementation step flow chart of step S3. Detailed implementation manners

[0087] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0088] The embodiments of the present application provide a method and a system for processing environmental perception data based on an intelligent helmet. The execution subjects of the method and system for processing environmental perception data based on the intelligent helmet include, but are not limited to, mechanical equipment, data processing platforms, cloud server nodes, network uploading devices, etc. that carry this system, which can be regarded as general computing nodes of the present application. The data processing platform includes, but is not limited to, at least one of an audio and image management system, an information management system, and a cloud data management system.

[0089] Please refer to Figures 1 to 4 , the present invention provides a method for processing environmental perception data based on an intelligent helmet. A high-definition camera and multi-source sensors are embedded inside the intelligent helmet. The method for processing environmental perception data based on the intelligent helmet includes the following steps:

[0090] Step S1: Obtain the scene image in front of the user and multi-source real-time environmental perception data; analyze the scene attribute status of the multi-source real-time environmental perception data, and construct a real-time scene attribute status graph;

[0091] Step S2: Perform contrast dynamic equalization adjustment on the scene image in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image;

[0092] Step S3: Perform object visual recognition on the super-resolution scene image, and perform object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0093] Step S4: Perform optical flow motion change analysis on multiple accurate scene object bounding boxes, and perform spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0094] Step S5: Perform dynamic structure evolution simulation on the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute status graph, and then perform environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0095] Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensation rendering model; perform three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model.

[0096] Through the construction of a real-time scene attribute status map, the present invention can provide the helmet system with an accurate understanding of the status and characteristics of the user's surrounding environment, providing basic data for subsequent processing. Scene attribute status analysis helps the system identify and understand the attributes of the environment, providing key information for the next processing steps. The construction of a super-resolution scene image improves the clarity and details of the image, enhancing the user's perception and understanding of the environment. Contrast dynamic equalization adjustment helps enhance the visual effect of the image, making details clearer and improving the user experience. The acquisition of accurate scene object bounding boxes helps the system accurately identify and locate objects in the environment, providing important information for subsequent scene analysis and processing. Object visual recognition and segmentation processing improve the system's understanding of objects in the environment, providing a basis for subsequent analysis and simulation. The acquisition of dynamic scene evolution feature data helps the system capture the movement and spatio-temporal changes of objects in the environment, providing richer scene information. Spatio-temporal scene structure evolution analysis helps the system understand the dynamic changes of the environment, providing important data support for subsequent simulation and visualization. Dynamic structure evolution simulation and environment attribute rendering help the system simulate and display the dynamic changes of the environment, providing a vivid scene perception experience. The generated dynamic attribute rendering scene model can present a more real and vivid environment scene for the user, enhancing the user's virtual reality experience. Dynamic distortion compensation and three-dimensional scene visualization processing help improve the realism and fidelity of scene rendering, enhancing the user's perception and experience of the environment. The scene rendering visualization model can present a more realistic environment scene for the user, providing an immersive virtual reality experience, enhancing user interaction and perception effects, helping the user label other objects in the scene, improving the visualization degree of the surrounding scene of the user, and greatly improving the driving safety of the user.

[0097] In an embodiment of the present invention, referring to Figure 1 , it is a schematic flowchart of the steps of a method for processing environmental perception data based on an intelligent helmet according to the present invention. In this example, the steps of the method include:

[0098] Step S1: Obtain the scene image in front of the user and multi-source real-time environmental perception data; perform scene attribute status analysis on the multi-source real-time environmental perception data to construct a real-time scene attribute status map;

[0099] In this embodiment, based on the high-definition camera inside the intelligent helmet to capture the scene image in front of the user, a mobile device (such as a smartphone or smart glasses) or a fixed camera can be selected. Using a camera interface (such as OpenCV or GStreamer) to capture the scene image in real time, setting an appropriate frame rate (such as 30fps) to ensure the continuity of the image, converting the captured image data into a format suitable for subsequent processing (such as RGB or grayscale image), and storing it in the memory for subsequent use. Integrate multiple sensors (such as temperature sensors, humidity sensors, lidar (LiDAR), IMU (inertial measurement unit), etc.) to obtain multi-dimensional environmental data, use a suitable interface (such as I2C, UART, CAN, etc.) to read data from the sensors, ensure that the sampling frequency of the data is synchronized with the image capture, select a suitable data fusion algorithm (such as Kalman filter, particle filter or multi-sensor fusion algorithm) to fuse the data from different sensors, extract more accurate environmental state information, conduct real-time evaluation according to the extracted attribute status, generate state indicators of the scene (such as safety, comfort), design the structure of the state diagram, including the visualization methods of each attribute (such as heat map, bar chart, icon, etc.), use a visualization library (such as Matplotlib, D3.js or OpenGL) to display the extracted attribute status in a graphical manner, ensure that the graph is updated synchronously with the real-time data, set a real-time update mechanism, so that the scene attribute status diagram can reflect the latest status immediately when the data changes, and ensure that the user obtains the latest information.

[0100] Step S2: Perform contrast dynamic equalization adjustment on the scene image in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image;

[0101] In this embodiment, the global contrast is calculated using the histogram equalization method (such as cv2.equalizeHist in OpenCV) to obtain the brightness distribution of the image. Local contrast calculation (such as local histogram equalization or CLAHE) is used to perform equalization processing on local regions of the image. This step can enhance regions with rich details in the image. The required contrast gain coefficient is calculated, usually adjusted using a non-linear function (such as Gamma correction) based on the comparison of global and local contrasts. The calculated gain coefficient is applied to adjust each pixel of the original image to generate a contrast-equalized image. A suitable super-resolution reconstruction algorithm is selected. Commonly used ones are SRGAN (Super-Resolution Generative Adversarial Network) or EDSR (Enhanced Deep Residual Networks) based on convolutional neural networks (CNNs). The contrast-equalized image is preprocessed as necessary, such as resizing and normalization, to meet the input requirements of the super-resolution model. The preprocessed image is input into the super-resolution model to perform the image reconstruction process. The model will generate a high-resolution image, restoring details and textures. If there are multi-source images (such as images from different angles or different time points), image fusion techniques (such as weighted averaging or image stitching) are used to combine multiple super-resolution images into a unified high-resolution scene image. The generated super-resolution scene image is saved as a new image file (such as PNG or JPEG) and prepared for subsequent processing.

[0102] Step S3: Perform object visual recognition on the super-resolution scene image and perform object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0103] In this embodiment, a pre-trained object recognition model is loaded using a deep learning framework (such as TensorFlow or PyTorch), ensuring that the model has been trained on a suitable dataset (such as COCO or Pascal VOC). The preprocessed super-resolution image is input into the object recognition model to obtain the output of the model, which usually includes: the category of the object recognized in the image, the confidence level indicating the model's confidence in each recognition result, the bounding box position of each object, usually represented by the upper-left and lower-right coordinates (x_min, y_min, x_max, y_max). The bounding box coordinates and the corresponding category information are extracted from the output of the object recognition model. A threshold is set according to the confidence score to filter out the bounding boxes with a confidence level higher than this threshold to reduce false detections. For example, the threshold is set to 0.5, and only the recognition results with a confidence level higher than 50% are retained. To process overlapping bounding boxes, the non-maximum suppression algorithm is applied to retain the bounding box with the highest confidence level and remove other boxes with a high overlap degree. This process can be achieved by calculating the IoU (Intersection over Union).

[0104] Step S4: Analyze the optical flow motion changes of multiple precise scene object bounding boxes and conduct a spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0105] In this embodiment, a suitable optical flow calculation algorithm is selected, such as the Lucas-Kanade method or the Horn-Schunck method, to calculate the optical flow field in the image sequence, obtaining a continuous sequence of scene images, ensuring that there is a time interval between the images (for example, between adjacent frames). The continuous image frames are processed to calculate the optical flow vector of each pixel, representing its motion over time. The optical flow characteristics within each bounding box are recorded, including speed, direction, and change amplitude. The optical flow information within each bounding box is analyzed to extract the motion change characteristics of the object, such as acceleration, impact force, etc. The optical flow characteristics of the object in different time frames are compared to identify the dynamic change patterns and record the trend of the motion. The motion change characteristics obtained from the optical flow analysis are integrated with the time information of the scene to establish a spatio-temporal data model. A suitable spatio-temporal evolution model is selected, such as a state space model or a dynamic Bayesian network, to describe the change of the scene. The dynamic scene evolution feature data, including the relative position, speed change, and interaction of the objects, are extracted from the trained spatio-temporal evolution model. The extracted dynamic scene evolution feature data are organized into structured data for subsequent analysis and application.

[0106] Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute status map, conduct a dynamic structure evolution simulation on the super-resolution scene image, and then perform environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0107] In this embodiment, a real-time scene attribute state diagram is imported to ensure that the environmental attributes in the diagram can be used for rendering. The initial state of the object is set in the simulation environment. According to the dynamic evolution feature data, the position, speed, and state of the object are updated frame by frame to ensure the coherence and authenticity of the movement. The interactions and collisions between objects are processed to ensure that the objects can move naturally in the dynamic environment. According to the real-time scene attribute state diagram, the rendering parameters are configured to ensure that the real-time scene attribute state diagram can be updated synchronously with the rendering process. For example, when the environmental temperature changes, the lighting and colors in the scene are adjusted accordingly. GPU acceleration technology is used for real-time rendering to ensure that the dynamic attribute rendering scene model can be presented smoothly. Shadows, reflections, and other visual effects are processed to enhance the realism of the scene. The rendered dynamic attribute rendering scene model is saved as a high-quality image or video file (such as PNG, JPEG, or MP4) for easy display and analysis. The user interaction function is tested in the generated scene model to ensure that the user can navigate and interact in the dynamic scene, and the reliability and accuracy of the model are verified.

[0108] Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensation rendering model; perform three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model.

[0109] In this embodiment, the dynamic rendering model is compared with the high-definition reference image to analyze whether the object contours in the model are clearly defined. The semantic segmentation technology is applied to extract the dynamic object regions from the image. For each extracted dynamic object region, an image super-resolution method based on a deep convolutional neural network is used for reconstruction to redefine the object contour details. The high-definition object reconstruction result after super-resolution is fused and replaced with the corresponding region of the original three-dimensional dynamic rendering model to achieve the compensation of edge details. For regions with continuous subtle change characteristics (such as faces, fingers, etc.), multi-time frame data is collected for modeling, and the continuous change details of these regions over time are reconstructed using a temporal model. The 3D model with increased dynamic details is imported into a virtual scene rendering engine (such as Unreal Engine) for realistic rendering. The augmented reality technology is used to present the rendered result scene, and the virtual scene is synchronously rendered into the real world using a head-mounted device to form a three-dimensional mixed reality effect. New data is continuously collected to continuously optimize the model and improve the accuracy of detail reconstruction and dynamic restoration.

[0110] In this embodiment, refer to Figure 2 which is the schematic diagram of the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0111] Step S11: Obtain the user's front scene image based on a high-definition camera;

[0112] Step S12: Continuously obtain multi-source real-time environmental perception data by using multi-source sensors. The multi-source real-time environmental perception data includes environmental wind attribute parameters and environmental temperature attribute parameters;

[0113] Step S13: Conduct an analysis of the environmental wind trend characteristics for the environmental wind attribute parameters to generate environmental wind trend characteristics;

[0114] Step S14: Conduct a calculation of the temperature interval change for the environmental temperature attribute parameters to generate a temperature interval change rate;

[0115] Step S15: Conduct an analysis of the scenario attribute status for the environmental wind trend characteristics and the temperature interval change rate, and construct a real-time scenario attribute status map.

[0116] In this embodiment, based on the high-definition camera on the intelligent head, the frame rate is set to 30 frames per second, and continuous scene images are continuously obtained. The acquired image data is stored in the local storage in JPEG format, and the files are named using timestamps. Combine the temperature sensor and the anemometer to ensure that they can transmit data in real time. Set the sensor to collect data once per second, record the environmental wind speed, wind direction, and air temperature, and store the collected wind attribute parameters and temperature parameters in CSV format in real time. Clean the wind attribute data to remove outliers exceeding the threshold (such as wind speed greater than 50 m / s). Use the moving average method to calculate the average values of the wind speed and wind direction in the past 10 minutes to generate wind speed trend characteristics (such as rising, falling, stable), and store them in the database in a structured form. Organize the temperature data into a time series format to ensure the continuity of the timestamps of the data. Calculate the temperature difference between each pair of adjacent time points to generate a temperature interval change rate (for example, the change per minute). Integrate the environmental wind trend characteristics and the temperature change rate into a unified dataset, use the decision tree algorithm to analyze the comprehensive data, judge the current environmental state (such as "strong wind, warm"), generate a real-time scenario attribute status map according to the analysis results, and use a chart library (such as Matplotlib) for visualization and export it in PNG format.

[0117] In this embodiment, the specific steps of Step S12 are as follows:

[0118] Conduct an average calculation of the wind force intensity for the environmental wind attribute parameters to generate an average environmental wind intensity;

[0119] Conduct a multi-time window partitioning process on the multi-source real-time environmental perception data to obtain multiple time period windows;

[0120] Based on the multiple time period windows, conduct a calculation of the wind force extreme values for the environmental wind attribute parameters, and extract the wind force extreme values for each time period;

[0121] Conduct an identification of the change amplitude of the wind force extreme values for each time period of the wind force extreme values to generate a multi-period extreme value change amplitude;

[0122] Perform time series fitting on the amplitude changes of extreme values in multiple time periods to construct a curve for the variation of wind extreme values;

[0123] Conduct an analysis of the wind direction distribution of environmental wind attribute parameters to generate environmental wind direction distribution data;

[0124] Conduct an analysis of the trend characteristics of environmental wind on the average value of environmental wind intensity, the curve of wind extreme value changes, and the environmental wind direction distribution data to generate environmental wind trend characteristics.

[0125] In this embodiment, real-time environmental wind speed data is collected through multiple environmental perception devices (such as wind speed sensors). The arithmetic mean of the multiple collected wind speed data points is calculated to obtain the average value of environmental wind intensity. This average wind speed can reflect the overall wind force intensity of the current environment. Determine the time window length (such as every hour, every minute), and set the start and end times of the time window. According to the set time window, the environmental wind attribute parameter data is segmented into multiple time period windows. Select the maximum and minimum value calculation methods to ensure that the wind extreme values of each time period can be obtained. Analyze the data of each time period window, extract the maximum and minimum wind speeds within this time period, calculate the change amplitude between the wind extreme values of adjacent time periods, generate the extreme value change data of multiple time periods, perform time series fitting on these extreme value change data, construct a curve for the change of wind extreme values. This kind of analysis can discover the change trend and fluctuation characteristics of environmental wind force. Select a suitable fitting model (such as linear regression, spline interpolation) to fit the amplitude changes of extreme values in multiple time periods, use the fitting algorithm to analyze the change amplitude, generate a curve for the change of wind extreme values, visualize the fitting result, and save it as a graphic file (such as PNG format). Conduct a correlation analysis on the aforementioned obtained average value of environmental wind intensity, the curve of wind extreme value changes, and the wind direction distribution data. Based on these data, summarize the overall trend characteristics of environmental wind force, such as the change of wind force strength, the change of wind direction, etc. The result of this comprehensive analysis can more comprehensively describe the wind force condition of the current environment.

[0126] In this embodiment, refer to Figure 3 , which is a schematic diagram of the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0127] Step S21: Perform global contrast deviation detection on the image of the scene in front of the user to obtain local contrast difference data;

[0128] Step S22: Perform pixel deviation traversal calculation on the local contrast difference data to obtain the pixel deviation values of the different parts;

[0129] Step S23: Based on the pixel deviation values of the different parts, perform dynamic contrast equalization adjustment on the image of the scene in front of the user to generate a scene image with equalized contrast;

[0130] Step S24: Identify the edge details of the contrast - equalized scene image and extract multiple edge - detail images of the image;

[0131] Step S25: Perform per - image adaptive filtering optimization on the multiple edge - detail images of the image to generate multiple filtered and optimized edge - detail images;

[0132] Step S26: Perform super - resolution fusion reconstruction on the multiple filtered and optimized edge - detail images to construct a super - resolution scene image.

[0133] In this embodiment, read the scene image in front of the user, ensure that the image format is suitable for processing (such as RGB), calculate the global contrast of the image, usually using the histogram equalization method to obtain the global brightness distribution, calculate the contrast of each local window (such as 3x3 or 5x5), compare it with the global contrast to generate local contrast difference data, identify the regions with significant contrast differences according to the local contrast difference data, traverse the pixels in the identified difference regions, calculate the deviation value of the brightness of each pixel from the average brightness of the region, record the deviation value of each pixel to generate deviation value data of the pixels in the difference part, adjust the contrast of different regions in the image according to the deviation value data of the pixels in the difference part, so that the regions with lower contrast are enhanced to generate a contrast - equalized scene image, use an edge - detection algorithm (such as Canny edge detection, Sobel operator, etc.) to process the contrast - equalized scene image, identify and extract the edge details in the image, save the detected edge details as multiple images for subsequent processing, select a suitable adaptive filtering algorithm (such as adaptive median filtering, bilateral filtering, etc.) to reduce noise and retain edge details, apply adaptive filtering to each edge - detail image to optimize the edge performance of each image, generate multiple filtered and optimized edge - detail images, select a suitable super - resolution reconstruction algorithm (such as SRGAN based on deep learning or traditional interpolation methods), input the multiple filtered and optimized edge - detail images into the super - resolution reconstruction algorithm to generate a high - resolution scene image, and save the constructed super - resolution scene image as a high - quality image file (such as TIFF or PNG).

[0134] In this embodiment, refer to Figure 4 , which is a schematic diagram of the detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0135] Step S31: Perform object visual recognition on the super - resolution scene image and mark the objects in the scene image;

[0136] Step S32: Perform positioning calculation on the objects in the scene image to obtain object positioning data;

[0137] Step S33: Perform object bounding box segmentation processing on the super-resolution scene image based on the object positioning data to obtain multiple scene object image bounding boxes;

[0138] Step S34: Analyze the road semantic features of the super-resolution scene image and extract the road semantic features within the scene;

[0139] Step S35: Perform precise correction of the border positioning for multiple scene object image bounding boxes according to the road semantic features within the scene to obtain multiple precise scene object bounding boxes.

[0140] In this embodiment, use object detection algorithms of deep learning (such as YOLO, Faster R-CNN, etc.) to identify and analyze the super-resolution scene image. The algorithm can detect various objects in the image and mark their position coordinates in the image. According to the object positioning data obtained in the previous step, draw the bounding boxes of each object on the super-resolution scene image. Using image segmentation technology, the boundary region of each object can be accurately extracted to form independent object images. Extract the bounding box information of each object from the object positioning data. According to the bounding box coordinates, crop multiple scene object images from the super-resolution scene image to generate independent object images. Save each cropped object image as a separate file, and the naming method includes the object category and position coordinates. Select a suitable semantic segmentation model (such as DeepLab, SegNet) to perform road semantic analysis on the super-resolution scene image. Run the semantic segmentation model to extract the road region and its features (such as road type, width) in the image. Store the extracted road semantic features in a structured form to ensure that the detailed information of the road features is recorded. Match the extracted road semantic features with the object bounding boxes to analyze the relative position relationship between the object and the road. Use a precise positioning correction algorithm (such as the least squares method) to adjust the bounding box of each object to ensure its consistency with the road features.

[0141] In this embodiment, step S4 includes the following steps:

[0142] Step S41: Perform image frame time series fitting on the super-resolution scene image to generate a time series scene image frame sequence;

[0143] Step S42: Perform inter-frame bounding box correspondence matching on multiple precise scene object bounding boxes based on the time series scene image frame sequence to identify the bounding box matching data between consecutive frames;

[0144] Step S43: Perform image frame optical flow tracking on the bounding box matching data between consecutive frames to obtain the inter-frame optical flow tracking data of the bounding box;

[0145] Step S44: Perform optical flow motion change analysis on the inter-frame optical flow tracking data of the bounding box to generate the motion change trajectory of the scene object;

[0146] Step S45: Perform spatio-temporal scene structure evolution analysis on the motion change trajectories of scene objects to obtain dynamic scene evolution feature data.

[0147] In this embodiment, continuous frames are extracted from the super-resolution scene image to ensure consistent frame rate. A suitable temporal fitting method (such as linear interpolation or polynomial fitting) is selected to process the image frames, generating a sequence of temporal scene image frames, ensuring smooth transitions between each frame, and saving them in video or image sequence format. Multiple precise scene object bounding box data are extracted from each frame image. A matching algorithm (such as the Hungarian algorithm or IoU-based matching) is used to perform corresponding matching of the bounding boxes between consecutive frames, identifying and recording the bounding box matching data between consecutive frames, including object IDs and corresponding frame information. A suitable optical flow algorithm (such as Lucas-Kanade or Farneback) is selected to calculate the optical flow within the bounding boxes. For each matched bounding box, its optical flow in consecutive frames is calculated to obtain the inter-frame optical flow tracking data of the bounding boxes. Combining the aforementioned bounding box matching and optical flow tracking data, the motion change trajectories of each object in the scene can be reconstructed. By analyzing these trajectory data, features such as the motion speed, acceleration, and motion direction of the objects can be extracted. The motion change trajectories of the objects are correlated with the overall spatio-temporal structure changes of the scene. By comparing the scene changes in different time periods, the evolution law of the dynamic scene is discovered, and the dynamic evolution feature data of the scene objects are extracted, including motion patterns and change trends.

[0148] In this embodiment, the specific steps of step S45 are as follows:

[0149] Extract the upper and lower frame images based on the sequence of temporal scene image frames;

[0150] Extract the timestamps of the upper and lower frame images and calculate the time interval between the upper and lower frame images;

[0151] Perform object displacement analysis within the time period of the motion change trajectory of the scene object according to the time interval between the upper and lower frame images to generate object displacement data within the time interval;

[0152] Traverse all sequences of temporal scene image frames to generate object displacement data within all time intervals;

[0153] Perform time-axis sampling on the object displacement data within all time intervals to generate object displacement trajectories at different times;

[0154] Perform scene structure change analysis within the time interval on the object displacement trajectories at different times to obtain scene structure change data for multiple time periods;

[0155] Calculate the optical flow change speed of the motion change trajectory of the scene object to generate the change speed of the scene object;

[0156] Perform dynamic scene structure evolution on the scene structure change data for multiple time periods based on the change speed of scene objects, so as to obtain dynamic scene evolution feature data.

[0157] In this embodiment, from the sequence of time-series scene image frames, extract two adjacent upper and lower frames of images. By analyzing the timestamps of these two frames of images, calculate the time interval between them. Utilize the previously obtained object motion change trajectory data to analyze the displacement of each object within the time interval between the upper and lower frames of images. Traverse the entire time-series image sequence to generate the object displacement data for all time intervals. Perform time-axis sampling on the previously obtained object displacement data for all time intervals. In this way, the displacement trajectories of each object at different time points can be extracted, analyze the changes in the object displacement trajectories within different time periods, and deduce the dynamic changes in the overall structure of the scene. For example, if the positions of objects change significantly within a certain time period, it can be inferred that the scene structure has changed significantly within that time period. This analysis result can reflect the structural evolution characteristics of the scene in the time dimension. Utilize the previously obtained optical flow change data to calculate the motion change speed of scene objects, and associate this change speed information with the previously analyzed scene structure change data, thereby deducing the evolution characteristics of the entire dynamic scene. This spatio-temporal correlation analysis can more comprehensively depict the interaction process between object motion and environmental changes in the scene.

[0158] In this embodiment, the specific steps of step S5 are as follows:

[0159] Step S51: Perform three-dimensional spatial structure analysis on the super-resolution scene image to generate three-dimensional spatial scene structure data;

[0160] Step S52: Perform three-dimensional point cloud modeling on the three-dimensional spatial scene structure data to construct a three-dimensional spatial scene model;

[0161] Step S53: Perform in-scene dynamic structure change evolution on the three-dimensional spatial scene structure data based on the dynamic scene evolution feature data, so as to generate scene dynamic structure evolution data;

[0162] Step S54: Use the scene dynamic structure evolution data to perform dynamic structure evolution simulation on the three-dimensional spatial scene model to generate a dynamic three-dimensional scene model;

[0163] Step S55: Render the environmental attributes of the dynamic three-dimensional scene model with the real-time scene attribute state diagram to generate a dynamic attribute rendering scene model.

[0164] In this embodiment, feature points and key regions are extracted from the super-resolution scene image. Computer vision libraries (such as OpenCV) are used for corner detection (such as using Harris corner detection or Shi-Tomasi algorithm). The SIFT (Scale-Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF) algorithm is used to describe and match the feature points to ensure that the feature points under different perspectives can be correctly corresponding. Stereo vision technology is used. Through binocular or multi-view geometry calculation, the triangulation method is applied to generate 3D point cloud data. Coordinate transformation is performed through the camera internal and external parameters to obtain the 3D spatial scene structure data. The RANSAC (Random Sample Consensus algorithm) is used to model the 3D point cloud, extract geometric shapes such as planes and spheres, enhance the quality of the point cloud. The voxel grid filter is used to reduce the density of the point cloud, remove noise, and retain important features. The processed 3D point cloud data is stored in the PLY or OBJ format, which supports rich geometric information and color information, facilitating subsequent 3D modeling and visualization. The previously generated dynamic scene evolution feature data (such as object motion trajectories and change trends) is input into the analysis model. Physical-based simulations (such as particle systems) or machine learning models (such as LSTM or RNN) are used to analyze the 3D spatial scene structure data, predict how objects change over time, generate dynamic structure evolution data, and record the state changes of objects at different time points, including information such as position, shape, and size. A 3D development environment such as Unity or Unreal Engine is configured, and the constructed 3D spatial model is imported. According to the dynamic structure evolution data, a physical engine (such as NVIDIA PhysX) is used for dynamic simulation to simulate the motion and interaction effects of objects in the scene. GPU acceleration technology is used to update the state of objects in the scene in real time, generating a dynamic 3D scene model to ensure that the changes in the scene can be reflected in real time. The real-time scene attribute status map (such as temperature, humidity, light, etc.) is imported into the rendering engine, and material properties (such as reflection, refraction), lighting models (such as Phong or Blinn-Phong), and shadow effects (such as shadow mapping) are configured to ensure a realistic rendering effect. Ray tracing technology or rasterization technology is used for final rendering to generate a dynamic attribute rendering scene model, which is saved as a high-quality image (PNG, JPEG) or video format (MP4).

[0165] In this embodiment, the specific steps of step S6 are as follows:

[0166] Step S61: Perform dynamic movement detail distortion detection on the dynamic attribute rendering scene model to identify the dynamic detail distortion data of object movement;

[0167] Step S62: Locate the distorted detail parts based on the dynamic detail distortion data of the object movement, and mark the distorted detail parts of the model;

[0168] Step S63: Perform dynamic distortion compensation on the distorted detail parts of the model to generate a distortion compensation rendering model;

[0169] Step S64: Perform three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model;

[0170] Step S65: Visually present the scene of the scene rendering visualization model to the smart helmet.

[0171] In this embodiment, the optical flow method is used to analyze the dynamic property rendering scene model to detect the distortion phenomenon that occurs when the object is moving. Specifically, the Lucas-Kanade optical flow algorithm can be used to calculate the motion between adjacent frames, define the distortion metric standard (such as the mean square error MSE or the structural similarity index SSIM), evaluate the difference between the moving details and the expected state, identify the significant distortion regions by comparing the dynamic properties of different frames, record the detected distortion data in a data structure, including the position, type, and degree of the distortion region, extract the spatial coordinates and boundary information of all distortion details, in the three-dimensional scene model, use visualization tools (such as OpenGL or Unity) to mark the distorted detail parts, and different colors or shapes can be used for marking to highlight the distortion regions for convenient subsequent processing. Store the information of the marked distorted detail parts in the database to ensure that these information can be quickly accessed during subsequent compensation processing. Select a suitable distortion compensation algorithm (such as an adaptive filter, a deep learning method, or a physics-based model) to repair the distortion region. Consider using a convolutional neural network (CNN) for image compensation, process each marked distorted detail part to generate a distortion compensation rendering model. This process includes interpolating the missing details or filling them using the surrounding pixel information. Verify the compensation effect by comparing the models before and after compensation to ensure the coherence and consistency of the dynamic details. Use professional three-dimensional visualization tools (such as Blender, Unity, or Unreal Engine) to process the distortion compensation rendering model, import the compensated model into the visualization environment, configure the lighting, materials, and camera perspective of the scene to ensure the realism and aesthetics of the scene presentation. Configure the rendering parameters (such as resolution, anti-aliasing, shadow effects, etc.) to generate a high-quality three-dimensional scene visualization model. Ensure the compatibility of the scene visualization model with the interface of the smart helmet, use the SDK or API for connection (such as the Oculus SDK or the HTC Vive SDK), and transmit the constructed scene visualization model to the smart helmet wirelessly or wiredly to ensure the stability and real-time nature of the data transmission. Perform real-time rendering in the smart helmet to ensure that the user can experience a high-quality three-dimensional scene in a dynamic environment and achieve interactive functions.

[0172] In this embodiment, an environment perception data processing system based on a smart helmet is provided, which is used to execute the environment perception data processing method based on the smart helmet as described above, including:

[0173] A scene attribute module, which acquires the scene image in front of the user and multi-source environmental real-time perception data; performs scene attribute state analysis on the multi-source environmental real-time perception data and constructs a real-time scene attribute state map;

[0174] An image reconstruction module that performs dynamic contrast equalization adjustment on the scene image in front of the user, and performs super-resolution fusion reconstruction to construct a super-resolution scene image;

[0175] A bounding box segmentation module that performs object visual recognition on the super-resolution scene image and performs object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes;

[0176] A scene structure evolution module that performs optical flow motion change analysis on multiple accurate scene object bounding boxes and performs spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data;

[0177] A dynamic attribute rendering module that performs dynamic structure evolution simulation on the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute status map, and then performs environmental attribute rendering to generate a dynamic attribute rendering scene model;

[0178] A scene visualization module that performs dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensation rendering model; performs three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model.

[0179] The present invention helps to understand the surrounding environment in real time, improve the user's perception ability and safety by obtaining the user's front scene image and multi-source environmental real-time perception data. Constructing a real-time scene attribute status map through state analysis of multi-source environmental real-time perception data provides basic data for subsequent processing, enhances the intelligence and practicability of the system. Adjusting the contrast dynamic balance and performing super-resolution fusion reconstruction of the user's front scene image can improve the image quality, enhance the display of scene details and recognition accuracy. Constructing a super-resolution scene image helps to provide a clearer and more realistic image, providing better input for subsequent object recognition and scene structure analysis. Performing object visual recognition and bounding box segmentation processing on the super-resolution scene image helps to accurately identify the objects in the scene, providing accurate object position information for subsequent scene structure analysis. Obtaining multiple precise scene object bounding boxes can help the system understand the scene more precisely, improving the accuracy of recognition and analysis. Through optical flow motion change analysis and spatio-temporal scene structure evolution analysis, the system can capture the dynamic changes of objects in the scene, helping to monitor the scene state and behavior in real time. Obtaining dynamic scene evolution feature data can provide a basis for subsequent dynamic property rendering, making the scene presentation more vivid and realistic. Based on the dynamic scene evolution feature data and the real-time scene attribute status map, performing dynamic structure evolution simulation on the super-resolution scene image helps to simulate the dynamic changes of objects in the scene, enhancing the realism and dynamics of the scene. Generating a dynamic property rendering scene model can provide a more vivid and interactive scene presentation, enhancing the user experience and immersion. Performing dynamic distortion compensation and three-dimensional scene visualization processing on the dynamic property rendering scene model can improve the quality and realism of scene rendering, making the scene more vivid and attractive. Constructing a scene rendering visualization model helps users better understand and perceive the environment, providing a more intuitive and vivid scene display.

[0180] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to cover all changes falling within the meaning and scope of the equivalent elements of the application documents within the present invention.

[0181] As described above, these are only the specific implementation manners of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. An environmental perception data processing method based on an intelligent helmet, characterized in that The intelligent helmet is embedded with a high-definition camera and multi-source sensors, including the following steps: Step S1: Obtain the image of the scene in front of the user and the real-time multi-source environmental perception data; analyze the scene attribute status of the real-time multi-source environmental perception data to construct a real-time scene attribute status map; Step S2: Perform contrast dynamic equalization adjustment on the image of the scene in front of the user, and perform super-resolution fusion reconstruction to construct a super-resolution scene image; Step S3: Perform object visual recognition on the super-resolution scene image, and perform object bounding box segmentation processing to obtain multiple accurate scene object bounding boxes; Step S4: Perform optical flow motion change analysis on multiple accurate scene object bounding boxes, and perform spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data; Step S5: Based on the dynamic scene evolution feature data and the real-time scene attribute status map, perform dynamic structure evolution simulation on the super-resolution scene image, and then perform environmental attribute rendering to generate a dynamic attribute rendering scene model; Step S6: Perform dynamic distortion compensation on the dynamic attribute rendering scene model to generate a distortion compensation rendering model; perform three-dimensional scene visualization processing on the distortion compensation rendering model to construct a scene rendering visualization model; Among them, the specific steps of Step S4 are: Step S41: Perform image frame time series fitting on the super-resolution scene image to generate a time series scene image frame sequence; Step S42: Based on the time series scene image frame sequence, perform inter-frame bounding box correspondence matching on multiple accurate scene object bounding boxes to identify the bounding box matching data between consecutive frames; Step S43: Perform image frame optical flow tracking on the bounding box matching data between consecutive frames to obtain the inter-frame optical flow tracking data of the bounding box; Step S44: Perform optical flow motion change analysis on the inter-frame optical flow tracking data of the bounding box to generate the motion change trajectory of the scene object; Step S45: Perform spatio-temporal scene structure evolution analysis on the motion change trajectory of the scene object to obtain dynamic scene evolution feature data; Among them, the specific steps of Step S5 are: Step S51: Perform three-dimensional space structure analysis on the super-resolution scene image to generate three-dimensional space scene structure data; Step S52: Perform three-dimensional point cloud modeling on the three-dimensional space scene structure data to construct a three-dimensional space scene model; Step S53: Based on the dynamic scene evolution feature data, perform in-scene dynamic structure change evolution on the three-dimensional space scene structure data to generate scene dynamic structure evolution data; Step S54: Use the scene dynamic structure evolution data to perform dynamic structure evolution simulation on the three-dimensional space scene model to generate a dynamic three-dimensional scene model; Step S55: Use the real-time scene attribute status map to perform environmental attribute rendering on the dynamic three-dimensional scene model to generate a dynamic attribute rendering scene model.

2. The environmental perception data processing method based on an intelligent helmet according to claim 1, wherein, The specific steps of Step S1 are: Step S11: Obtain the image of the scene in front of the user based on the high-definition camera; Step S12: Continuously obtain real-time multi-source environmental perception data using multi-source sensors, and the real-time multi-source environmental perception data includes environmental wind attribute parameters and environmental temperature attribute parameters; Step S13: Perform environmental wind trend feature analysis on the environmental wind attribute parameters to generate environmental wind trend features; Step S14: Perform temperature interval change calculation on the environmental temperature attribute parameters to generate a temperature interval change rate; Step S15: Perform scenario attribute status analysis on the environmental wind trend characteristics and the temperature interval change rate, and construct a real-time scenario attribute status map.

3. The environmental perception data processing method based on an intelligent helmet according to claim 2, wherein The specific steps of Step S13 are as follows: Perform average calculation of the wind force intensity on the environmental wind attribute parameters to generate an average environmental wind intensity; Perform multi-time window partitioning processing on the multi-source environmental real-time perception data to obtain multiple time period windows; Based on the multiple time period windows, perform wind force extreme value calculation on the environmental wind attribute parameters, and extract the wind force extreme values for each time period; Identify the amplitude of change of the wind force extreme values for each time period to generate the amplitude of change of multi-period extreme values; Perform time series fitting on the amplitude of change of multi-period extreme values to construct a wind force extreme value change curve; Perform wind direction distribution analysis on the environmental wind attribute parameters to generate environmental wind direction distribution data; Perform environmental wind trend characteristic analysis on the average environmental wind intensity, the wind force extreme value change curve, and the environmental wind direction distribution data to generate environmental wind trend characteristics.

4. The method for processing environmental perception data based on an intelligent helmet according to claim 1, wherein The specific steps of Step S2 are as follows: Step S21: Perform global contrast deviation detection on the user's front scene image to obtain local contrast difference data; Step S22: Perform pixel deviation traversal calculation on the local contrast difference data to obtain the pixel deviation values of the different parts; Step S23: Based on the pixel deviation values of the different parts, perform contrast dynamic equalization adjustment on the user's front scene image to generate a contrast equalized scene image; Step S24: Identify the edge details of the contrast equalized scene image and extract multiple image edge detail images; Step S25: Perform per-image adaptive filtering optimization on the multiple image edge detail images to generate multiple filtered and optimized edge detail images; Step S26: Perform super-resolution fusion reconstruction on the multiple filtered and optimized edge detail images to construct a super-resolution scene image.

5. The method for processing environmental perception data based on an intelligent helmet according to claim 1, wherein The specific steps of Step S3 are as follows: Step S31: Perform object visual recognition on the super-resolution scene image and mark the objects in the scene image; Step S32: Perform positioning calculation on the objects in the scene image to obtain object positioning data; Step S33: Based on the object positioning data, perform object bounding box segmentation processing on the super-resolution scene image to obtain multiple scene object image bounding boxes; Step S34: Perform road semantic feature analysis on the super-resolution scene image and extract the road semantic features in the scene; Step S35: According to the road semantic features in the scene, perform precise correction of the bounding box positioning for the multiple scene object image bounding boxes to obtain multiple precise scene object bounding boxes.

6. The method for processing environmental perception data based on an intelligent helmet according to claim 5, characterized in that, The specific steps of Step S45 are as follows: Extract the upper and lower frame images based on the time series scene image frame sequence; Extract the timestamps of the upper and lower frame images and calculate the time interval between the upper and lower frame images; According to the time interval between the upper and lower frame images, perform object displacement analysis within the time period for the movement change trajectory of the scene object to generate object displacement data within the time interval; Traverse all the time series scene image frame sequences to generate object displacement data for all time intervals; Perform time axis sampling on the object displacement data for all time intervals to generate object displacement trajectories at different times; Analyze the change of scene structure within the time interval of the displacement trajectory of an object at different times to obtain the scene structure change data for multiple time periods; Calculate the optical flow change speed of the movement change trajectory of the scene object to generate the change speed of the scene object; Based on the change speed of the scene object, conduct dynamic scene structure evolution on the scene structure change data for multiple time periods to obtain dynamic scene evolution feature data.

7. The method for processing environmental perception data based on an intelligent helmet according to claim 1, wherein The specific steps of step S6 are as follows: Step S61: Detect the dynamic movement detail distortion of the dynamically attributed rendered scene model and identify the dynamic detail distortion data of object movement; Step S62: Locate the distorted detail parts based on the dynamic detail distortion data of object movement and mark the distorted detail parts of the model; Step S63: Perform dynamic distortion compensation on the distorted detail parts of the model to generate a distortion compensation rendered model; Step S64: Perform three-dimensional scene visualization processing on the distortion compensation rendered model to construct a scene rendering visualization model; Step S65: Visually present the scene of the scene rendering visualization model to the smart helmet.

8. An environmental perception data processing system based on an intelligent helmet, characterized in that, For executing the environmental perception data processing method based on a smart helmet as described in claim 1, including: A scene attribute module, which acquires the scene image in front of the user and multi-source environmental real-time perception data; analyzes the scene attribute state of the multi-source environmental real-time perception data and constructs a real-time scene attribute state diagram; An image reconstruction module, which performs dynamic contrast equalization adjustment on the scene image in front of the user and conducts super-resolution fusion reconstruction to construct a super-resolution scene image; A bounding box segmentation module, which performs object visual recognition on the super-resolution scene image and conducts object bounding box segmentation processing to obtain multiple precise scene object bounding boxes; A scene structure evolution module, which conducts optical flow movement change analysis on multiple precise scene object bounding boxes and conducts spatio-temporal scene structure evolution analysis to obtain dynamic scene evolution feature data; A dynamic attribute rendering module, which conducts dynamic structure evolution simulation on the super-resolution scene image based on the dynamic scene evolution feature data and the real-time scene attribute state diagram, and then conducts environmental attribute rendering to generate a dynamically attributed rendered scene model; A scene visualization module, which performs dynamic distortion compensation on the dynamically attributed rendered scene model to generate a distortion compensation rendered model; performs three-dimensional scene visualization processing on the distortion compensation rendered model to construct a scene rendering visualization model.

Images