An assisted driving method based on deep learning for data fitting and related device

Abstract

The application provides an assisted driving method based on deep learning data fitting and related equipment, and relates to the field of automatic control. In the method, the road surface space-time data set is determined by collecting historical road surface information, collecting time points and road surface position coordinates. Then, based on the road surface space-time data set, the road surface information data group including road surface temperature, humidity, wear state, pothole condition and repair record is taken as input data, and after determining the road surface condition category label, a deep learning model is trained, which can output the road surface condition category matched with the input data. Then, the real-time information of the road surface is obtained through the sensor and the monitoring equipment, and is input into the deep learning model to determine the real-time condition category of the road surface. Finally, the automatic driving strategy of the target vehicle is adjusted according to the weather forecast, the vehicle tire data, the real-time condition category of the road surface and the preset automatic driving behavior optimization model, so that safer and more intelligent driving is realized.

Description

Technical Field

[0001] This application relates to the field of automatic control, and more particularly to an assisted driving method and related equipment based on deep learning for data fitting. Background Technology

[0002] With the advancement of technology, autonomous driving technology has gradually become a research hotspot, and the perception component in autonomous driving systems is crucial for the accurate judgment of road conditions.

[0003] In existing technologies, sensors and monitoring devices, such as cameras and radar, are typically used to acquire real-time road information. These devices mainly detect basic visible conditions on the road surface, such as obstacles and lane markings, through vision and radar. They then perform preliminary analysis of the acquired real-time information based on relatively simple preset rules and algorithms to determine the basic condition of the road surface, such as the presence of obstacles or lane departure, and adjust the driving mode accordingly.

[0004] However, this approach has significant drawbacks. Sensors and monitoring devices often acquire real-time, localized information, making it difficult to comprehensively understand the historical conditions and spatiotemporal changes of the road surface. Moreover, the relatively simple, pre-set rules and algorithms are insufficient to accurately assess potential risks and complex road conditions. Therefore, the accuracy of existing methods in assessing complex road conditions is not high enough. Summary of the Invention

[0005] This application provides a driving assistance method and related equipment based on deep learning for data fitting, which can achieve a safer, smarter and more comfortable driving experience when the autonomous driving strategy is difficult to adjust accurately according to real-time road conditions, weather conditions, passenger conditions and road safety conditions in the prior art.

[0006] Firstly, this application provides an assisted driving method based on deep learning for data fitting, applied to a server. The method includes: determining a spatiotemporal road data set based on collected historical road surface information, collection time points, and road surface location coordinates; using a road surface information data set as input data for a deep learning model, the road surface information data set including road surface temperature, road surface humidity, road surface wear status, road surface repair records, and road surface pothole status; training a deep learning model after determining a road surface condition category label matching the input data, the deep learning model outputting a road surface condition category matching the input data; inputting real-time road surface information obtained from sensors and monitoring equipment into the deep learning model to determine the real-time road surface condition category; obtaining weather forecast information from a meteorological terminal and real-time tire data of the target vehicle from sensors; adjusting the autonomous driving strategy of the target vehicle based on the weather forecast information, real-time tire data, real-time road surface condition category, and a pre-set autonomous driving behavior optimization model, the autonomous driving behavior optimization model being pre-trained based on different weather forecast information, real-time tire data, real-time road surface condition categories, and corresponding braking distances.

[0007] By adopting the above technical solutions and applying deep learning models for road condition prediction, and training the model with historical data, real-time road conditions can be more accurately assessed. The integration of multi-source heterogeneous data, including historical road data, real-time road monitoring, weather forecasts, and tire sensor data, allows for more comprehensive road condition assessment through correlation analysis. Furthermore, the autonomous driving strategy can be adjusted in real time based on road conditions, achieving smarter and safer assisted driving and improving the driving experience.

[0008] In some embodiments of the first aspect, after obtaining real-time road information from sensors and monitoring devices and inputting that real-time road information into the deep learning model to determine the real-time road condition category, the method further includes: monitoring the number of vehicles within a set range in the current lane in real time using monitoring devices; comparing the number of vehicles with a preset congestion threshold to determine the congestion level of the current lane; monitoring the vehicle distribution in the two lanes matching the current lane in real time using monitoring devices; inputting the current lane congestion level, the vehicle distribution in the two lanes, and traffic regulation information obtained by querying a map database into a preset autonomous driving strategy model, which is established by training based on multiple actual driving data and traffic rules; and obtaining autonomous driving control parameters under the current road conditions.

[0009] By adopting the above technical solution, the server can monitor the number of vehicles in the current lane and the vehicle distribution in both lanes in real time. Combined with traffic regulation information, it uses an autonomous driving strategy model to determine the autonomous driving control parameters under the current road conditions. These technical features interact, enabling the autonomous driving strategy to adjust according to actual road conditions, avoiding congested sections and improving driving efficiency. Simultaneously, the cooperation between monitoring equipment and the autonomous driving strategy model allows for better handling of complex traffic situations, improving the safety and reliability of autonomous driving.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of using the road information data set as input data for the deep learning model based on the road spatiotemporal data set, the method further includes: after receiving real-time weather data from a meteorological terminal, determining the current weather conditions based on the real-time weather data, wherein the current weather conditions include at least sunny, rainy, or snowy weather; if it is determined to be sunny, determining the road surface humidity by monitoring the road moisture content through sensors; if it is determined to be rainy or snowy, correcting the road surface humidity parameters by combining the road moisture content and real-time precipitation; and determining a matching autonomous driving mode based on the corrected road surface humidity, wherein the autonomous driving mode includes at least slippery mode, normal mode, and snow mode.

[0011] By employing the above technical solution, the server can determine the current weather conditions based on real-time weather data sent from the meteorological station and correct the road surface humidity parameter by incorporating road moisture content. Then, it determines the appropriate autonomous driving mode based on the corrected road surface humidity. In this way, the autonomous driving strategy can be adjusted according to different weather and road conditions. For example, in slippery mode, the vehicle can adopt a more cautious driving strategy, improving driving safety. Simultaneously, the interaction between real-time weather data and road surface humidity correction makes the switching of autonomous driving modes more accurate and timely, enhancing the adaptability of autonomous driving.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, after adjusting the autonomous driving strategy of the target vehicle according to the real-time road condition category and the preset autonomous driving behavior optimization model, the method further includes: determining a real-time image inside the vehicle using an image acquisition device installed inside the vehicle; determining whether there are child passengers inside the vehicle based on the real-time image inside the vehicle combined with an image processing algorithm; and adjusting the autonomous driving strategy according to a preset child riding mode when it is determined that there are child passengers inside the vehicle.

[0013] By adopting the above technical solution, the server can determine whether there are child passengers in the carriage through image acquisition equipment and adjust the autonomous driving strategy according to the preset child riding mode. In this way, the autonomous driving strategy can pay more attention to the safety of child passengers, such as adjusting the vehicle speed and maintaining a safer distance. At the same time, the combination of image acquisition equipment and image processing algorithms can accurately identify child passengers, providing a reliable basis for adjusting the autonomous driving strategy and improving the safety and humanization of autonomous driving.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, after adjusting the autonomous driving strategy according to a preset child riding mode when it is determined that there are child passengers in the vehicle compartment, the method further includes: after receiving real-time location data sent by the in-vehicle navigation device, matching the information of accident-prone road sections pre-stored in the map database to determine whether the vehicle is within a set range of accident-prone road sections, where the accident-prone road section refers to a road area where the number of traffic accidents occurring within a set time period exceeds a preset threshold; when the vehicle is within the set range of the accident-prone road section, adjusting the corresponding optimized autonomous driving strategy according to the historical accident characteristics corresponding to the accident-prone road section.

[0015] By adopting the above technical solution, the server can adjust and optimize the autonomous driving strategy based on the historical accident characteristics of the accident-prone road segment when the vehicle enters the designated area. This allows the autonomous driving strategy to handle accident-prone road segments more cautiously. Simultaneously, the matching of information from the in-vehicle navigation device, map database, and accident-prone road segments can accurately and promptly determine the vehicle's location, providing timely basis for adjusting the autonomous driving strategy and improving the safety of autonomous driving.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after obtaining real-time road information based on sensors and monitoring devices, inputting the real-time road information into the deep learning model to determine the real-time road condition category, the method further includes: determining passenger status information and passenger vehicle use demand information monitored by in-vehicle monitoring devices, wherein the passenger status information includes at least passenger fatigue status and rest status, and the passenger vehicle use demand information includes at least passenger travel demand and emergency demand; adjusting the autonomous driving strategy of the target vehicle based on the real-time road condition category, the passenger status information, and the passenger vehicle use demand information, wherein the autonomous driving strategy includes, after determining travel demand, reducing vehicle speed within a preset scenic spot range in conjunction with map navigation information.

[0017] By adopting the above technical solutions, the server can adjust the autonomous driving strategy of the target vehicle based on real-time road conditions, passenger status information, and passenger travel needs. For example, after determining travel needs, reducing speed within a pre-defined scenic area using map navigation information can provide passengers with a better sightseeing experience. These technical features interact to enable the autonomous driving strategy to better meet passenger needs and improve passenger satisfaction. Simultaneously, considering passenger status information, such as passenger fatigue, can make the autonomous driving strategy more human-centered, ensuring passenger safety and comfort.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of adjusting the autonomous driving strategy of the target vehicle based on the weather forecast information, the real-time tire data, the real-time road condition category, and the preset autonomous driving behavior optimization model, the method further includes: receiving user feedback information on the autonomous driving strategy, the feedback information including the user's satisfaction with the autonomous driving strategy and specific opinions; analyzing the feedback information, and optimizing the autonomous driving strategy.

[0019] By adopting the above technical solution, the server can receive user feedback on the autonomous driving strategy and analyze and optimize it. In this way, the autonomous driving strategy can be continuously improved and refined to better meet user needs. User satisfaction and specific opinions provide important references for optimizing the autonomous driving strategy, enabling the autonomous driving system to continuously improve performance and enhance the user experience. At the same time, this feedback mechanism also promotes the development and progress of autonomous driving technology.

[0020] In a second aspect, this application provides a server comprising: one or more processors and a memory; the memory being coupled to the one or more processors, the memory being used to store computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the server to perform the methods described in the first aspect and any possible implementation thereof.

[0021] Thirdly, this application provides a computer-readable storage medium including instructions that, when executed on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer program product that, when run on a server, causes the server to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. The system determines the spatiotemporal data set of road surfaces based on historical road information. Then, using this dataset containing various road surface information as input to a deep learning model, the model is trained to output the road condition category after determining the road condition category. Real-time road surface information is then obtained and input into the model to determine the real-time condition category. Simultaneously, weather forecast information is obtained from a meteorological station, and real-time tire data of the target vehicle is acquired by sensors. Finally, based on weather forecast information, real-time tire data, real-time road condition categories, and a pre-set autonomous driving behavior optimization model, the autonomous driving strategy of the target vehicle is adjusted, effectively improving the safety and stability of autonomous driving.

[0025] 2. By employing image acquisition equipment and image processing algorithms to determine whether there are child passengers in the carriage and adjusting the autonomous driving strategy according to the preset child riding mode, the technical problem of insufficient attention to the needs of passengers in the vehicle in the existing technology is effectively solved, thereby achieving the technical effect of improving the safety and humanization of autonomous driving.

[0026] 3. By adopting the technical means of receiving and analyzing user feedback on autonomous driving strategies, the technical problem that existing autonomous driving strategies are unable to meet users' personalized needs is effectively solved, thereby achieving the technical effect of continuously improving autonomous driving performance and user experience. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an assisted driving method based on deep learning for data fitting in an embodiment of this application.

[0028] Figure 2 This is another flowchart illustrating the assisted driving method based on deep learning for data fitting in the embodiments of this application;

[0029] Figure 3 This is a schematic diagram of the physical device structure of a server in an embodiment of this application. Detailed Implementation

[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0032] For ease of understanding, the method provided in this implementation is described in process below. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a data fitting assisted driving method based on deep learning in an embodiment of this application.

[0033] S101. Based on the collected historical road surface information, collection time points, and road surface location coordinates, determine the road surface spatiotemporal data set;

[0034] When the server performs this step, it first needs to collect and organize a large amount of historical road surface data. This historical road surface information is a comprehensive collection of various data, reflecting the road's conditions and characteristics over a past period. This includes road surface temperature, humidity, wear and tear, repair records, road material information, and pothole conditions. These can be detected beforehand using equipment such as cameras and radar. Workers can use cameras to capture images of the road surface and analyze the image's color, texture, and other features, comparing them with an existing road surface feature table to determine the road condition. The road surface feature table contains typical color feature value ranges, texture descriptions, and corresponding road condition labels for different road surface conditions. For example, the color feature values ​​of a wet, slippery road surface may be within a specific range of brightness and saturation, and its texture may exhibit relatively uniform reflection without obvious graininess; icy roads typically have a lighter hue, and their texture is smooth and flat with almost no texture variation; pothole-prone roads may exhibit significant color differences due to shadows and the mixing of different materials, and their texture is characterized by irregular bumps and undulations with potential for scattered light and shadow. By comparing the features obtained from image analysis with the features in the road surface feature table, the road surface condition can be determined relatively accurately.

[0035] These data sources may also include: regular inspection reports from road management departments, real-time road condition information uploaded by vehicle sensors, satellite remote sensing images, historical weather data provided by meteorological departments, and road maintenance records from transportation departments. The server organizes this data according to time sequence and geographical location, forming a database containing multi-dimensional information. For example, for a specific road, the database might contain the following information:

[0036] On January 1, 2023, the location coordinates were (longitude: 116.3, latitude: 39.9), the road surface temperature was 5°C, the road surface humidity was 30%, the road surface wear condition was slight, and the last repair date was November 15, 2022.

[0037] On April 15, 2023, the location coordinates were (longitude: 116.3, latitude: 39.9), the road surface temperature was 18°C, the road surface humidity was 45%, the road surface wear condition was moderate, and the last repair date was March 20, 2023.

[0038] On July 30, 2023, the location coordinates were (longitude: 116.3, latitude: 39.9), the road surface temperature was 32°C, the road surface humidity was 20%, the road surface wear condition was severe, and the last repair date was March 20, 2023.

[0039] The server connects these data points to form a road surface condition curve that changes over time. This curve not only reflects the seasonal changes in road surface conditions but also shows the gradual wear and tear of the road surface as it gets used more, as well as the effect of road maintenance on improving road surface conditions.

[0040] S102. Based on the spatiotemporal data set of the road surface, the road surface information data set is used as the input data of the deep learning model. The road surface information data set includes road surface temperature, road surface humidity, road surface wear status, road surface repair records, and road surface pothole status.

[0041] In this step, the server delves into the spatiotemporal data set of the road surface, extracting data closely related to road conditions, such as road surface temperature, road surface humidity, road surface wear status, road surface repair records, and road surface pothole status, to form a road surface information data set. For example, the server might filter out road surface temperature data for a specific road segment within a specific time period from the spatiotemporal data set. If the road surface temperature of that segment fluctuates significantly at different points in time during this period, the server will record these temperature data points along with their corresponding time points, as part of the road surface information data set.

[0042] For road surface moisture, the server also retrieves relevant information from the dataset. For example, in rainy or humid weather conditions, the road surface moisture will increase significantly, and the server will record this moisture change data along with the corresponding weather and time information. Road surface wear is also an important consideration. The server analyzes information in the dataset regarding road surface smoothness, scratches, cracks, etc., to assess the degree of road surface wear. If a road section is found to have severe wear, the server will add the relevant wear data to the road surface information data group. Road surface repair records are equally important. If a road section has been repaired, the server will obtain information such as the repair time, the repaired location, and the post-repair condition, and add these records to the road surface information data group.

[0043] S103. After determining the road condition category label that matches the input data, a deep learning model is trained. This deep learning model is used to output the road condition category that matches the input data.

[0044] When the server performs this step, it first needs to define the category labels for the road surface conditions. These labels can be set according to actual needs and experience, and can be divided into different categories such as good, fair, poor, slippery, and icy. Each category label represents a specific road surface condition.

[0045] Next, the server associates the previously compiled road surface information data sets with corresponding road condition category labels. For example, a set of data with high road surface temperature, low road surface humidity, slight road surface wear, and no recent repair records might be labeled as a "good" road surface condition; while a set of data with low road surface temperature, high road surface humidity, severe road surface wear, and recent repair records might be labeled as a "poor" road surface condition. The server then uses this labeled data to train a deep learning model. Deep learning models typically have a multi-layered structure and can automatically learn features and patterns from the input data. During training, the model continuously adjusts its parameters to ensure that the output road surface condition category matches the labels of the input data as closely as possible. For example, the server can use a Convolutional Neural Network (CNN) as the deep learning model. CNNs can automatically extract useful features from the road surface information data sets, such as trends in road surface temperature, distribution of road surface humidity, and characteristics of road surface wear. These features are then combined and classified through fully connected layers to ultimately output the road surface condition category.

[0046] During training, the server inputs a large amount of road surface information data into the model and compares the model's output with the actual labels to calculate the loss function. Then, the backpropagation algorithm is used to update the model's parameters, gradually reducing the loss function. Through continuous iterative training, the model learns the complex relationship between different road surface information data sets and road condition categories. To better illustrate this, assume the server has the following data sets: Data 1: Road surface temperature 25°C, road surface humidity 40%, road surface wear condition slightly, no recent repair records, labeled "Good". Data 2: Road surface temperature 10°C, road surface humidity 80%, road surface wear condition moderate, recent repair records, labeled "Average". Data 3: Road surface temperature 0°C, road surface humidity 90%, road surface wear condition severely, recent repair records, labeled "Poor".

[0047] The server will feed this data into the deep learning model multiple times for training. In the early stages of training, the model's output may differ significantly from the actual labels, but as training progresses, the model will gradually learn the features and patterns of this data, and the output will become increasingly closer to the actual labels.

[0048] S104. After obtaining real-time road surface information from sensors and monitoring equipment, input the real-time road surface information into the deep learning model to determine the real-time road surface condition category.

[0049] During this step, the server receives real-time road surface information from sensors and monitoring devices. This information includes, but is not limited to, current road surface temperature, humidity, and wear condition. For example, a sensor might detect a current road surface temperature of 20°C, humidity of 50%, slight wear, and no obvious repair marks. The server compiles this real-time information into a dataset and inputs it into a pre-trained deep learning model. The deep learning model analyzes and judges the input real-time information based on previously learned patterns and features. The model automatically extracts key features from this information and compares and matches them with knowledge learned during training. For example, based on the input road surface temperature and humidity information, the model might determine that the current road surface is unlikely to be slippery or icy; based on the road wear condition information, it might determine that the road surface smoothness is good. Combining these judgments, the model outputs a real-time road surface condition category, such as "good."

[0050] Furthermore, the server continuously receives new real-time information and promptly inputs it into the model for judgment. This allows the model to adjust its output category in a timely manner based on changes in road conditions, ensuring the timeliness and accuracy of the autonomous driving strategy. For example, during vehicle operation, the server may acquire real-time road information every few seconds and input it into the model. If the vehicle moves from a dry road surface to a recently rained-off road surface, the server will promptly detect the change in road surface moisture and adjust the road condition category based on the model's judgment, thereby adjusting the autonomous driving strategy accordingly, such as reducing speed or increasing safe distance.

[0051] In some embodiments, the server can also receive data from in-vehicle monitoring devices, which may include cameras or other sensors. By analyzing this data, the server can determine the passenger's status information. Specifically, the server can analyze facial images of the passenger captured by the camera, detecting indicators such as blinking frequency and yawning frequency, as well as preset fatigue state thresholds, to determine whether the passenger is fatigued. The server can also determine whether the passenger is resting by analyzing data such as the passenger's body posture and breathing rate sent by the detection devices, as well as standard rest state data. For example, if the server detects that the passenger's blinking frequency has significantly decreased and they have yawned multiple times within 30 minutes, it may determine that the passenger is fatigued.

[0052] The server can also analyze passenger behavior patterns, destination information, and interactions with the in-vehicle system to determine passenger travel needs. Specifically, this includes travel needs, which can be determined by analyzing the destination set in the navigation system and whether passengers frequently check information about attractions along the route; and urgent needs, which can be determined by analyzing whether passengers frequently check arrival times in the in-vehicle system's map data, or by obtaining information from passenger conversations with voice devices.

[0053] Based on the above information, the server comprehensively considers real-time road conditions, passenger status, and passenger travel needs to adjust the autonomous driving strategy of the target vehicle. Specific strategies may include: when tourism needs are identified, the server will reduce speed within a preset scenic area using map navigation information. When a passenger is determined to have an urgent need, the server may choose a faster route and slightly increase the speed within permissible limits, provided safety is ensured. When a passenger is detected resting, the server may suggest adjusting to a quieter road section and further control the in-vehicle volume control to reduce interior noise. In this way, the server can dynamically adjust the autonomous driving strategy based on real-time road conditions, passenger status, and needs, ensuring driving safety while maximizing the satisfaction of passengers' personalized needs, providing a more intelligent and humanized riding experience.

[0054] S105. Obtain weather forecast information through the meteorological terminal and obtain real-time tire data of the target vehicle through sensors;

[0055] The server establishes a communication connection with the meteorological terminal. The meteorological terminal can be a service interface provided by a professional meteorological agency, or it can be the result of data interaction between the vehicle's own meteorological monitoring module and a broader meteorological network. The server sends a request to the meteorological terminal to obtain weather forecast information for the target vehicle's location and surrounding area. Upon receiving the request, the meteorological terminal determines the weather conditions of the relevant area based on the vehicle's location information (obtained through the vehicle's GPS module and provided to the server). Weather forecast information includes, but is not limited to, weather type (e.g., sunny, cloudy, rainy, snowy), temperature range, humidity range, wind speed, and wind direction. Additionally, the target vehicle is equipped with various tire sensors that monitor relevant tire parameters in real time, including but not limited to tire pressure sensors and tire temperature sensors. The tire pressure sensor measures the internal tire pressure; excessively high or low tire pressure can affect vehicle handling and braking performance, and may also lead to accelerated tire wear. The tire temperature sensor monitors tire temperature. During vehicle operation, tire friction with the road surface generates heat; excessively high tire temperature may reduce tire lifespan and even cause safety hazards such as tire blowouts. Real-time monitoring of tire temperature allows for timely detection of abnormalities and appropriate measures.

[0056] S106. Adjust the autonomous driving strategy of the target vehicle according to the weather forecast information, the real-time tire data, the real-time road condition category, and the preset autonomous driving behavior optimization model. The autonomous driving behavior optimization model is trained in advance based on different weather forecast information, the real-time tire data, the real-time road condition category, and the corresponding braking distance.

[0057] Specifically, to build an autonomous driving behavior optimization model that can adapt to various environments, a large amount of comprehensive data can be collected in advance, covering various weather conditions such as sunny, rainy, and snowy days. For each weather condition, detailed weather parameters such as temperature, humidity, wind speed, wind direction, and precipitation are recorded. Simultaneously, vehicle driving data under these weather conditions is collected, including vehicle speed, acceleration, braking distance, and steering angular velocity. This data reflects the vehicle's actual driving status and performance in different weather environments. Regarding road conditions, data on different road surface types, such as braking distance parameters for dry, wet, and icy surfaces, are collected. Furthermore, detailed data under different road surface condition categories are collected, including parameters such as road surface temperature, humidity, wear level, pothole severity, and road material. This road-related data helps the model understand the impact of different road surface conditions on vehicle driving. For tire conditions, relevant data, including tire pressure parameters, are collected. Tire pressure and temperature affect the contact area and friction between the tire and the road surface, thus affecting vehicle driving performance. By collecting vehicle driving data under different tire pressures, the impact of tire pressure on vehicle driving can be assessed. The collected raw data needs to be preprocessed to better facilitate training in deep neural networks. For continuous numerical data such as vehicle speed, acceleration, braking distance, and steering angular velocity, normalization is typically used. The convolutional layers of the autonomous driving behavior optimization model perform convolution operations by sliding convolution kernels across the input data, automatically extracting local features. Taking a combination of vehicle speed and weather parameters as an example, the convolutional layer can extract pattern features of vehicle speed changes under different weather conditions. For instance, vehicle speed may decrease in rainy weather; the convolutional layer can capture the correlation between this speed change and rainy weather parameters. Similarly, it can extract data such as tire temperature, tire pressure, and braking distance under different road surface and weather conditions. Subsequently, a fully connected layer integrates and comprehensively analyzes all the extracted features. It considers the interrelationships between various environmental factors and their combined impact on autonomous driving behavior. For example, it comprehensively considers factors such as road surface temperature, humidity, wear level, weather conditions, and tire pressure to determine the final autonomous driving behavior adjustment strategy.

[0058] During model training, the model is repeatedly trained using a large amount of collected training data. For each set of input data (containing various environmental parameters) and the corresponding expected output (correct autonomous driving behavior adjustment), the error between the model's predicted output and the expected output is calculated. Common error calculation methods include mean squared error. Then, based on this error, the network's weight parameters are updated using the backpropagation algorithm. The backpropagation algorithm is a gradient descent-based optimization algorithm that adjusts the network weights according to the direction of the error gradient, making the model's predicted output gradually approach the expected output. When the model receives different environmental inputs, it adjusts the vehicle speed based on the learned patterns and relationships. For example, in rainy and wet road conditions, the model may reduce the vehicle speed based on learned features such as increased braking distance and decreased tire grip when the road surface is wet, as well as the corresponding speed adjustment strategy. The specific speed reduction may be fine-tuned based on actual conditions, such as precipitation intensity and road surface humidity. In snowy or icy road conditions, due to lower road friction, the vehicle's braking distance increases significantly, and the model will reduce the vehicle speed more significantly to ensure driving safety.

[0059] Through the detailed technical means and processes described above, the autonomous driving behavior optimization model can continuously learn and adapt to different environmental inputs, thereby gaining the ability to accurately adjust autonomous driving behaviors such as vehicle speed, steering, and braking according to different environmental inputs.

[0060] In some embodiments, the server monitors the number of vehicles within a set range in the current lane in real time using monitoring devices. These monitoring devices may include sensors such as cameras or radars pre-installed at predetermined locations on target vehicles, or other devices with monitoring capabilities, which are not limited here. They are capable of capturing vehicle information in the lane in real time. For example, a camera can capture images of the lane and identify the number and location of vehicles using image processing technology; radar can detect the presence and distance of vehicles by emitting electromagnetic waves and receiving reflected waves. The server compares the monitored number of vehicles in the current lane with a preset congestion threshold to determine the level of congestion in the current lane. The preset congestion threshold can be set based on experience and traffic rules. For example, if the number of vehicles per kilometer in the lane exceeds a certain number, the lane can be considered congested. Assuming the preset congestion threshold is 30 vehicles per kilometer, when the number of vehicles within a set range in the current lane is detected to be 40 vehicles per kilometer, the server will determine that the current lane is highly congested.

[0061] Simultaneously, the server monitors the vehicle distribution in both lanes corresponding to the current lane in real time using monitoring devices. These devices collect information such as the number, speed, and spacing of vehicles in both lanes. For example, radar can detect the relative speed and distance of vehicles in both lanes, thus understanding the vehicle distribution. Next, the server inputs the current lane congestion level, the vehicle distribution in both lanes, and traffic regulation information obtained by querying a map database into a pre-set autonomous driving strategy model. This autonomous driving strategy model is built based on multiple real-world driving data and traffic rules. It can comprehensively consider various factors based on the input information to formulate the most suitable autonomous driving strategy. For example, the map database may contain traffic regulation information such as speed limits and lane divisions for that road segment. Suppose in a specific scenario, the server detects that there are many vehicles in the current lane, the speed is slow, and the congestion level is high; the vehicles in the lanes on both sides are relatively few and the speed is fast. The traffic regulation information in the map database shows that vehicles are allowed to switch lanes safely in this road segment. Based on this input information, the autonomous driving strategy model calculates the optimal autonomous driving control parameters, such as suggesting that the vehicle switch to the left lane at an appropriate time and adjust the speed to adapt to the new lane conditions. In this way, the server can obtain the autonomous driving control parameters under the current road conditions, thereby achieving more intelligent and safer autonomous driving.

[0062] In some practical examples, the server can also actively receive user feedback on the autonomous driving strategy. This feedback includes user satisfaction with the autonomous driving strategy and specific suggestions. For example, users may evaluate the performance of the autonomous driving strategy during a trip through the in-vehicle system interface or related applications, expressing their satisfaction with aspects such as speed control, lane keeping, and braking sensitivity. They may also offer specific suggestions, such as hoping the vehicle can accelerate more smoothly in certain situations or decelerate earlier on specific road sections.

[0063] After receiving this feedback, the server analyzes it using a series of technical methods. First, the server uses natural language processing (NLP) to analyze the text of the user's specific opinions, extracting key information such as the specific problems mentioned and the expected directions for improvement. Then, the server quantifies the user satisfaction data, for example, dividing satisfaction into different levels and assigning a corresponding score to each level. Next, the server optimizes the autonomous driving strategy based on the analysis results. For example, if multiple users report that the vehicle accelerates too abruptly in a certain type of road condition, the server will adjust the relevant control parameters to allow the vehicle to accelerate more smoothly in similar road conditions. Or, if users generally want the vehicle to maintain a higher cruising speed on highways, the server will appropriately increase the default cruising speed on highways, provided it is safe to do so.

[0064] Furthermore, the server utilizes machine learning algorithms to analyze historical feedback information to predict potential user needs and preferences. For example, by analyzing feedback from a large number of users under different seasons and road conditions, the server can identify patterns, such as higher user interest in vehicle performance in snow mode during winter, or specific expectations regarding autonomous driving strategies near tourist attractions during peak seasons. Based on these predictions, the server can proactively optimize autonomous driving strategies to better meet user needs.

[0065] In this embodiment, by using a deep learning model to learn and analyze historical road information, and combining it with multi-dimensional data such as real-time road conditions, weather conditions, passenger conditions, and road safety information, the system can more accurately judge the current road conditions and make corresponding driving strategy adjustments. This effectively solves the problem that the autonomous driving strategy in the prior art is difficult to adjust accurately according to complex road conditions, thereby achieving a safer, smarter, and more comfortable autonomous driving experience.

[0066] In certain scenarios, such as family travel, especially family road trips or daily transportation with child passengers, or for some special medical vehicles for children, such as pediatric ambulances or shuttle buses to children's hospitals, there is a need for autonomous driving modes that can improve the safety of child passengers.

[0067] In light of the above scenarios, the method provided in this implementation will now be described in more detail. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating the assisted driving method based on deep learning for data fitting in this application embodiment.

[0068] S201. Determine real-time images of the interior of the vehicle using image acquisition equipment installed inside the vehicle;

[0069] When the server performs this step, it activates the image acquisition equipment installed inside the vehicle. This equipment typically includes high-definition cameras that can capture real-time images of the passenger compartment from all angles. For example, cameras are installed in the front and rear seats, as well as in various corners of the vehicle, to ensure coverage of the entire passenger space. The cameras continuously capture images at a certain frame rate, converting the real-time situation inside the passenger compartment into image data. This image data is transmitted to the server in real time for processing. After receiving the image data, the server performs preliminary processing and analysis. First, the server performs noise reduction and enhancement processing on the images to improve their quality and clarity. For example, by removing noise points from the images and enhancing their contrast and brightness, objects and passengers inside the passenger compartment are presented more clearly. Then, the server segments and identifies the images, distinguishing different areas and objects inside the passenger compartment. For example, it divides the area into seat areas, passenger areas, and luggage areas for more accurate passenger identification later. In practical applications, assuming the vehicle is in motion, the image acquisition equipment inside the vehicle continuously captures real-time images of the passenger compartment. The server receives this image data and performs the processing described above.

[0070] S202. Determine whether there are child passengers in the carriage based on real-time images of the carriage and image processing algorithms;

[0071] The server uses object detection algorithms to detect the presence of human figures in the image. These algorithms are typically based on deep learning techniques, such as convolutional neural networks, which can automatically extract features from the image and determine the presence of human silhouettes. If a human figure is detected, the server further analyzes its features.

[0072] For each detected human body, the server extracts multiple features, such as height, body shape, and head proportion. For example, it estimates height by analyzing the pixel height of the human body in the image, determines body size by analyzing the outline shape, and identifies whether the individual is a child by comparing the head-to-body ratio. In addition, the server considers other features, such as facial features and clothing. When determining whether a passenger is a child, the server uses preset criteria. For example, it may identify individuals with a height below a certain threshold, a large head proportion, and a small body as potentially child passengers. Simultaneously, the server incorporates facial feature analysis, such as children's unique facial contours and expressions, to improve the accuracy of the judgment.

[0073] To illustrate this better, suppose the server receives a real-time image of the interior of a train carriage. After preprocessing, an object detection algorithm detects two human figures in the image. Then, feature extraction is performed on these two figures. One figure is found to be shorter, with a larger head and smaller body, and facial features suggest it might be a child. The server combines this information and determines that this figure is likely a child passenger. Simultaneously, analysis of the other figure reveals that it is taller and larger, not matching the characteristics of a child, and is therefore identified as an adult passenger. In this way, the server can accurately identify the presence of child passengers from real-time images of the carriage, providing crucial information for subsequent adjustments to the autonomous driving strategy.

[0074] If it is determined that there are no child passengers in the carriage, proceed to step S203;

[0075] If it is determined that there are child passengers in the carriage, proceed to step S204.

[0076] S203, Do not adjust the autonomous driving strategy;

[0077] When the server determines, based on the judgment result of step S202, that there are no child passengers in the carriage, or determines in step S205 that the vehicle is not within the set range of accident-prone road sections, it will decide not to adjust the autonomous driving strategy. In this case, the autonomous driving strategy will maintain its previous settings to adapt to the current road conditions, traffic conditions, and other relevant factors.

[0078] S204. Adjust the autonomous driving strategy according to the preset child passenger mode;

[0079] When executing this step, the server adjusts the autonomous driving strategy accordingly based on the preset child passenger mode. The preset child passenger mode typically takes into account the special needs and safety factors of children to ensure their safety and comfort during the ride. For example, the server will reduce the vehicle's speed. Because children's bodies are relatively fragile and their reaction ability in sudden situations is not as good as adults, a lower speed can reduce the severity of injury in the event of an accident. Assuming the vehicle's normal speed is 60 km / h, when a child passenger is detected, the server will reduce the speed to 40 km / h. Simultaneously, the server increases the safety distance between the vehicle and the vehicle in front. This provides more braking time, avoiding impact to the child during emergency braking. For example, the safety distance may be increased from 50 meters to 80 meters.

[0080] In addition, the server adjusts the vehicle's acceleration and braking sensitivity to make acceleration and braking smoother, reducing the impact of inertia on children. For example, it reduces acceleration to avoid discomfort for children during sudden acceleration; and it adjusts braking sensitivity to make braking gentler, preventing sudden braking that could cause children to fall. Regarding steering, the server makes the vehicle's steering smoother, avoiding sharp turns and other abrupt maneuvers to ensure the stability of children inside the vehicle.

[0081] S205. After receiving the real-time location data sent by the vehicle navigation device, the system matches the information of accident-prone road sections pre-stored in the map database to determine whether the vehicle is within the set range of accident-prone road sections. The accident-prone road sections refer to road areas where the number of traffic accidents that have occurred in the past set time exceeds a preset threshold.

[0082] During this step, the server receives real-time location data from the in-vehicle navigation device. This location data includes the vehicle's current latitude and longitude information. The server matches this real-time location data with pre-stored accident-prone road segment information in the map database. The map database contains a large amount of road information, including historical traffic accident data for various road segments. Based on this data, the server determines which road segments have experienced more traffic accidents than a preset threshold within a set time period, thus marking these road segments as accident-prone sections. When the server receives the vehicle's real-time location data, it determines whether the vehicle is within the defined range of these accident-prone road segments. For example, the defined range of a certain accident-prone road segment is an area with a radius of 500 meters centered on that road segment. If the vehicle's real-time location is within this range, the server determines that the vehicle is within the defined range of the accident-prone road segment.

[0083] If it is determined that the vehicle is within the set range of the accident-prone road section, then step S206 is executed;

[0084] If it is determined that the vehicle is not within the set range of accident-prone road sections, then step S203 is executed.

[0085] S206. Adjust the corresponding optimized autonomous driving strategy based on the historical accident characteristics of the accident-prone road section.

[0086] When performing this step, the server conducts a thorough analysis of the historical accident characteristics corresponding to the accident-prone road segment. These characteristics may include the time of the accident, weather conditions, accident type (such as collision, rear-end collision, etc.), cause of the accident (such as speeding, illegal lane changing, etc.), and traffic flow at the time of the accident. For example, if historical accident data shows that there are more accidents on this road segment in rainy weather, and the main reason is that the road surface is slippery and causes vehicles to lose control, the server will adjust the autonomous driving strategy based on real-time weather data. This could include reducing vehicle speed, increasing braking response sensitivity, and adjusting tire traction control parameters to improve vehicle stability on slippery roads. Furthermore, if historical accident characteristics indicate that the road segment is prone to rear-end collisions during peak hours, the server will appropriately increase the safe distance from the vehicle in front when the vehicle enters the road segment during peak hours, while closely monitoring the dynamics of vehicles in both lanes and avoiding frequent lane changes to reduce the risk of rear-end collisions. For example, if it is found that accidents on this section of road occur frequently at night and the cause is poor visibility, the server will adjust the vehicle's lighting settings, increase the brightness and illumination range of the headlights, and remind the driver to remain highly vigilant.

[0087] By conducting detailed analysis of historical accident characteristics on accident-prone road sections and adjusting autonomous driving strategies accordingly, the server can effectively improve vehicle driving safety on these road sections and reduce the probability of accidents.

[0088] In this embodiment, by comprehensively considering the situation of passengers inside the vehicle, vehicle location information, and road safety conditions, and adjusting the autonomous driving strategy precisely based on these factors, the problem that existing autonomous driving technologies cannot fully take into account various factors to ensure driving safety and passenger comfort is effectively solved, thereby achieving the goal of improving the safety, humanization, and adaptability of autonomous driving.

[0089] The server in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference]. Figure 3 This is a schematic diagram of the physical device structure of a server in an embodiment of this application.

[0090] It should be noted that, Figure 3 The server structure shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0091] like Figure 3As shown, the server includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 302 or a program loaded from storage portion 308 into Random Access Memory (RAM) 303, such as performing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0092] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0093] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the various functions defined in the present invention.

[0094] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0096] Specifically, the server in this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the assisted driving method based on deep learning for data fitting provided in the above embodiment.

[0097] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the server described in the above embodiments; or it may exist independently and not assembled into the server. The storage medium carries one or more computer programs that, when executed by a processor of the server, cause the server to implement the deep learning-based data fitting assisted driving method provided in the above embodiments.

[0098] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0099] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0100] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for assisted driving based on deep learning data fitting, applied to a server, comprising: receiving a driving data set; inputting the driving data set into a deep learning model to obtain a driving data fitting result; and sending the driving data fitting result to a terminal device. The method comprises: determining a road surface space-time data set according to each historical road surface information, a collection time point and a road surface position coordinate; inputting a road surface information data group as input data of a deep learning model based on the road surface space-time data set, the road surface information data group comprising road surface temperature, road surface humidity, road surface wear state, road surface repair record and road surface pothole state; training a deep learning model after determining a road surface condition category label matched with the input data, the deep learning model being used to output a road surface condition category matched with the input data; inputting real-time road surface information obtained according to a sensor and a monitoring device into the deep learning model to determine a real-time road surface condition category; obtaining weather forecast information through a weather terminal and obtaining real-time tire data of a target vehicle through a sensor; adjusting an automatic driving strategy of the target vehicle according to the weather forecast information, the real-time tire data, the real-time road surface condition category and a preset automatic driving behavior optimization model, the automatic driving behavior optimization model being trained in advance according to different weather forecast information, real-time tire data, real-time road surface condition categories and corresponding braking distances.

2. The method of claim 1, wherein, After the step of inputting the real-time road surface information into the deep learning model to determine a real-time road surface condition category, the method further comprises: monitoring a number of vehicles within a set range on a current lane in real time through a monitoring device; comparing the number of vehicles with a preset congestion threshold to determine a congestion degree of the current lane; monitoring vehicle distribution conditions of two lanes matched with the current lane in real time through the monitoring device; inputting the congestion degree of the current lane, the vehicle distribution conditions of the two lanes and traffic regulation information obtained by querying a map database into a preset automatic driving strategy model, the automatic driving strategy model being established according to a plurality of actual driving data and traffic rules; obtaining an automatic driving control parameter under a current road condition.

3. The method of claim 1, wherein, After the step of inputting the road surface information data group as input data of the deep learning model based on the road surface space-time data set, the method further comprises: judging a current weather condition according to real-time weather data received from the weather terminal, the current weather condition at least comprising sunny, rainy or snowy weather; if the current weather condition is determined to be sunny, determining the road surface humidity by monitoring road moisture content through the sensor; if the current weather condition is determined to be rainy or snowy, correcting a parameter of the road surface humidity in combination with the road moisture content and real-time weather precipitation; determining a matched automatic driving mode according to the corrected road surface humidity, the automatic driving mode at least comprising a wet and slippery mode, a normal mode and a snowy mode.

4. The method of claim 1, wherein, After the step of adjusting the automatic driving strategy of the target vehicle according to the weather forecast information, the real-time tire data, the real-time road surface condition category and the preset automatic driving behavior optimization model, the method further comprises: determining real-time images in a vehicle cabin through an image acquisition device arranged in the vehicle cabin; determining whether a child passenger exists in the vehicle cabin according to real-time images in the vehicle cabin and an image processing algorithm; adjusting the automatic driving strategy according to a preset child driving mode when it is determined that a child passenger exists in the vehicle cabin.

5. The method of claim 4, wherein, after the step of adjusting the automatic driving strategy according to the preset child driving mode when it is determined that a child passenger exists in the vehicle cabin, the method further comprises: after receiving the real-time position data sent by the vehicle navigation device, matching the pre-stored accident-prone road section information in the map database to determine whether the vehicle is within the set range of the accident-prone road section, the accident-prone road section refers to a road area where the number of traffic accidents within a set time exceeds a preset threshold; adjusting the corresponding optimized automatic driving strategy according to the historical accident characteristics corresponding to the accident-prone road section when the vehicle is within the set range of the accident-prone road section.

6. The method of claim 1, wherein, after inputting the real-time road surface information into the deep learning model to determine the real-time road surface condition category, the method further comprises: determining passenger state information and passenger vehicle demand information monitored by the monitoring device in the vehicle, the passenger state information at least includes passenger fatigue state and rest state, and the passenger vehicle demand information at least includes passenger tourism demand and urgent demand; adjusting the automatic driving strategy of the target vehicle according to the real-time road surface condition category, the passenger state information and the passenger vehicle demand information, the automatic driving strategy including reducing the vehicle speed within a preset scenic spot range after determining the tourism demand in combination with the map navigation information.

7. The method of claim 1, wherein, after adjusting the automatic driving strategy of the target vehicle according to the weather forecast information, the real-time tire data, the real-time road surface condition category and the preset automatic driving behavior optimization model, the method further comprises: receiving feedback information of the automatic driving strategy from a user, the feedback information including user satisfaction and specific opinions on the automatic driving strategy; analyzing the feedback information to optimize the automatic driving strategy.

8. A server, characterized by The server comprises one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors invoke the computer instructions to enable the server to perform the method of any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions run on the server, the server performs the method of any one of claims 1-7.

10. A computer program product, characterised in that, When the computer program product runs on the server, the server performs the method of any one of claims 1-7.

Images