Methods and devices for generating vehicle speed limits, vehicles
By combining a multimodal end-to-end model and a reference speed optimizer with user preference information, personalized vehicle speed limits are generated, solving the problems of large speed limit control errors and long tails in existing technologies, and achieving accuracy and consistency in global speed limits.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MOMENTA (SUZHOU) TECHNOLOGY CO LTD
- Filing Date
- 2024-08-02
- Publication Date
- 2026-06-30
AI Technical Summary
Existing vehicle speed limit control methods cannot meet the flexibility requirements, have large control errors and are difficult to cope with complex road conditions, resulting in long-tail problems and failing to achieve accuracy and consistency in global speed limits.
By acquiring vehicle navigation information, user preference information, and route information, speed limit optimization is performed using a multimodal end-to-end model. By combining a reference speed optimizer and user preference weights, personalized vehicle speed limit information is generated, and accurate speed limit generation is achieved by fusing navigation data through a neural network.
It improves the accuracy of speed limit generation, reduces long-tail problems, and achieves global speed limit consistency and meets users' personalized needs during autonomous driving.
Smart Images
Figure CN121459613B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving technology, and in particular to a method and device for generating vehicle speed limits, and a vehicle. Background Technology
[0002] With the rapid development of intelligent driving technology, driverless vehicles will automatically drive along a planned path when driving on the road. In this process, in order to avoid vehicle collisions, it is necessary to control the vehicle's driving according to the maximum or minimum speed limit.
[0003] Currently, vehicle speed control typically employs fixed speed limits, such as configuring matching upper or lower speed limits on different routes to ensure safe autonomous driving within these limits. However, relying solely on configured speed limits fails to meet the vehicle's flexible speed control requirements. Furthermore, speed limit control generates significant control errors, leading to long-tail problems. It also struggles to effectively and accurately plan for global vehicle control, particularly in situations like construction, detours, rainy nights, and narrow roads—problems that are difficult to describe verbally. Summary of the Invention
[0004] In view of this, this application provides a method and device for generating vehicle speed limits, as well as a vehicle, with the main purpose of solving the problem of poor accuracy in existing global speed limit control for vehicles.
[0005] According to one aspect of this application, a method for generating vehicle speed limits is provided, comprising:
[0006] The system acquires the target vehicle's navigation information, user preference information, vehicle route information, and driving information, including navigation map information, camera bird's-eye view, and navigation broadcast information.
[0007] The driving information is classified based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: driving training samples of different vehicles are obtained, and a deep learning model is trained based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples, camera bird's-eye view samples, and navigation broadcast samples with speed limit classification for different vehicles. The deep learning model is constructed based on a multimodal neural network.
[0008] Speed limit optimization is performed based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information to obtain vehicle speed limit information. The multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model.
[0009] This application embodiment uses user preferences as the basis for determining speed limits and combines a multimodal end-to-end model to optimize the speed limit solution by combining user preferences with path information and navigation information. Based on the large model, it provides users with personalized choices, which greatly improves the accuracy of speed limit generation for different scenarios. Furthermore, by using a multimodal end-to-end model, it reduces the long-tail problem in determining speed limits and realizes a numerical description of speed limits, thereby improving the consistency of global speed limits in principle during autonomous driving.
[0010] Furthermore, the speed limit optimization process based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information includes:
[0011] The vehicle speed limit information is obtained by optimizing the input parameters of the reference speed optimizer based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information.
[0012] The reference speed optimizer is constructed based on the functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables, and the optimization control variables are constrained based on the user preference information.
[0013] Furthermore, the optimization process based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information as input parameters for the reference speed optimizer to obtain the vehicle speed limit information includes:
[0014] Determine the preference weights corresponding to the user preference information, and determine the solution coefficients of the optimization control variables based on the preference weights. The user preference information includes at least one of exercise preference information, energy saving preference information, and comfort preference information.
[0015] The reference speed optimizer is numerically solved based on the solved coefficients to generate vehicle speed limit information.
[0016] Furthermore, after obtaining the vehicle speed limit information, the method further includes:
[0017] Obtain historical speed limit information and historical vehicle speed information;
[0018] Based on the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information, a speed-distance comparison curve is generated and displayed.
[0019] Furthermore, the method also includes:
[0020] The original images captured by the vehicle camera are fused using a neural network to obtain a bird's-eye view of the camera, and navigation broadcast information, navigation map information, vehicle navigation information and user preference information are collected based on the vehicle communication equipment.
[0021] According to another aspect of this application, a vehicle speed limit generation device is provided, comprising:
[0022] The acquisition module is used to acquire the target vehicle's navigation information, user preference information, vehicle route information, and driving information. The driving information includes navigation map information, camera bird's-eye view, and navigation broadcast information.
[0023] The first processing module is used to classify the driving information based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: obtaining driving training samples of different vehicles, and training a deep learning model based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples with marked speed limit classifications for different vehicles, camera bird's-eye view samples, and navigation broadcast samples. The deep learning model is constructed based on a multimodal neural network.
[0024] The second processing module is used to perform speed limit optimization processing based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information to obtain vehicle speed limit information. The multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model.
[0025] Furthermore, the second processing module is specifically used to perform optimization and solution based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information as input parameters of the reference speed optimizer to obtain the vehicle speed limit information;
[0026] The reference speed optimizer is constructed based on the functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables, and the optimization control variables are constrained based on the user preference information.
[0027] Furthermore, the processing module is specifically used to determine the preference weights corresponding to the user preference information, and to determine the solution coefficients of the optimization control variable based on the preference weights. The user preference information includes at least one of motion preference information, energy saving preference information, and comfort preference information. The reference speed optimizer is numerically solved based on the solution coefficients to generate vehicle speed limit information.
[0028] Furthermore, the device also includes:
[0029] The output module is specifically used to acquire historical speed limit information and historical vehicle speed information; generate a speed-distance comparison curve based on the historical speed limit information, the historical vehicle speed information and the vehicle speed limit information, and display the speed-distance comparison curve.
[0030] Furthermore, the device also includes:
[0031] The acquisition module is used to perform neural network fusion on the raw images acquired by the vehicle camera to obtain a bird's-eye view of the camera, and to acquire navigation broadcast information, navigation map information, vehicle navigation information and user preference information based on the vehicle communication equipment.
[0032] According to one aspect of this application, a vehicle is provided, including the aforementioned vehicle speed limit generating device.
[0033] According to another aspect of this application, a readable storage medium is provided that stores a program or instructions thereon, which, when executed by a processor, implement the steps of the vehicle speed limit generation method described above.
[0034] According to another aspect of this application, a computer device is provided, comprising: at least one processor coupled to a memory, the memory storing a program or instructions that run on the processor, the program or instructions, when executed by the processor, implementing the steps of the vehicle speed limit generation method described above.
[0035] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0036] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0037] Figure 1 A flowchart illustrating a method for generating vehicle speed limits according to an embodiment of this application is shown;
[0038] Figure 2 This illustration shows a schematic diagram of a multimodal speed limiting information provided in an embodiment of this application;
[0039] Figure 3 This illustration shows a final speed limit diagram provided in an embodiment of this application;
[0040] Figure 4 This illustration shows a schematic diagram of the entire process of multimodal speed limiting provided in an embodiment of this application;
[0041] Figure 5 This paper shows a block diagram of a vehicle speed limit generation device according to an embodiment of this application;
[0042] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation
[0043] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0044] This application provides a method for generating vehicle speed limits, such as... Figure 1 As shown, the method includes:
[0045] 101. Obtain the target vehicle's navigation information, user preference information, vehicle route information, and driving information.
[0046] In this embodiment, during the operation of the autonomous vehicle, the autonomous driving processor, as the current execution entity, can be a processor configured on the vehicle itself or a cloud server matched with the vehicle. During this time, as the autonomous vehicle travels along a predetermined route or a real-time planned route, it can generate real-time vehicle navigation information, user preference information, and vehicle path information. The vehicle navigation information is the navigation information generated by the navigation system to guide the target vehicle to its destination, such as a navigation route. Different navigation system operators provide navigation maps in different formats. In this case, prior information can be used to determine the navigation map, thereby knowing the expected route the target vehicle will take. This embodiment does not impose specific limitations on this. User preference information represents the preferences selected by the user when choosing manual driving or riding in the target vehicle. User preference information includes at least one of the following: exercise preference information, energy-saving preference information, and comfort preference information. This embodiment does not impose specific limitations on this. Vehicle path information is the path recorded during the vehicle's operation or the planned path for autonomous driving. The path includes the curvature and gradient of the vehicle's path. This embodiment does not impose specific limitations on this. In addition, during the vehicle's operation, user preference information can be obtained based on the vehicle's front-end interface, and vehicle route information can be loaded based on the autonomous driving system. This application embodiment does not impose specific limitations on this. Driving information includes navigation map information, camera bird's-eye view (BEV), and navigation broadcast information. The camera bird's-eye view is formed by fusing raw images captured by the vehicle's cameras through a neural network and can serve as a road environment image. Each frame of the road environment image is a BEV image obtained by fusing raw images captured by multiple vehicle cameras based on a neural network Transformer model. That is, after the vehicle cameras capture raw images of the area around the vehicle, they are input into the neural network model Transformer for fusion to obtain multiple frames of BEV images. Navigation broadcast information is the broadcast content generated by the navigation system indicating the route the target vehicle is traveling to its destination. This information can be obtained simultaneously with the navigation map information, and this application embodiment does not impose specific limitations on this.
[0047] It should be noted that the vehicles in the autonomous driving scenario are those equipped with automatic control systems, including passenger cars and commercial vehicles. Common passenger car models include, but are not limited to, sedans, SUVs, and multi-person commercial vehicles. Common commercial vehicle models include, but are not limited to, pickup trucks, minivans, dump trucks, cargo trucks, tractor units, trailers, and mining vehicles. In this case, the vehicles can achieve autonomous driving based on the automatic control system.
[0048] 102. The driving information is classified and processed based on the reference speed multimodal end-to-end model to obtain multimodal speed limit information.
[0049] In this embodiment, multimodal speed limit information is used to characterize different reference speed limits. This multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model. The multimodal end-to-end model is a multi-input multi-output neural network model for end-to-end speed limit classification. Preferably, the multimodal end-to-end model is a Transformer neural network model; however, this embodiment does not impose specific limitations. The navigation map information can be a map generated by the navigation system according to the destination, matching the navigation route. The navigation announcement information can be voice content generated by the navigation system according to the destination, matching the navigation route, such as "Turn left in 100 meters." This embodiment does not impose specific limitations. Furthermore, the driving information is classified based on the reference speed multimodal end-to-end model that has completed model training, resulting in different speed limit classification results as multimodal speed limit information. In different speed limit implementation scenarios, different speed limit classification results can include comfort speed limit classification, sport speed limit classification, and energy-saving speed limit classification, etc. This embodiment does not impose specific limitations.
[0050] It should be noted that the reference velocity multimodal end-to-end model is trained in the following way:
[0051] The driving training samples of different vehicles are obtained, and the deep learning model is trained based on the driving training samples to obtain a reference speed multimodal end-to-end model. The driving training samples include navigation map samples with marked speed limit classifications, camera bird's-eye view samples, and navigation broadcast samples of different vehicles. The deep learning model is constructed based on a multimodal neural network.
[0052] To improve the consistency of global speed limits from a fundamental level, and because it is necessary to classify multi-dimensional driving information to generate multi-dimensional speed limit results, and then combine this with user preferences to further determine the optimal speed limit, thereby achieving a more accurate global speed limit generation, this embodiment of the application pre-acquires driving information from different vehicles for model training. The driving information from different vehicles can be data collected by the data collection vehicle or data transmitted back by vehicle users; this embodiment of the application does not specifically limit this. Since driving information is a multi-dimensional input parameter, the reference speed multimodal end-to-end model is a multi-input multi-output Transformer model, thus obtaining... Figure 2The different speed limit classification results shown indicate that, in this embodiment, the Transformer model directly classifies the acquired image data, broadcast content, user preferences, and other front-end collected data, outputting the classification results from the end, achieving end-to-end processing efficiency and generalization. The Transformer model is a machine learning model based on an attention mechanism. It can improve model training speed through self-attention and process data in parallel, enabling weighted combination of different driving information when processing various speeds, thus better understanding the meaning of the navigation map information, camera view, and navigation broadcast information as input parameters. The Transformer model structure in this embodiment consists of an encoder and a decoder. The encoder transforms the input sequence (e.g., navigation broadcast text) into a series of contextualized embeddings, composed of multiple identical layers. Each layer consists of two sub-layers: a self-attention layer and a feedforward fully connected layer. The decoder takes the encoder's output and the target sequence (e.g., speed limits at different driving positions) as input to generate a probability distribution for each position in the target sequence. The decoder consists of multiple identical layers, each layer comprising three sub-layers: a self-attention layer, an encoder-decoder attention layer, and a feedforward fully connected layer. It trains a multimodal end-to-end model of reference speed using sample data based on navigation map information, camera bird's-eye views, and navigation announcements. Based on the trained model, it classifies the real-time driving information to obtain multiple different categories of reference speed limits. These different categories of speed limits can correspond to user preferences, including but not limited to speed limits based on driving preferences, energy-saving preferences, and comfort preferences; this embodiment does not impose specific limitations.
[0053] 103. Based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information, speed limit optimization processing is performed to obtain vehicle speed limit information.
[0054] After determining the multimodal speed limit information, the current executing entity uses the multimodal speed limit information, vehicle navigation information, user preference information, and vehicle route information as input parameters for a reference speed optimizer to obtain the speed limit information for different vehicle positions, i.e., the vehicle speed limit information. During the optimization process, a functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables can be constructed as the reference speed optimizer. This relationship is then used for polynomial solving in the mathematical domain to obtain the final vehicle speed limit information, reducing long-tail problems and solving the problem of speed limits being difficult to describe in words.
[0055] In another embodiment of this application, the step of optimizing the speed limit based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information includes:
[0056] The vehicle speed limit information is obtained by optimizing the input parameters of the reference speed optimizer based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information.
[0057] To better align speed limits with user preferences and meet the flexible speed limit needs of different users, this application embodiment addresses the flexibility of speed limit generation based on multimodal input. During speed limit optimization, the current executing entity constructs a reference speed optimizer to solve the speed limit optimization problem. The reference speed optimizer is constructed based on the functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables, and can be based on the relationship between speed and distance. Specifically, the speed limit distance interval used in this application embodiment is represented as s. total =max(v ego *10,60), s total v represents the distance corresponding to the real-time vehicle speed. ego For real-time vehicle speed, and therefore, s total Divide into N equal parts, v i For s i The speed limit value in the multimodal speed limit information of the location, a i For s i The acceleration of position, and the expression for constructing the reference vehicle speed optimizer, is Equation 1:
[0058] Among them, u i To optimize the control variables, the optimized control variables are constrained based on the user preference information, that is, they can be constrained based on at least one of the motion preference information, energy saving preference information, and comfort preference information, so as to optimize the reference speed optimizer and obtain the vehicle speed limit information.
[0059] In another embodiment of this application, the step of optimizing the vehicle speed limit information based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information as input parameters of the reference speed optimizer to obtain the vehicle speed limit information includes:
[0060] Determine the preference weights corresponding to the user preference information, and determine the solution coefficients of the optimization control variables based on the preference weights;
[0061] The reference speed optimizer is numerically solved based on the solved coefficients to generate vehicle speed limit information.
[0062] To effectively solve the reference speed optimizer and use it as the final vehicle speed limit value to improve the accuracy of vehicle speed control, the current executing entity first determines the preference weights of user preference information when optimizing the reference speed optimizer, thereby determining the solution coefficients. Since user preference information includes at least one of motion preference information, energy-saving preference information, and comfort preference information, corresponding preference weights are assigned for motion preference, energy-saving preference, and comfort preference. These preference weights are used to determine the solution coefficients of the optimization control variables, achieving the goal of providing users with personalized choices based on the large model.
[0063] Specifically, the constraint expression for the optimized control variable, Formula 2, is as follows:
[0064] The cost of motion is represented as: J eff_i =λ eff (v i -v set_i ) 2 v set_i Let J be the desired maximum speed limit. The cost of energy saving is expressed as: J energy_i =λ energy (F d_i ), F d_i As the driving force, F d_i =f(a i ,v i ,slope i ), a i For acceleration, v i For vehicle speed, slope i Let the slope be the acceleration cost.
[0065] λ acc , λ brake Used to control the acceleration from not exceeding the upper limit of acceleration a. upper_i and the lower bound of deceleration a lower_iThe control cost is expressed as: λ eff For motion preference weights, λ energy For the weight of energy-saving preferences, λ a For comfort preference weighting, the coefficients to be calculated include motion cost, energy saving cost, and acceleration cost, in order to optimally solve for the control variable u and obtain the vehicle speed limit information. Furthermore, the motion cost characterizes the vehicle's performance under different motion states, the energy saving cost characterizes the vehicle's performance under different energy saving states, and the acceleration cost characterizes the vehicle's performance under different comfort levels. Then, through a predetermined control coefficient λ... control The constraint expression (Formula 1) is obtained by combining the optimal control variables, which serve as the control cost characterizing the optimal control state of the vehicle, with the motion cost, energy saving cost, and acceleration cost. At this point, the control coefficient λ... control The configuration can be based on the optimization solution requirements, and the embodiments in this application do not impose specific limitations.
[0066] It should be noted that the current execution entity can be pre-configured with motion preference weights matching different motion preference information, energy-saving preference weights matching different energy-saving preference information, and comfort preference weights matching different comfort preference information. This allows users to directly retrieve the motion preference weights, energy-saving preference weights, and comfort preference weights after entering motion preference information, energy-saving preference information, and comfort preference information in the front-end interface. This application embodiment does not impose any specific limitations.
[0067] In another embodiment of this application, the steps further include:
[0068] Obtain historical speed limit information and historical vehicle speed information;
[0069] Based on the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information, a speed-distance comparison curve is generated and displayed.
[0070] To achieve a visual representation of speed limit information and provide users with flexible options, in this embodiment, after obtaining the vehicle speed limit information, the current executing entity can output this information on the target vehicle's front-end interface. By adding a display device for vehicle speed and distance, the user gains greater confidence in viewing the information. At this time, the vehicle front-end interface can be either the front-end interface of an in-vehicle device or the front-end interface of a user terminal, allowing the user to view or choose whether to control the vehicle speed according to this speed limit information. This embodiment does not impose specific limitations on this.
[0071] Specifically, when the current executing entity obtains historical speed limit information and historical vehicle speed information, it ensures speed limits for a long distance in the future through global speed limits. The stability of speed limits is greatly improved through the visualized relationship between distance and vehicle speed. At this time, the historical speed limit information represents the maximum or minimum speed limit within a predetermined historical time period, which can be scheduled based on a cloud storage server or terminal memory; this application embodiment does not impose specific limitations. Furthermore, since the optimal speed limit obtained can be represented as a curve relating distance and speed, this curve is further converted into a curve relating vehicle speed / acceleration and time, expressed as (s... i ,v i ,a i ) i＝0,1...N Furthermore, historical speed limit information, historical vehicle speed information, and vehicle speed limit information are used to generate speed-distance comparison curves for display, such as... Figure 3 As shown, based on the user's comfort preferences, the corresponding historical speed limit, historical vehicle speed, and the optimally solved vehicle speed limit are displayed using different visualization curves to meet the user's visualization needs.
[0072] In another embodiment of this application, the steps further include:
[0073] The speed of the target vehicle is controlled based on the vehicle speed limit information.
[0074] In a specific implementation scenario of this application, after obtaining vehicle speed limit information, the current executing entity controls the driving speed of the vehicle during the autonomous driving process based on this vehicle speed limit information. For example, it obtains the real-time speed of the target vehicle and calculates based on acceleration that after a predetermined time, the predicted speed will be greater than the maximum speed limit in the vehicle speed limit information, indicating that the acceleration is too fast and will cause a large power consumption. Then, the current executing entity can adjust the vehicle's acceleration to decelerate in order to save power. This application embodiment is not specifically limited.
[0075] In another specific implementation scenario of this application, after obtaining the vehicle speed limit information, the current executing entity controls the deceleration of the vehicle during obstacle avoidance in the autonomous driving process based on this vehicle speed limit information. For example, if an obstacle is detected in front of the vehicle, the collision avoidance speed is calculated. If this collision avoidance speed is greater than the maximum speed limit in the vehicle speed limit information, it indicates that the deceleration is too fast, which will generate a large inertia and poor comfort. In this case, the current executing entity can adjust the deceleration point position of the vehicle to extend the braking distance, thereby improving the comfort of vehicle deceleration. This application embodiment does not make specific limitations.
[0076] In another specific implementation scenario of this application, after obtaining the vehicle speed limit information, the current executing entity controls the different motion states of the vehicle during the autonomous driving process based on this vehicle speed limit information. For example, if the target vehicle is a stable and sporty autonomous driving style, after collecting the expected average driving speed of the vehicle, if this average speed is greater than the maximum speed limit in the vehicle speed limit information, the expected average driving speed will be reduced so that the vehicle maintains a constant motion style during driving and meets different driving experience needs.
[0077] In another embodiment of this application, the steps further include:
[0078] The original images captured by the vehicle camera are fused using a neural network to obtain a bird's-eye view of the camera, and navigation broadcast information, navigation map information, vehicle navigation information and user preference information are collected based on the vehicle communication equipment.
[0079] To achieve multimodal speed classification, the current executing entity generates a camera bird's-eye view based on real-time images captured by cameras on each vehicle. This view includes frame image data of vehicles at different times, containing the vehicle's position at each time. The vehicle-side cameras can be mounted on the vehicle or fixed-position cameras capturing images of the vehicle; this embodiment does not impose specific limitations. Furthermore, since the current executing entity acts as the terminal optimizing speed limits, it can be a cloud controller or a terminal server, etc. The navigation broadcast information, navigation map information, and vehicle navigation information are generated by the navigation system and can be collected from the cloud navigation system via vehicle communication devices, pre-determined destination navigation map information and navigation broadcast information; this embodiment does not impose specific limitations. In a specific implementation scenario, the user can input their preferred information for autonomous driving control of the vehicle based on in-vehicle devices, which is then fed back to the current executing entity via vehicle communication devices; this embodiment does not impose specific limitations.
[0080] In one embodiment of this application, such as Figure 4As shown, after the target vehicle's multi-frame images are acquired by the vehicle-side camera Multi-Camera, the Transformer model first performs image processing to obtain the camera bird's-eye view, namely BEV-(1)...BEV-(i), and then combines the navigation broadcast information and navigation map (sd map) information to determine the multimodal speed limit information through the multimodal end-to-end model Reference Speed Transformer that has completed model training. At this time, at least three multimodal speed limits can be obtained and displayed based on the vehicle speed and distance curve. Simultaneously, the system collects real-time driving information of the target vehicle (such as navigation map information, road curvature, and slope), personalized user preference information, and vehicle navigation information (such as cloud navigation information) through 5G communication / GPS modules. This information, combined with multimodal speed limit information, is then input into the Reference Speed Optimizer for optimization, ultimately yielding the target vehicle's final speed limit. The front-end interface displays the speed curve corresponding to the prior information of the empirical speed limit, the speed curve selected based on the user's preferred mode, and the solved final speed limit, all based on distance and speed curves, for user viewing. Furthermore, if the user selects a speed limit based on this preference, it can be input into the Motion Planning & Control system for autonomous driving planning; this embodiment does not impose specific limitations.
[0081] This application proposes a method for generating vehicle speed limits. By using user preferences as the basis for determining speed limits and combining a multimodal end-to-end model to optimize the speed limit solution by combining user preferences with path information and navigation information, the accuracy of speed limit generation for different scenarios is greatly improved, meeting the personalized speed limit needs of different users. Furthermore, the multimodal end-to-end model reduces the long-tail problem in determining speed limits, making the global speed limit of the vehicle more consistent during autonomous driving.
[0082] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this application provides a vehicle speed limit generation device, such as... Figure 5 As shown, the device includes:
[0083] The acquisition module 31 is used to acquire the target vehicle's navigation information, user preference information, vehicle route information, and driving information, wherein the driving information includes navigation map information, camera bird's-eye view, and navigation broadcast information.
[0084] The first processing module 32 is used to classify the driving information based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: obtaining driving training samples of different vehicles, and training a deep learning model based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples, camera bird's-eye view samples, and navigation broadcast samples with speed limit classification for different vehicles. The deep learning model is constructed based on a multimodal neural network.
[0085] The second processing module 33 is used to perform speed limit optimization processing based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information to obtain vehicle speed limit information. The multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model.
[0086] Furthermore, the second processing module is specifically used to perform optimization and solution based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information as input parameters of the reference speed optimizer to obtain the vehicle speed limit information;
[0087] The reference speed optimizer is constructed based on the functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables, and the optimization control variables are constrained based on the user preference information.
[0088] Furthermore, the processing module is specifically used to determine the preference weights corresponding to the user preference information, and to determine the solution coefficients of the optimization control variable based on the preference weights. The user preference information includes at least one of motion preference information, energy saving preference information, and comfort preference information. The reference speed optimizer is numerically solved based on the solution coefficients to generate vehicle speed limit information.
[0089] Furthermore, the device also includes:
[0090] The output module is specifically used to acquire historical speed limit information and historical vehicle speed information; generate a speed-distance comparison curve based on the historical speed limit information, the historical vehicle speed information and the vehicle speed limit information, and display the speed-distance comparison curve.
[0091] Furthermore, the device also includes:
[0092] The acquisition module is used to perform neural network fusion on the raw images acquired by the vehicle camera to obtain a bird's-eye view of the camera, and to acquire navigation broadcast information, navigation map information, vehicle navigation information and user preference information based on the vehicle communication equipment.
[0093] This application proposes a vehicle speed limit generation device. By using user preferences as the basis for speed limit determination and combining a multimodal end-to-end model to optimize the speed limit solution by combining user preferences with path information and navigation information, the device greatly improves the accuracy of speed limit generation for different scenarios, meets the personalized speed limit needs of different users, and reduces the long-tail problem of speed limit determination through the multimodal end-to-end model, making the global speed limit of the vehicle more consistent during autonomous driving.
[0094] According to one embodiment of this application, a vehicle is provided, including the aforementioned vehicle speed limit generation device.
[0095] According to one embodiment of this application, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the vehicle speed limit generation method described above.
[0096] Figure 6 A schematic diagram of a computer device according to an embodiment of this application is shown, including at least one processor coupled to a memory. The memory stores a program or instructions that run on the processor, which, when executed by the processor, implement the steps of the vehicle speed limit generation method described above. The specific embodiments of this application do not limit the specific implementation of the computer device.
[0097] like Figure 6 As shown, the computer device may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.
[0098] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.
[0099] Communication interface 404 is used to communicate with other network elements such as clients or other servers.
[0100] The processor 402 is used to execute program 410, specifically to execute the relevant steps in the above-described vehicle speed limit generation method embodiment.
[0101] Specifically, program 410 may include program code that includes computer operation instructions.
[0102] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.
[0103] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0104] Specifically, program 410 can be used to cause processor 402 to perform the following operations:
[0105] The system acquires the target vehicle's navigation information, user preference information, vehicle route information, and driving information, including navigation map information, camera bird's-eye view, and navigation broadcast information.
[0106] The driving information is classified based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: driving training samples of different vehicles are obtained, and a deep learning model is trained based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples, camera bird's-eye view samples, and navigation broadcast samples with speed limit classification for different vehicles. The deep learning model is constructed based on a multimodal neural network.
[0107] Speed limit optimization is performed based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information to obtain vehicle speed limit information. The multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model.
[0108] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.
[0109] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method of generating a vehicle speed limit, characterized by, include: The system acquires the target vehicle's navigation information, user preference information, vehicle route information, and driving information, including navigation map information, camera bird's-eye view, and navigation broadcast information. The driving information is classified based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: driving training samples of different vehicles are obtained, and a deep learning model is trained based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples, camera bird's-eye view samples, and navigation broadcast samples with speed limit classification for different vehicles. The deep learning model is constructed based on a multimodal neural network. Based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information, speed limit optimization processing is performed to obtain vehicle speed limit information.
2. The method according to claim 1, characterized in that, The speed limit optimization process based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information includes: The vehicle speed limit information is obtained by optimizing the input parameters of the reference speed optimizer based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information. The reference speed optimizer is constructed based on the functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables, and the optimization control variables are constrained based on the user preference information.
3. The method according to claim 2, characterized in that, The process of optimizing the vehicle speed limit information based on the multimodal speed limit information, vehicle navigation information, user preference information, and vehicle route information as input parameters for the reference speed optimizer, to obtain the vehicle speed limit information includes: Determine the preference weights corresponding to the user preference information, and determine the solution coefficients of the optimization control variables based on the preference weights. The user preference information includes at least one of exercise preference information, energy saving preference information, and comfort preference information. The reference speed optimizer is numerically solved based on the solved coefficients to generate vehicle speed limit information.
4. The method according to claim 3, characterized in that, After obtaining the vehicle speed limit information, the method further includes: Obtain historical speed limit information and historical vehicle speed information; Based on the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information, a speed-distance comparison curve is generated and displayed.
5. The method according to any one of claims 1-4, characterized in that, The method further includes: The original images captured by the vehicle camera are fused using a neural network to obtain a bird's-eye view of the camera, and navigation broadcast information, navigation map information, vehicle navigation information and user preference information are collected based on the vehicle communication equipment.
6. A vehicle speed limit generation device, characterized in that, include: The acquisition module is used to acquire the target vehicle's navigation information, user preference information, vehicle route information, and driving information. The driving information includes navigation map information, camera bird's-eye view, and navigation broadcast information. The first processing module is used to classify the driving information based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: obtaining driving training samples of different vehicles, and training a deep learning model based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples with marked speed limit classifications for different vehicles, camera bird's-eye view samples, and navigation broadcast samples. The deep learning model is constructed based on a multimodal neural network. The second processing module is used to perform speed limit optimization processing based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information to obtain vehicle speed limit information. The multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model.
7. A vehicle, characterized in that, Includes the vehicle speed limit generation device as described in claim 6.
8. A computer device, characterized in that, It includes at least one processor coupled to a memory, the memory storing a program or instructions that run on the processor, the program or instructions, when executed by the processor, implementing the steps of the method for generating a vehicle speed limit as claimed in any one of claims 1 to 5.
9. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the method for generating vehicle speed limits as described in any one of claims 1 to 5.