Vehicle speed limit generation method and device and vehicle

Abstract

The invention discloses a vehicle speed limit generation method and device and a vehicle. The invention relates to the technical field of intelligent driving, and the method comprises the following steps: obtaining vehicle navigation information, user preference information, vehicle path information and driving information of a target vehicle, the driving information comprising navigation map information, camera aerial view and navigation broadcast information; classifying the driving information based on a reference speed multi-modal end-to-end model to obtain multi-modal speed limit information, the multi-modal speed limit information comprising different speed limit classification results; speed limit optimization processing is carried out based on the multi-mode speed limit information, the vehicle navigation information, the user preference information and the vehicle path information, and vehicle speed limit information is obtained. According to the method, the more accurate vehicle speed limit can be generated, so that the flexible vehicle control requirement of a user is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to a vehicle speed limit generation method and device and a vehicle. BACKGROUND

[0002] With the rapid development of intelligent driving technology, when an unmanned vehicle drives on the road, it will automatically drive according to the planned path. In this process, in order to avoid vehicle collision, the vehicle needs to be controlled according to the maximum or minimum limit value of the vehicle speed.

[0003] At present, when controlling the vehicle speed limit, a fixed speed limit is usually used for control, for example, configuring a matching speed upper limit or speed lower limit on different routes to meet the purpose of safe automatic driving according to the speed limit. However, only based on the configured speed limit for control, the flexible speed control demand of the vehicle cannot be met, and a large control error will be generated in the speed limit control process, which causes a long tail problem, and it is difficult to describe the display problem with language for construction, detour, rainy night, narrow road and the like, and the vehicle global control cannot be effectively and accurately planned. SUMMARY

[0004] Therefore, the present application provides a vehicle speed limit generation method and device and a vehicle, which mainly aims to solve the problem of poor accuracy of existing vehicle global speed limit control.

[0005] According to one aspect of the present application, a vehicle speed limit generation method is provided, comprising:

[0006] obtaining vehicle navigation information, user preference information, vehicle path information and driving information of a target vehicle, wherein the driving information includes navigation map information, camera bird's eye view and navigation broadcast information;

[0007] classifying and processing the driving information based on a reference speed multi-modal end-to-end model to obtain multi-modal speed limit information, wherein the multi-modal speed limit information includes different speed limit classification results, and wherein the reference speed multi-modal end-to-end model is obtained by training in the following manner: obtaining driving training samples of different vehicles, performing model training on a deep learning model based on the driving training samples to obtain a reference speed multi-modal end-to-end model, wherein the driving training samples include navigation map samples, camera bird's eye view samples and navigation broadcast samples of different vehicles labeled with speed limit classification, and the deep learning model is constructed based on a multi-modal neural network;

[0008] performing speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information to obtain vehicle speed limit information, wherein the multi-modal speed limit information is obtained by processing the driving information of the vehicle based on a multi-modal end-to-end model.

[0009] The embodiment of the application takes user preferences as the basis for speed limit determination, and combines a multi-modal end-to-end model to optimize the speed limit of user preferences combined with path information and navigation information. On the basis of a large model, the user is given personalized selection, which greatly improves the accuracy of speed limit generation for different scenarios, reduces the long-tail problem of determining the speed limit through a multi-modal end-to-end model, realizes the numerical description of the speed limit, and improves the consistency of the global speed limit in the automatic driving process.

[0010] Further, the speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information comprises:

[0011] Based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information as input parameters of a reference speed optimizer, the vehicle speed limit information is obtained by optimization and solution.

[0012] The reference speed optimizer is constructed based on the functional relationship between the vehicle speed, the vehicle acceleration, the vehicle position and the optimization control variable, and the optimization control variable is constrained based on the user preference information.

[0013] Further, the speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information as input parameters of a reference speed optimizer, the vehicle speed limit information is obtained by optimization and solution.

[0014] The preference weight corresponding to the user preference information is determined, and the solution coefficient of the optimization control variable is determined based on the preference weight, and the user preference information includes at least one of motion preference information, power saving preference information and comfort preference information.

[0015] The reference speed optimizer is numerically solved based on the solution coefficient to generate vehicle speed limit information.

[0016] Further, after obtaining the vehicle speed limit information, the method further comprises:

[0017] Obtain historical speed limit information and historical vehicle speed information.

[0018] Generate a speed-distance comparison curve based on the historical speed limit information, the historical vehicle speed information and the vehicle speed limit information, and display the speed-distance comparison curve.

[0019] Further, the method further comprises:

[0020] The original image collected by the vehicle camera is fused by a neural network to obtain a camera bird's eye view, and navigation broadcast information, navigation map information, vehicle navigation information and user preference information are collected based on a vehicle communication device.

[0021] According to another aspect of the present application, a vehicle speed limit generation device is provided, comprising:

[0022] An acquisition module is configured to acquire vehicle navigation information, user preference information, vehicle path information and driving information of a target vehicle, wherein the driving information comprises navigation map information, a camera bird's eye view and navigation broadcast information;

[0023] A first processing module is configured to perform classification processing on the driving information based on a reference speed multi-modal end-to-end model to obtain multi-modal speed limit information, wherein different speed limit classification results are included in the multi-modal speed limit information, and the reference speed multi-modal end-to-end model is obtained by training a deep learning model based on driving training samples of different vehicles, wherein the driving training samples include navigation map samples, camera bird's eye view samples and navigation broadcast samples of different vehicles with labeled speed limit classification, and the deep learning model is constructed based on a multi-modal neural network.

[0024] A second processing module is configured to perform speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information to obtain vehicle speed limit information, wherein the multi-modal speed limit information is obtained by processing the driving information of the vehicle based on a multi-modal end-to-end model.

[0025] Further, the second processing module is specifically configured to perform optimization solving based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information as input parameters of a reference speed optimizer to obtain the vehicle speed limit information.

[0026] Further, the second processing module is specifically configured to perform optimization solving based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information as input parameters of a reference speed optimizer to obtain the vehicle speed limit information.

[0027] Further, the processing module is specifically configured to determine a preference weight corresponding to the user preference information, and determine a solving coefficient of the optimization control variable based on the preference weight, wherein the user preference information includes at least one of motion preference information, power saving preference information and comfort preference information; and perform numerical solving on the reference speed optimizer based on the solving coefficient to generate vehicle speed limit information.

[0028] Further, the device further comprises:

[0029] The output module is specifically configured to acquire historical speed limit information and historical vehicle speed information; generate a speed-distance comparison curve based on the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information, and display the speed-distance comparison curve.

[0030] Further, the device further comprises:

[0031] The acquisition module is configured to perform neural network fusion on original images collected by a vehicle camera to obtain a camera bird's eye view, and collect navigation broadcast information, navigation map information, vehicle navigation information, and user preference information based on a vehicle communication device.

[0032] According to an aspect of the present application, a vehicle is provided, comprising the above-mentioned vehicle speed limit generation device.

[0033] According to another aspect of the present application, a readable storage medium is provided, which stores a program or instructions, and the program or instructions are executed by a processor to implement the steps of the above-mentioned vehicle speed limit generation method.

[0034] According to still another aspect of the present application, a computer device is provided, comprising: at least one processor, the processor and a memory are coupled, the memory stores a program or instructions running on the processor, and the program or instructions are executed by the processor to implement the steps of the above-mentioned vehicle speed limit generation method.

[0035] The above description is only a summary of the technical solutions of the present application, in order to enable the technical means of the present application to be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0036] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings indicate the same or similar components. In the drawings:

[0037] Figure 1 A flow chart of a vehicle speed limit generation method provided by an embodiment of the present application is shown;

[0038] Figure 2 A multi-modal speed limit information diagram provided by an embodiment of the present application is shown;

[0039] Figure 3 A final speed limit diagram provided by an embodiment of the present application is shown;

[0040] Figure 4 This illustration shows a schematic diagram of the entire process of multimodal speed limiting provided in an embodiment of this application;

[0041] Figure 5 This paper shows a block diagram of a vehicle speed limit generation device according to an embodiment of this application;

[0042] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0043] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0044] This application provides a method for generating vehicle speed limits, such as... Figure 1 As shown, the method includes:

[0045] 101. Obtain the target vehicle's navigation information, user preference information, vehicle route information, and driving information.

[0046] In the embodiments of the present application, the automatic driving processor as the current execution subject during the driving process of the unmanned vehicle can be a processor configured for the vehicle end, or a cloud server matched with the vehicle, etc. At this time, the target vehicle of the unmanned vehicle can generate vehicle navigation information, user preference information and vehicle path information in real time during the driving process according to the predetermined route or the real-time planned route. The vehicle navigation information is navigation information generated by a navigation system for the target vehicle to drive to the destination, for example, a navigation route. Different navigation system operators provide navigation maps in different forms, at this time, the navigation map can be determined through prior information, so as to obtain the road expected to be driven by the target vehicle, which is not limited in the embodiments of the present application. The user preference information is the preference selected by the user when selecting manual driving or riding the target vehicle, and the user preference information includes at least one of motion preference information, power saving preference information and comfort preference information, which is not limited in the embodiments of the present application. The vehicle path information is the path recorded during the driving process of the vehicle or the planned path expected to be automatically driven by the vehicle, etc. At this time, the path includes the curvature and slope of the vehicle driving, which is not limited in the embodiments of the present application. In addition, the target vehicle can obtain the user preference information based on the front-end interface of the vehicle during the driving process, and load the vehicle path information based on the automatic driving system, which is not limited in the embodiments of the present application. The driving information includes navigation map information, camera bird's eye view (BEV) and navigation broadcast information. The camera bird's eye view is formed by fusing the original images collected by the vehicle camera through the neural network, which can be used as a road environment image. Each frame of road environment image is obtained by fusing the original images collected by multiple vehicle cameras based on the neural network Transformer model, that is, the original images collected by the vehicle camera around the vehicle are input into the neural network model Transformer for fusion to obtain multiple frames of BEV images. The navigation broadcast information is the broadcast content of the route driving condition generated by the navigation system for the target vehicle driving to the destination, which can be obtained at the same time when the navigation map information is obtained, which is not limited in the embodiments of the present application.

[0047] It should be noted that the vehicle is a vehicle with an automatic control system in an automatic driving scene, including passenger cars and commercial vehicles. Common models of passenger cars include but are not limited to cars, sport utility vehicles, multi-person business cars, etc. Common models of commercial vehicles include but are not limited to pickup trucks, microbuses, self-loading vehicles, cargo trucks, tractors, trailers and mining vehicles, etc. At this time, the vehicle can realize automatic driving based on the automatic control system.

[0048] 102, classifying and processing the driving information based on the reference speed multi-modal end-to-end model to obtain multi-modal speed limit information.

[0049] In the embodiment of the present application, the multi-modal speed limit information is used to represent different reference speed limits. At this time, the multi-modal speed limit information is obtained by processing the driving information of the vehicle based on a multi-modal end-to-end model. The multi-modal end-to-end model is a multi-input and multi-output neural network model for end-to-end speed limit classification. Preferably, the multi-modal end-to-end model is a Transformer neural network model, which is not limited in the embodiment of the present application. The navigation map information can be a map of the matching navigation route generated by the navigation system according to the destination. The navigation broadcast information can be the voice content of the matching navigation route generated by the navigation system according to the destination, such as a left turn 100 meters ahead, which is not limited in the embodiment of the present application. Further, the reference speed multi-modal end-to-end model based on the completed model training classifies and processes the driving information to obtain different speed limit classification results as multi-modal speed limit information. In different speed limit implementation scenarios, different speed limit classification results can include comfortable speed limit classification, sports speed limit classification, and power saving speed limit classification, which are not limited in the embodiment of the present application.

[0050] It should be noted that the reference speed multi-modal end-to-end model is trained in the following way:

[0051] The driving training samples of different vehicles are obtained, and the deep learning model is trained based on the driving training samples to obtain the reference speed multi-modal end-to-end model. The driving training samples include navigation map samples, camera bird's eye view samples, and navigation broadcast samples of different vehicles with labeled speed limit classification. The deep learning model is constructed based on a multi-modal neural network.

[0052] In order to improve the consistency of global speed limit from the principle layer, and since the multi-dimensional driving information needs to be classified to obtain multi-dimensional speed limit results, the optimal speed limit is further determined in combination with user preferences to achieve the effect of global speed limit, and the speed limit generation is more accurate. In the embodiment of the present application, the driving information of different vehicles is obtained in advance for model training. The driving information of different vehicles can be data collected by a data collection vehicle or data returned by a vehicle user, which is not limited in the embodiment of the present application. Since the driving information is a multi-dimensional input parameter, the reference speed multi-modal end-to-end model is a multi-input and multi-output Transformer model, so that the multi-modal speed limit information is obtained as Figure 2According to the different speed limit classification results shown, in the embodiment of the present application, the Transformer model directly classifies the data collected by the front end, such as image data, broadcast content, and user preferences, and outputs the classification results from the end, realizing the efficiency and generalization of end-to-end processing. The Transformer model is a machine learning model based on attention mechanism, which can improve the model training speed through self-attention mechanism and can process data in parallel, so that when processing each speed, different driving information can be weighted and combined to better understand the meaning of the navigation map information, camera bird's eye view and navigation broadcast information as input parameters. The Transformer model structure in the embodiment of the present application can be composed of an encoder (Encoder) and a decoder (Decoder). The encoder converts the input sequence (such as navigation broadcast text) into a series of contextualized embedding vectors, which is composed of multiple identical layers. Each layer is composed of two sub-layers, namely self-attention layer and feedforward layer. The decoder takes the output of the encoder and the target sequence (such as the speed limit value of different driving positions) as input to generate the probability distribution of each position in the target sequence. The decoder is composed of multiple identical layers, each of which is composed of three sub-layers, namely self-attention layer, encoder-decoder attention layer and feedforward layer, etc. to train the multi-modal end-to-end model based on the sample data of navigation map information, camera bird's eye view and navigation broadcast information, and classify the real-time collected driving information based on the trained model to obtain multiple different categories of reference speed limit values. At this time, different categories of speed limits can be corresponded based on the type of user preferences, including but not limited to sports preference reference speed limit, energy saving preference reference speed limit, and comfort preference reference speed limit, which are not limited in the embodiment of the present application.

[0053] 103. Perform speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information, and the vehicle path information to obtain vehicle speed limit information.

[0054] The current execution subject determines the multi-modal speed limit information, and optimizes and solves a reference speed optimizer based on the multi-modal speed limit information, vehicle navigation information, user preference information and vehicle path information as input parameters of the reference speed optimizer to obtain speed limit information of different vehicle positions, i.e., vehicle speed limit information. During the optimization and solving process, a function relationship between the vehicle speed, the vehicle acceleration, the vehicle position and the optimization control variable can be selected as the reference speed optimizer to perform polynomial solving in the mathematical field based on the function relationship to obtain the final vehicle speed limit information, thereby reducing the long tail problem and solving the problem that it is difficult to describe the speed limit in language.

[0055] In another embodiment of the present application, the step of performing speed limit optimization based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information includes:

[0056] Optimizing and solving based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information as input parameters of the reference speed optimizer to obtain the vehicle speed limit information.

[0057] In order to make the determination of the speed limit more in line with the user's preference, thereby realizing the flexible speed limit demand of different users to meet the user's self-defined demand and solving the flexibility of speed limit generation based on multi-modal input, in the embodiment of the present application, when performing speed limit optimization, the current execution subject constructs a reference speed optimizer to perform speed limit optimization and solving based on the reference speed optimizer. The reference speed optimizer is constructed based on the function relationship between the vehicle speed, the vehicle acceleration, the vehicle position and the optimization control variable, and can be constructed based on the relationship between the speed and the distance. Specifically, the speed limit distance interval adopted in the embodiment of the present application is represented as s total =max(v ego *10,60), s total is the distance corresponding to the real-time speed, v ego is the real-time speed, and then s total is divided into N equal parts, v i is the speed limit value in the multi-modal speed limit information at the s i position, a i is the acceleration at the s i position, and the expression of the reference speed optimizer is formula 1:

[0058] Wherein, u i is the optimization control variable, at this time, the optimization control variable is constrained based on the user preference information, i.e., at least one of the motion preference information, the power saving preference information and the comfort preference information can be constrained, thereby optimizing and solving the reference speed optimizer to obtain the vehicle speed limit information.

[0059] In another embodiment of the present application, the step is based on the multi-modal speed limit information, the vehicle navigation information, the user preference information, and the vehicle path information as input parameters of a reference speed optimizer to obtain the vehicle speed limit information, including:

[0060] determining a preference weight corresponding to the user preference information, and determining a solving coefficient of the optimization control variable based on the preference weight;

[0061] numerically solving the reference speed optimizer based on the solving coefficient to generate vehicle speed limit information.

[0062] In order to effectively solve the reference speed optimizer, thereby serving as the final vehicle speed limit value, and to improve the control accuracy of the vehicle speed limit, the current execution subject determines the preference weight of the user preference information when optimizing and solving the reference speed optimizer, and specifically, first determines the preference weight of the user preference information to determine the solving coefficient. Since the user preference information includes at least one of the motion preference information, the power saving preference information, and the comfort preference information, the corresponding preference weight corresponds to the motion preference weight, the power saving preference weight, and the comfort preference weight, so as to determine the solving coefficient of the optimization control variable based on the preference weight, and to achieve the purpose of giving the user personalized selection on the basis of a large model.

[0063] Specifically, the constraint expression formula 2 of the optimization control variable is:

[0064] wherein, the motion cost cost is represented as: J eff_i = λ eff (v i -v set_i ) 2 , v set_i is the expected maximum speed limit value. The power saving cost cost is represented as: J energy_i = λ energy (F d_i ), F d_i is the driving force, F d_i = f(a i , v i , slope i ), a i is the acceleration, v i is the vehicle speed, and slope i is the slope. The acceleration cost cost is represented as:

[0065] λ acc , λ brake are used to control the acceleration not to exceed the upper limit of the acceleration a upper_i and the lower limit of the deceleration a lower_iThe control cost is represented as: λ eff λ is a motion preference weight, energy λ is a power saving preference weight, a λ is a comfort preference weight, at this time, the solving coefficient includes the motion cost, the power saving cost, and the acceleration cost, so as to perform optimal solving of the control variable u, to obtain the vehicle speed limit information. In addition, the motion cost is used to represent the case that the vehicle meets different motion states, the power saving cost is used to represent the case that the vehicle meets different power saving states, and the acceleration cost is used to represent the case that the vehicle meets different comfort states. Furthermore, the predetermined control coefficient λ control The optimization control variable is combined with the motion cost, the power saving cost, and the acceleration cost as the control cost for representing the optimization control case of the vehicle, to obtain the constraint expression (Formula 1), at this time, the control coefficient λ control The control coefficient λ can be configured based on the optimization solving requirement, and the embodiments of the present application are not limited in this regard.

[0066] It should be noted that the motion preference weight matched with different motion preference information, the power saving preference weight matched with different power saving preference information, and the comfort preference weight matched with different comfort preference information can be pre-configured in the current execution subject, so that the user can directly call the motion preference weight, the power saving preference weight, and the comfort preference weight after inputting the motion preference information, the power saving preference information, and the comfort preference information in the front-end interface, and the embodiments of the present application are not limited in this regard.

[0067] In another embodiment of the present application, the step further includes:

[0068] obtaining historical speed limit information and historical vehicle speed information;

[0069] generating a speed-distance comparison curve diagram based on the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information, and displaying the speed-distance comparison curve diagram.

[0070] In order to realize the visual effect of the speed limit information for the user to make flexible selection, in the embodiments of the present application, when the vehicle speed limit information is solved, the current execution subject can output the vehicle speed limit information in the front-end interface of the target vehicle, and increase the display device of the vehicle speed and distance to give the user stronger confidence for the user to view. At this time, the front-end interface of the vehicle can be the front-end interface of the vehicle-mounted device, or the front-end interface of the user terminal, so that the user can view or select whether to control the vehicle speed according to the vehicle speed limit information, and the embodiments of the present application are not limited in this regard.

[0071] Specifically, when the current execution subject obtains the historical speed limit information and the historical vehicle speed information, the global speed limit is guaranteed for a long distance in the future, and the stability of the speed limit is greatly improved through the visual relationship between distance and speed. At this time, the historical speed limit information is the maximum or minimum limit speed in the predetermined historical time period, which can be scheduled based on a cloud storage server or a terminal memory, and the embodiments of the application are not limited specifically. In addition, the optimal speed limit obtained by solving can be represented as a relationship curve between distance and speed, and then the relationship curve is converted into a relationship curve between speed / acceleration and time, represented as (s i ,v i ,a i ) i＝0,1...N . Furthermore, the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information are generated into a speed-distance comparison curve diagram for display. As shown in Figure 3 , the corresponding historical speed limit, historical vehicle speed, and optimal vehicle speed limit obtained by solving are displayed in different visual curves to meet the user's visual effect according to the user's comfort preference.

[0072] In another embodiment of the application, the steps further include:

[0073] Controlling the speed of the target vehicle based on the vehicle speed limit information.

[0074] In a specific application scenario of the application, when the vehicle speed limit information is obtained, the current execution subject controls the driving speed of the vehicle during automatic driving based on the vehicle speed limit information. For example, the real-time speed of the target vehicle is obtained, and based on the acceleration, it is calculated that after a predetermined time, the predicted speed will be greater than the maximum limit value in the vehicle speed limit information, indicating that the acceleration is too fast, which will cause large power consumption. Therefore, the current execution subject can adjust the acceleration of the vehicle to slow down to save power, and the embodiments of the application are not limited specifically.

[0075] In another specific application scenario of the application, when the vehicle speed limit information is obtained, the current execution subject controls the deceleration of the vehicle during automatic driving to avoid obstacles based on the vehicle speed limit information. For example, when an obstacle is detected in front of the vehicle, the obstacle avoidance speed is calculated, and if the obstacle avoidance speed is greater than the maximum limit value in the vehicle speed limit information, it indicates that the deceleration is too fast, which will generate large inertia and poor comfort. Therefore, the current execution subject can adjust the deceleration point position of the vehicle to prolong the braking distance, thereby improving the comfort of vehicle deceleration, and the embodiments of the application are not limited specifically.

[0076] In another specific implementation scenario of the present application, when the vehicle speed limit information is obtained, the current execution subject controls different motion states of the vehicle in the automatic driving process of the vehicle based on the vehicle speed limit information. For example, the target vehicle is a steady motion type automatic driving style. After the expected average speed of the vehicle is collected, if the average speed is greater than the maximum speed limit in the vehicle speed limit information, the expected average speed is reduced to keep the constant motion style of the vehicle during driving and meet different driving experience requirements.

[0077] In another embodiment of the present application, the steps further comprise:

[0078] The original image collected by the vehicle camera is fused by the neural network to obtain a camera bird's eye view, and the navigation broadcast information, navigation map information, vehicle navigation information and user preference information are collected based on the vehicle communication device.

[0079] In order to realize the speed classification purpose of multi-modal, the current execution subject generates a camera bird's eye view based on the image collected by each vehicle-side camera in real time, that is, the frame image data of the vehicle at different time points is included, and the frame image data includes the driving position of the vehicle at different time points. At this time, the vehicle-side camera can be a camera installed on the vehicle, or a fixed position camera for shooting the vehicle, and the present application embodiment is not limited. In addition, since the current execution subject is a terminal for optimizing the speed limit processing, it can be a cloud controller or a terminal server, and the navigation broadcast information, navigation map information and vehicle navigation information are generated by the navigation system. The navigation map information and navigation broadcast information of the predetermined destination in the cloud navigation system can be collected through the vehicle communication device, and the present application embodiment is not limited. In a specific implementation scenario, the user can input the preference information selected for automatic driving control of the vehicle based on the vehicle-mounted device, so as to feed back to the current execution subject through the vehicle communication device, and the present application embodiment is not limited.

[0080] In one embodiment of the present application, as Figure 4As shown, after the multi-frame images of the target vehicle are collected based on the vehicle-end camera Multi-Camera, image processing is first performed by the Transformer model to obtain camera bird's-eye view, i.e., BEV-(1)...BEV-(i), and the multi-modal speed limit information is determined by the multi-modal end-to-end model Reference Speed (reference speed) Transformer combined with the navigation broadcast information and the navigation map (sd map) information after the model training. At this time, at least three multi-modal speed limits can be obtained, and the speed and distance curve is displayed based on the vehicle speed and distance curve. At the same time, the driving information (such as the navigation map information, road curvature, slope, etc. of the target vehicle) of the target vehicle is collected in real time through the 5G communication / GPS module, the user preference information obtained through the personalized configuration, and the vehicle navigation information (such as the cloud navigation information), and the multi-modal speed limit information are input into the reference speed optimizer Reference Speed Optimizer for optimization and solution, and the final speed limit of the target vehicle is finally obtained. The speed curve corresponding to the prior information of the empirical speed limit, the speed curve selected by the user preference mode, and the final speed limit solved are displayed in the front-end interface based on the distance and speed curve, so that the user can view. Further, if the user selects the speed limit based on the preference and inputs it into the motion planning and control system Motion Planning&Control for automatic driving planning, the embodiment of the application is not limited.

[0081] The embodiment of the application provides a vehicle speed limit generation method, which takes user preference as a speed limit determination basis, and optimizes and solves the speed limit of the user preference combined with path information and navigation information through a multi-modal end-to-end model, greatly improves the speed limit generation accuracy in different scenes, meets the personalized speed limit demand of different users, and reduces the long-tail problem of determining the speed limit through the multi-modal end-to-end model, so that the global speed limit of the vehicle in the automatic driving process is more consistent.

[0082] Further, as an implementation of the method described above Figure 1 The embodiment of the application provides a vehicle speed limit generation device, as shown in the device includes: Figure 5

[0083] The acquisition module 31 is configured to acquire vehicle navigation information, user preference information, vehicle path information, and driving information of a target vehicle, wherein the driving information includes navigation map information, camera bird's-eye view, and navigation broadcast information.

[0084] ​The first processing module 32 is configured to perform classification processing on the driving information based on a reference speed multi-modal end-to-end model to obtain multi-modal speed limit information, wherein the multi-modal speed limit information includes different speed limit classification results, and the reference speed multi-modal end-to-end model is obtained by the following method: obtaining driving training samples of different vehicles, performing model training on a deep learning model based on the driving training samples to obtain the reference speed multi-modal end-to-end model, wherein the driving training samples include navigation map samples, camera bird's eye view samples and navigation broadcast samples of different vehicles with labeled speed limit classification, and the deep learning model is constructed based on a multi-modal neural network.

[0085] The second processing module 33 is configured to perform speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information to obtain vehicle speed limit information.

[0086] Further, the second processing module is specifically configured to perform optimization solving based on the multi-modal speed limit information, the vehicle navigation information, the user preference information and the vehicle path information as input parameters of a reference speed optimizer to obtain the vehicle speed limit information.

[0087] The reference speed optimizer is constructed based on a function relationship between vehicle speed, vehicle acceleration, vehicle position and optimization control variables, and the optimization control variables are constrained based on the user preference information.

[0088] Further, the processing module is specifically configured to determine a preference weight corresponding to the user preference information, and determine a solving coefficient of the optimization control variable based on the preference weight, wherein the user preference information includes at least one of motion preference information, power saving preference information and comfort preference information; and perform numerical solving on the reference speed optimizer based on the solving coefficient to generate vehicle speed limit information.

[0089] Further, the device further comprises:

[0090] The output module is specifically configured to obtain historical speed limit information and historical vehicle speed information, generate a speed-distance comparison curve graph based on the historical speed limit information, the historical vehicle speed information and the vehicle speed limit information, and display the speed-distance comparison curve graph.

[0091] Further, the device further comprises:

[0092] The acquisition module is used to perform neural network fusion on the raw images acquired by the vehicle camera to obtain a bird's-eye view of the camera, and to acquire navigation broadcast information, navigation map information, vehicle navigation information and user preference information based on the vehicle communication equipment.

[0093] This application proposes a vehicle speed limit generation device. By using user preferences as the basis for speed limit determination and combining a multimodal end-to-end model to optimize the speed limit solution by combining user preferences with path information and navigation information, the device greatly improves the accuracy of speed limit generation for different scenarios, meets the personalized speed limit needs of different users, and reduces the long-tail problem of speed limit determination through the multimodal end-to-end model, making the global speed limit of the vehicle more consistent during autonomous driving.

[0094] According to one embodiment of this application, a vehicle is provided, including the aforementioned vehicle speed limit generation device.

[0095] According to one embodiment of this application, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the vehicle speed limit generation method described above.

[0096] Figure 6 A schematic diagram of a computer device according to an embodiment of this application is shown, including at least one processor coupled to a memory. The memory stores a program or instructions that run on the processor, which, when executed by the processor, implement the steps of the vehicle speed limit generation method described above. The specific embodiments of this application do not limit the specific implementation of the computer device.

[0097] like Figure 6 As shown, the computer device may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.

[0098] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0099] Communication interface 404 is used to communicate with other network elements such as clients or other servers.

[0100] The processor 402 is used to execute program 410, specifically to execute the relevant steps in the above-described vehicle speed limit generation method embodiment.

[0101] Specifically, program 410 may include program code that includes computer operation instructions.

[0102] The processor 402 can be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to perform the embodiments of the present application. The computer device can include one or more processors, which can be the same type of processor, such as one or more CPUs, or different types of processors, such as one or more CPUs and one or more ASICs.

[0103] The memory 406 is used to store programs 410. The memory 406 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0104] The programs 410 can specifically be used to cause the processor 402 to perform the following operations:

[0105] Obtain vehicle navigation information of a target vehicle, user preference information, vehicle path information, and travel information, the travel information including navigation map information, camera bird's eye view, and navigation broadcast information;

[0106] Classify and process the travel information based on a reference speed multi-modal end-to-end model to obtain multi-modal speed limit information, which includes different speed limit classification results, wherein the reference speed multi-modal end-to-end model is obtained by the following method: obtaining travel training samples of different vehicles, performing model training on a deep learning model based on the travel training samples to obtain a reference speed multi-modal end-to-end model, the travel training samples including navigation map samples, camera bird's eye view samples, and navigation broadcast samples of different vehicles labeled with speed limit classification, and the deep learning model is constructed based on a multi-modal neural network;

[0107] Perform speed limit optimization processing based on the multi-modal speed limit information, the vehicle navigation information, the user preference information, and the vehicle path information to obtain vehicle speed limit information, the multi-modal speed limit information being obtained by processing the travel information of the vehicle based on a multi-modal end-to-end model.

[0108] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with a general purpose computing device, which can be centralized on a single computing device or distributed on a network of multiple computing devices, and optionally implemented with program codes executable by a computing device, which can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be executed in an order different from that shown, or made into individual integrated circuit modules, or made into a single integrated circuit module. Thus, the present application is not limited to any particular combination of hardware and software.

[0109] The preferred embodiments of the present application described above are only used to explain the technical solutions of the present application and not to limit the present application. Any modifications, equivalent replacements, improvements, and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating vehicle speed limits, characterized in that, include: The system acquires the target vehicle's navigation information, user preference information, vehicle route information, and driving information, including navigation map information, camera bird's-eye view, and navigation broadcast information. The driving information is classified based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: driving training samples of different vehicles are obtained, and a deep learning model is trained based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples, camera bird's-eye view samples, and navigation broadcast samples with speed limit classification for different vehicles. The deep learning model is constructed based on a multimodal neural network. Based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information, speed limit optimization processing is performed to obtain vehicle speed limit information.

2. The method according to claim 1, characterized in that, The speed limit optimization process based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information includes: The vehicle speed limit information is obtained by optimizing the input parameters of the reference speed optimizer based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information. The reference speed optimizer is constructed based on the functional relationship between vehicle speed, vehicle acceleration, vehicle position, and optimization control variables, and the optimization control variables are constrained based on the user preference information.

3. The method according to claim 2, characterized in that, The process of optimizing the vehicle speed limit information based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information as input parameters for the reference speed optimizer, to obtain the vehicle speed limit information includes: Determine the preference weights corresponding to the user preference information, and determine the solution coefficients of the optimization control variables based on the preference weights. The user preference information includes at least one of exercise preference information, energy saving preference information, and comfort preference information. The reference speed optimizer is numerically solved based on the solved coefficients to generate vehicle speed limit information.

4. The method according to claim 3, characterized in that, After obtaining the vehicle speed limit information, the method further includes: Obtain historical speed limit information and historical vehicle speed information; Based on the historical speed limit information, the historical vehicle speed information, and the vehicle speed limit information, a speed-distance comparison curve is generated and displayed.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The original images captured by the vehicle camera are fused using a neural network to obtain a bird's-eye view of the camera, and navigation broadcast information, navigation map information, vehicle navigation information and user preference information are collected based on the vehicle communication equipment.

6. A vehicle speed limit generation device, characterized in that, include: The acquisition module is used to acquire the target vehicle's navigation information, user preference information, vehicle route information, and driving information. The driving information includes navigation map information, camera bird's-eye view, and navigation broadcast information. The first processing module is used to classify the driving information based on a reference speed multimodal end-to-end model to obtain multimodal speed limit information. The multimodal speed limit information includes different speed limit classification results. The reference speed multimodal end-to-end model is trained in the following way: obtaining driving training samples of different vehicles, and training a deep learning model based on the driving training samples to obtain the reference speed multimodal end-to-end model. The driving training samples include navigation map samples with marked speed limit classifications for different vehicles, camera bird's-eye view samples, and navigation broadcast samples. The deep learning model is constructed based on a multimodal neural network. The second processing module is used to perform speed limit optimization processing based on the multimodal speed limit information, the vehicle navigation information, the user preference information, and the vehicle route information to obtain vehicle speed limit information. The multimodal speed limit information is obtained by processing the vehicle's driving information based on a multimodal end-to-end model.

7. A vehicle, characterized in that, Includes the vehicle speed limit generation device as described in claim 6.

8. A computer device, characterized in that, It includes at least one processor coupled to a memory, the memory storing a program or instructions that run on the processor, the program or instructions, when executed by the processor, implementing the steps of the method for generating a vehicle speed limit as claimed in any one of claims 1 to 5.

9. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the method for generating vehicle speed limits as described in any one of claims 1 to 5.

Images