Remote driving assistance method, device, equipment, storage medium and program product
By displaying the vehicle's surrounding environment information and predicting safety boundaries on the remote driving end, the problem of inaccurate driving safety perception during remote driving is solved, and the risk of remote driving errors is reduced.
Patent Information
- Application Number
- CN202410804837.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-21
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-06-21
AI Technical Summary
In the remote driving scenario, the driver is not aware of the driving safety of the remote driving vehicle, which leads to errors in remote control commands, which increases the probability of remote driving errors, especially when the vehicle is moving at high speed.
In the display screen of the remote driving end, the vehicle's surrounding environment information is displayed and the predicted vehicle safety boundary line is superimposed. By obtaining the vehicle's surrounding environment information and predicting safety boundary point sequence information, the vehicle's safety boundary line and trajectory are drawn to remind the driver to remind the driver Potential risk of rollover.
It improves the driver's intuitive perception of the driving safety of remote driving vehicles, reduces the probability of remote driving errors, and helps the driver to perform remote control more accurately through auxiliary support.
Smart Images

Figure CN118387127B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of vehicle remote driving control, and in particular to remote driving assistance methods, devices, equipment, storage media and program products. Background Art
[0002] In the remote driving scenario, the driver remotely controls the vehicle at the remote driving center. In order to ensure accurate perception of the vehicle's surrounding environment, it is necessary to install a variety of sensors on the vehicle, such as high-definition cameras, infrared sensors, etc. These sensors can effectively capture images, infrared information and other information around the vehicle. By processing and analyzing this information and transmitting it to the remote driving center, it can help the remote driver perceive the road conditions, such as traffic signs, road vehicles and pedestrians, and remotely control the vehicle based on the perceived road conditions.
[0003] However, when remote control is performed in the above manner, there is a problem of a high probability of remote driving errors, especially when the vehicle is moving at high speed. Summary of the invention
[0004] The embodiments of the present application provide a remote driving assistance method, apparatus, device, storage medium and program product, so as to enable the driver at the remote driving center to intuitively observe the driving safety of the remotely driven vehicle and reduce the probability of remote driving errors.
[0005] In a first aspect, an embodiment of the present application provides a remote driving assistance method, which is applied to a remote driving end. The remote driving assistance method includes:
[0006] Acquiring surrounding environment information and predicted safety boundary point sequence information of a remotely driven vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information within a future time period predicted based on current driving data of the vehicle;
[0007] On the display screen of the remote driving end, a surrounding environment picture corresponding to the surrounding environment information is displayed, and a predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0008] In a possible implementation, the predicted vehicle safety boundary line is a predicted vehicle moving boundary line at which the vehicle does not roll over when turning, the predicted vehicle safety boundary line includes a first vehicle safety boundary line and a second vehicle safety boundary line, the first vehicle safety boundary line is a moving boundary line of the left body line of the vehicle, and the second vehicle safety boundary line is a moving boundary line of the right body line of the vehicle, and the superimposing and displaying the predicted vehicle safety boundary line on the surrounding environment screen includes:
[0009] At least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line is drawn on the surrounding environment picture to overlay and display the predicted vehicle safety boundary line on the surrounding environment picture.
[0010] In a possible implementation manner, the vehicle safety boundary point information includes first vehicle safety boundary point information and second vehicle safety boundary point information, the surrounding environment picture includes an environment picture in front of the vehicle, and drawing at least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line on the surrounding environment picture includes:
[0011] Based on the first vehicle safety boundary point information, at least a portion of the first vehicle safety boundary line is drawn on the environment picture in front of the vehicle, and / or, based on the second vehicle safety boundary point information, at least a portion of the second vehicle safety boundary line is drawn on the environment picture in front of the vehicle.
[0012] In a possible implementation, the remote driving assistance method further includes:
[0013] Acquire predicted trajectory point sequence information of the vehicle, wherein the predicted trajectory point sequence information includes driving trajectory point information within a future time period predicted based on current driving data of the vehicle;
[0014] The predicted driving trajectory is superimposed and displayed on the surrounding environment picture, and the predicted driving trajectory is drawn based on the predicted trajectory point sequence information.
[0015] In a possible implementation, the remote driving assistance method further includes:
[0016] Acquiring a real-time moving radius of the vehicle, and acquiring a critical curvature radius corresponding to current speed information of the vehicle;
[0017] Calculating the difference between the real-time motion radius and the critical curvature radius;
[0018] If the difference is less than a preset difference threshold, a reminder message is generated, and the reminder message is used to remind the user that the vehicle is at risk of rollover.
[0019] In a second aspect, an embodiment of the present application provides another remote driving assistance method, which is applied to a vehicle side. The remote driving assistance method includes:
[0020] Obtain the vehicle's surrounding environment information and current driving data;
[0021] Based on the current driving data, predict the driving safety boundary line of the vehicle to obtain predicted safety boundary point sequence information of the vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information of the vehicle within a future time period;
[0022] The surrounding environment information and the predicted safety boundary point sequence information are sent to a remote driving terminal, so that when the remote driving terminal displays a surrounding environment picture corresponding to the surrounding environment information, a predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0023] In a possible implementation, the current driving data includes current speed information, the vehicle stores a mapping relationship between a pre-calibrated speed and a critical curvature radius, and the predicting of the driving safety boundary line of the vehicle based on the current driving data to obtain the predicted safety boundary point sequence information of the vehicle includes:
[0024] Querying a mapping relationship between the speed and the critical curvature radius to obtain a critical curvature radius corresponding to the speed information, wherein the critical curvature radius is a minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information;
[0025] Based on the critical curvature radius, determining a reference point moving path of the vehicle; wherein the reference point of the vehicle is a rear axle center point of the vehicle;
[0026] In the vehicle motion simulation system, the vehicle is simulated to move along the reference point moving path, and the position information of the left and right corner points of the vehicle are collected at each position point to which the vehicle moves;
[0027] Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained.
[0028] In a possible implementation manner, determining the reference point moving path of the vehicle based on the critical curvature radius includes:
[0029] Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period;
[0030] The motion trajectory of the vehicle in circular motion within the preset time period is determined based on the center and radius of the circular motion of the vehicle within the preset time period and the current motion direction information of the vehicle, and the motion trajectory is determined as the reference point moving path of the vehicle.
[0031] In a possible implementation manner, before predicting the driving safety boundary line of the vehicle based on the current driving data to obtain the predicted safety boundary point sequence information of the vehicle, the remote driving assistance method further includes:
[0032] It is determined that a speed value corresponding to the current speed information of the vehicle is greater than a preset speed threshold.
[0033] In a third aspect, an embodiment of the present application provides a remote driving assistance device, which is applied to a remote driving end, and the remote driving assistance device includes:
[0034] A first acquisition module is used to acquire surrounding environment information and predicted safety boundary point sequence information of a remotely driven vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information within a future time period predicted based on current driving data of the vehicle;
[0035] A display module is used to display a surrounding environment picture corresponding to the surrounding environment information on a display screen of the remote driving terminal, and to superimpose a predicted vehicle safety boundary line on the surrounding environment picture, wherein the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0036] In a possible implementation, the predicted vehicle safety boundary line is a predicted vehicle moving boundary line at which the vehicle does not roll over when turning, the predicted vehicle safety boundary line includes a first vehicle safety boundary line and a second vehicle safety boundary line, the first vehicle safety boundary line is a moving boundary line of the left body line of the vehicle, and the second vehicle safety boundary line is a moving boundary line of the right body line of the vehicle, and the display module is specifically used to:
[0037] At least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line is drawn on the surrounding environment picture to overlay and display the predicted vehicle safety boundary line on the surrounding environment picture.
[0038] In a possible implementation manner, the vehicle safety boundary point information includes first vehicle safety boundary point information and second vehicle safety boundary point information, the surrounding environment picture includes a picture of the environment in front of the vehicle, and the display module is specifically used for:
[0039] Based on the first vehicle safety boundary point information, at least a portion of the first vehicle safety boundary line is drawn on the environment picture in front of the vehicle, and / or, based on the second vehicle safety boundary point information, at least a portion of the second vehicle safety boundary line is drawn on the environment picture in front of the vehicle.
[0040] In a possible implementation manner, the remote driving assistance device is further used for:
[0041] Acquire predicted trajectory point sequence information of the vehicle, wherein the predicted trajectory point sequence information includes driving trajectory point information within a future time period predicted based on current driving data of the vehicle;
[0042] The predicted driving trajectory is superimposed and displayed on the surrounding environment picture, and the predicted driving trajectory is drawn based on the predicted trajectory point sequence information.
[0043] In a possible implementation manner, the remote driving assistance device is further used for:
[0044] Acquiring a real-time moving radius of the vehicle, and acquiring a critical curvature radius corresponding to current speed information of the vehicle;
[0045] Calculating the difference between the real-time motion radius and the critical curvature radius;
[0046] If the difference is less than a preset difference threshold, a reminder message is generated, and the reminder message is used to remind the user that the vehicle is at risk of rollover.
[0047] In a fourth aspect, an embodiment of the present application provides another remote driving assistance device, which is applied to a vehicle side, and the remote driving assistance device includes:
[0048] A first acquisition module, used to acquire the surrounding environment information and current driving data of the vehicle;
[0049] A prediction module, configured to predict the driving safety boundary line of the vehicle based on the current driving data, and obtain predicted safety boundary point sequence information of the vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information of the vehicle within a future time period;
[0050] A sending module is used to send the surrounding environment information and the predicted safety boundary point sequence information to a remote driving terminal, so that when the remote driving terminal displays the surrounding environment picture corresponding to the surrounding environment information, a predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0051] In a possible implementation manner, the current driving data includes current speed information, the vehicle stores a pre-calibrated mapping relationship between speed and critical curvature radius, and the prediction module is specifically used to:
[0052] Querying a mapping relationship between the speed and the critical curvature radius to obtain a critical curvature radius corresponding to the speed information, wherein the critical curvature radius is a minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information;
[0053] Based on the critical curvature radius, determining a reference point moving path of the vehicle; wherein the reference point of the vehicle is a rear axle center point of the vehicle;
[0054] In the vehicle motion simulation system, the vehicle is simulated to move along the reference point moving path, and the position information of the left and right corner points of the vehicle are collected at each position point to which the vehicle moves;
[0055] Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained.
[0056] In a possible implementation, the prediction module is specifically used to:
[0057] Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period;
[0058] The motion trajectory of the vehicle in circular motion within the preset time period is determined based on the center and radius of the circular motion of the vehicle within the preset time period and the current motion direction information of the vehicle, and the motion trajectory is determined as the reference point moving path of the vehicle.
[0059] In a possible implementation manner, the remote driving assistance device is further used for:
[0060] It is determined that a speed value corresponding to the current speed information of the vehicle is greater than a preset speed threshold.
[0061] In a fifth aspect, an embodiment of the present application provides a remote driving assistance device, including: a memory, a processor;
[0062] The memory stores computer-executable instructions;
[0063] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.
[0064] In a sixth aspect, an embodiment of the present application provides a vehicle, including a memory and a processor;
[0065] The memory stores computer-executable instructions;
[0066] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above second aspect and / or various possible implementations of the second aspect.
[0067] In a seventh aspect, an embodiment of the present application provides a remote driving assistance system, including a vehicle and a remote driving assistance device;
[0068] The vehicle is used to obtain the surrounding environment information and current driving data of the vehicle; based on the current driving data, predict the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle, wherein the predicted safety boundary point sequence information includes the vehicle safety boundary point information of the vehicle within a future time period; and send the surrounding environment information and the predicted safety boundary point sequence information to a remote driving terminal;
[0069] The remote driving assistance device is used to obtain the surrounding environment information and the predicted safety boundary point sequence information; on the display screen of the remote driving end, the surrounding environment picture corresponding to the surrounding environment information is displayed, and the predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0070] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-readable storage medium is stored computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the first aspect and / or various possible implementation methods of the first aspect as above, or to implement the second aspect and / or various possible implementation methods of the second aspect as above.
[0071] In a ninth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the first aspect above and / or various possible implementations of the first aspect, or implements the second aspect above and / or various possible implementations of the second aspect.
[0072] The remote driving assistance method, apparatus, device, storage medium and program product provided in the embodiments of the present application can provide auxiliary support for the driver's remote driving by superimposing the predicted vehicle safety boundary line of the vehicle while displaying the surrounding environment information of the remotely driven vehicle on the display screen of the remote driving end, so that the driver can more accurately perceive the current driving safety situation of the remotely driven vehicle. Through the above method, the driver at the remote driving center can intuitively observe the driving safety situation of the remotely driven vehicle, thereby reducing the probability of remote driving errors. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0074] Figure 1 A schematic diagram of a scenario of a remote driving assistance method provided in this application;
[0075] Figure 2A schematic diagram of a remote driving assistance method provided in this application Figure 1 ;
[0076] Figure 3 A schematic diagram of a remote driving assistance method provided in this application Figure 2 ;
[0077] Figure 4 A schematic diagram of the mapping relationship between vehicle speed and critical curvature radius provided in this application;
[0078] Figure 5 A schematic diagram of a remote driving assistance method provided in this application Figure 3 ;
[0079] Figure 6 A schematic diagram of a remote driving assistance method provided in this application Figure 4 ;
[0080] Figure 7 A schematic diagram of a model for determining the moving path of the vehicle reference point for this application;
[0081] Figure 8 A schematic diagram of a model for determining the predicted safety boundary point sequence information of a vehicle for this application;
[0082] Fig. 9 A schematic diagram of the structure of a remote driving assistance device provided in this application Figure 1 ;
[0083] Fig.10 A schematic diagram of the structure of a remote driving assistance device provided in this application Figure 2 ;
[0084] Fig.11 A schematic diagram of the structure of a remote driving assistance device provided in this application;
[0085] Fig.12 A schematic diagram of the structure of a vehicle provided for this application;
[0086] Fig.13 A schematic diagram of the structure of a remote driving assistance system provided in this application.
[0087] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0088] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0089] First, the terms involved in this application are explained:
[0090] Remote driving: The driver at the remote driving center transmits vehicle control information to the off-site vehicle through the network and remotely controls the off-site vehicle.
[0091] Remote Driving Center: includes a remote cockpit. In the remote cockpit, by controlling a set of driving hardware, including the same seats and steering wheels as the vehicle, the instructions are sent to the physical vehicle console in real time through network transmission to execute operations and achieve the effect of remote driving.
[0092] Figure 1 A scenario diagram of a remote driving assistance method provided in this application, such as Figure 1 As shown, the application scenario of the present application is specifically a scenario in which the driver remotely controls the vehicle in a remote driving center. When the vehicle is driving, the vehicle's surrounding environment information detected by the vehicle's sensors can be transmitted to the remote driving center through a specific transmission channel. The driver of the remote driving center can perceive the vehicle's current driving environment based on the acquired vehicle's surrounding environment information, and then issue corresponding remote control instructions.
[0093] In view of the above scenario, in the prior art, the remote driving center often receives audio and video stream data collected 200ms ago instead of real-time audio and video stream data. When the vehicle is moving at high speed, the driver in the remote driving center remotely controls the vehicle based on the delayed audio and video stream data, which may cause remote driving errors.
[0094] The present application provides a remote driving assistance method, in which, on the display screen of the remote driving end, while displaying the surrounding environment information of the remotely driven vehicle, the predicted vehicle safety boundary line of the vehicle is superimposed and displayed. The predicted vehicle safety boundary line is a predicted driving safety boundary line. If the vehicle approaches or exceeds the driving safety boundary line, it indicates that there is a safety risk in the driving of the vehicle. The predicted vehicle safety boundary line provides auxiliary support for the driver's remote driving, so that the driver can perceive the current driving safety situation of the remotely driven vehicle more accurately. The driver can comprehensively consider the surrounding environment information of the vehicle and the predicted vehicle safety boundary line to issue remote control instructions, which solves the technical problem in the prior art that the driver of the remote driving center issues erroneous remote control instructions due to inaccurately perceived driving safety situation of the remotely driven vehicle, resulting in accidents.
[0095] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0096] Figure 2 A schematic diagram of a remote driving assistance method provided in this application Figure 1 ,The remote driving assistance method can be applied to the remote driving end, such as Figure 2 As shown, the remote driving assistance method includes:
[0097] S201. Acquire surrounding environment information and predicted safety boundary point sequence information of a remotely driven vehicle, where the predicted safety boundary point sequence information includes vehicle safety boundary point information within a future time period predicted based on current driving data of the vehicle.
[0098] The surrounding environment information may be the environment information around the vehicle captured by the vehicle sensor.
[0099] In the present application, the remote driving end can obtain the surrounding environment information and predicted safety boundary point sequence information of the remotely driven vehicle. Specifically, the remote driving end can directly obtain the surrounding environment information and predicted safety boundary point sequence information of the vehicle from the vehicle end, or the vehicle end sends the collected surrounding environment information and predicted safety boundary point sequence information to a preset server, and the remote driving end obtains it from the preset server.
[0100] S202. Displaying a surrounding environment picture corresponding to the surrounding environment information on a display screen of the remote driving terminal, and superimposing a predicted vehicle safety boundary line on the surrounding environment picture, wherein the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0101] The predicted vehicle safety boundary line is a predicted safety boundary line for vehicle driving. When the vehicle drives close to or exceeds the safety boundary line, it can be determined that there is a safety risk in the vehicle driving.
[0102] In the present application, the display screen of the remote driving end not only displays the surrounding environment picture corresponding to the surrounding environment information of the remotely driven vehicle, but also identifies the predicted vehicle safety boundary line based on the surrounding environment picture, thereby providing auxiliary support for the driver's remote driving.
[0103] Furthermore, there may be multiple display styles for predicting the vehicle safety boundary line, and the driver at the remote driving end may choose from a variety of display styles, and the embodiments of the present application do not impose any specific restrictions on this.
[0104] The remote driving assistance method provided in the embodiment of the present application can provide auxiliary support for the driver's remote driving by superimposing the predicted vehicle safety boundary line of the vehicle while displaying the surrounding environment information of the remotely driven vehicle on the display screen of the remote driving end, so that the driver can more accurately perceive the current driving safety situation of the remotely driven vehicle. Through the above method, the driver at the remote driving center can intuitively observe the driving safety situation of the remotely driven vehicle, thereby reducing the probability of remote driving errors.
[0105] Figure 3 A schematic diagram of a remote driving assistance method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, a remote driving assistance method is described in detail. The remote driving assistance method can be applied to a remote driving end. The remote driving assistance method includes:
[0106] S301. Acquire surrounding environment information and predicted safety boundary point sequence information of a remotely driven vehicle, where the predicted safety boundary point sequence information includes vehicle safety boundary point information within a future time period predicted based on current driving data of the vehicle.
[0107] S302: Displaying a surrounding environment picture corresponding to the surrounding environment information on a display screen of the remote driving terminal.
[0108] The surrounding environment picture includes the environment picture in front of the vehicle, the environment picture on the left side of the vehicle and the environment picture on the right side of the vehicle.
[0109] In one example, the remote driving terminal may include at least three display screens, wherein the first display screen is used to display the environment in front of the vehicle, the second display screen is used to display the environment on the left side of the vehicle, and the third display screen is used to display the environment on the right side of the vehicle; in another example, the remote driving terminal may include only one display screen, which can provide at least three video screens, wherein the first video screen is used to display the environment in front of the vehicle, the second video screen is used to display the environment on the left side of the vehicle, and the third video screen is used to display the environment on the right side of the vehicle.
[0110] It should be noted that, when the vehicle is reversing, the environment picture in front of the vehicle refers to the environment picture behind the vehicle.
[0111] S303. Draw at least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line on the surrounding environment picture to overlay and display the predicted vehicle safety boundary line on the surrounding environment picture. The predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information. The predicted vehicle safety boundary line is a moving boundary line of the vehicle that is predicted not to roll over when the vehicle turns. The first vehicle safety boundary line is a moving boundary line of the left body line of the vehicle, and the second vehicle safety boundary line is a moving boundary line of the right body line of the vehicle.
[0112] The predicted vehicle safety boundary line includes a first vehicle safety boundary line and a second vehicle safety boundary line. Specifically, at least a portion of the first vehicle safety boundary line and / or at least a portion of the second vehicle safety boundary line can be drawn on the surrounding environment screen by using SVG (Scalable Vector Graphics) or Canvas (bitmap) drawing technology.
[0113] In a possible implementation, the vehicle safety boundary point information includes first vehicle safety boundary point information and second vehicle safety boundary point information, and drawing at least part of the first vehicle safety boundary line and / or the second vehicle safety boundary line on the surrounding environment screen may specifically include:
[0114] Based on the first vehicle safety boundary point information, at least a portion of the first vehicle safety boundary line is drawn on the environment picture in front of the vehicle, and / or, based on the second vehicle safety boundary point information, at least a portion of the second vehicle safety boundary line is drawn on the environment picture in front of the vehicle.
[0115] In this embodiment, by drawing a predicted vehicle safety boundary line on the environmental picture in front of the vehicle, the driver can intuitively perceive the current driving safety situation of the remote-driven vehicle.
[0116] In a possible implementation, the remote driving assistance method of the present application may further include:
[0117] Acquire predicted trajectory point sequence information of the vehicle, where the predicted trajectory point sequence information includes driving trajectory point information within a future time period predicted based on current driving data of the vehicle;
[0118] The predicted driving trajectory is superimposed on the surrounding environment picture, and the predicted driving trajectory is drawn based on the predicted trajectory point sequence information.
[0119] In the present application, a surrounding environment picture corresponding to the surrounding environment information of the remotely driven vehicle is displayed on the display screen of the remote driving end, and the predicted driving trajectory of the vehicle is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is superimposed on the surrounding environment picture. By observing whether the predicted driving trajectory of the vehicle is close to the predicted vehicle safety boundary line, the driver can intuitively observe the driving safety status of the remotely driven vehicle at the current moment, thereby assisting the driver in remote driving.
[0120] The predicted driving trajectory may include a predicted moving trajectory of a left body line of the vehicle and a predicted moving trajectory of a right body line of the vehicle, and may be drawn on a front environment screen of the vehicle.
[0121] In a possible implementation, the remote driving assistance method of the present application may further include:
[0122] Obtaining the real-time moving radius of the vehicle and the critical curvature radius corresponding to the current speed information of the vehicle;
[0123] Calculate the difference between the real-time motion radius and the critical curvature radius;
[0124] If the difference is less than the preset difference threshold, a reminder message is generated to remind the user that the vehicle is at risk of rollover.
[0125] The critical radius of curvature is the minimum radius of curvature at which the vehicle does not roll over when turning.
[0126] In this application, the mapping relationship between the vehicle speed and the critical curvature radius can be pre-calibrated. Figure 4 As shown, it is a schematic diagram of the mapping relationship between vehicle speed and critical curvature radius provided by the present application, wherein each vehicle speed value corresponds to a curvature radius value one by one.
[0127] In the present application, the current real-time movement radius of the vehicle and the critical curvature radius corresponding to the current speed information of the vehicle can be obtained, and the difference between the real-time movement radius and the critical curvature radius can be calculated. If the difference is less than the preset difference threshold, it can be determined that the vehicle is at risk of rollover and a corresponding reminder message can be generated. The reminder message can be a voice reminder message or a text reminder message.
[0128] For the real-time motion radius, the remote driving end can obtain the vehicle's current speed information and yaw angular velocity information from the vehicle end, and based on the vehicle's current speed information and yaw angular velocity information, the vehicle's current real-time motion radius can be calculated.
[0129] When the difference between the real-time motion radius and the critical curvature radius is less than the preset difference threshold, it means that the predicted driving trajectory of the vehicle is close to or coincides with the predicted vehicle safety boundary line. At this time, generating a reminder message can assist the driver in remotely driving the vehicle.
[0130] The remote driving assistance method provided in the embodiment of the present application can provide auxiliary support for the driver's remote driving by superimposing the predicted vehicle safety boundary line of the vehicle while displaying the surrounding environment information of the remotely driven vehicle on the display screen of the remote driving end, so that the driver can more accurately perceive the current driving safety situation of the remotely driven vehicle. Through the above method, the driver at the remote driving center can intuitively observe the driving safety situation of the remotely driven vehicle, thereby reducing the probability of remote driving errors.
[0131] The above-mentioned embodiment explains the technical solution of the present application from the perspective of the remote driving end. The following explains the technical solution of the present application from the perspective of the vehicle end.
[0132] Figure 5 A schematic diagram of a remote driving assistance method provided in this application Figure 3 ,This remote driving assistance method can be applied to the vehicle side, such as Figure 5 As shown, the remote driving assistance method includes:
[0133] S501, obtaining the surrounding environment information and current driving data of the vehicle.
[0134] A variety of environmental perception sensors can be installed on the vehicle, such as: cameras, lidar, millimeter-wave radar, infrared, ultrasonic radar, etc. These sensors can monitor the vehicle's surrounding environment, detect the vehicle's driving status, and obtain the vehicle's surrounding environment information and current driving data.
[0135] S502: Based on the current driving data, predict the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle, where the predicted safety boundary point sequence information includes the vehicle safety boundary point information of the vehicle in the future time period.
[0136] On the vehicle side, the vehicle's driving safety boundary line in the future is predicted based on the current driving data obtained, and the vehicle's predicted safety boundary point sequence information is obtained.
[0137] In a possible implementation, based on the current driving data, the driving safety boundary line of the vehicle is predicted, and before obtaining the predicted safety boundary point sequence information of the vehicle, the remote driving assistance method of the present application may further specifically include:
[0138] It is determined that the speed value corresponding to the current speed information of the vehicle is greater than a preset speed threshold.
[0139] In this embodiment, the driving safety boundary line of the vehicle can be predicted only when the current vehicle speed is greater than a preset speed threshold, which can effectively reduce the consumption of vehicle computing resources.
[0140] S503. Send surrounding environment information and predicted safety boundary point sequence information to the remote driving terminal, so that when the remote driving terminal displays the surrounding environment picture corresponding to the surrounding environment information, the predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0141] The remote driving assistance method provided in the embodiment of the present application can provide auxiliary support for the driver's remote driving by superimposing the predicted vehicle safety boundary line of the vehicle while displaying the surrounding environment information of the remotely driven vehicle on the display screen of the remote driving end, so that the driver can more accurately perceive the current driving safety situation of the remotely driven vehicle. Through the above method, the driver at the remote driving center can intuitively observe the driving safety situation of the remotely driven vehicle, thereby reducing the probability of remote driving errors.
[0142] Figure 6 A schematic diagram of a remote driving assistance method provided in this application Figure 4 ,like Figure 6 As shown, in this embodiment Figure 5 Based on the embodiment, a remote driving assistance method is described in detail. The remote driving assistance method can be applied to a vehicle side. The remote driving assistance method includes:
[0143] S601. Acquire surrounding environment information and current driving data of the vehicle, where the current driving data includes current speed information, and the vehicle stores a pre-calibrated mapping relationship between speed and critical curvature radius.
[0144] S602: Query the mapping relationship between the speed and the critical curvature radius to obtain the critical curvature radius corresponding to the speed information, where the critical curvature radius is the minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information.
[0145] S603. Determine a reference point moving path of the vehicle based on the critical curvature radius, wherein the reference point of the vehicle is a rear axle center point of the vehicle.
[0146] When the vehicle is moving, a section can be made in the direction of the vehicle's movement, and the center of the vehicle's rear axle can be used as the vehicle's reference point. The moving path of the reference point can be determined based on the critical curvature radius corresponding to the vehicle's current speed information.
[0147] In a possible implementation, determining the reference point moving path of the vehicle based on the critical curvature radius may specifically include:
[0148] Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period;
[0149] The motion trajectory of the vehicle in circular motion within the preset time period is determined based on the center and radius of the vehicle in circular motion within the preset time period and the current motion direction information of the vehicle, and the motion trajectory is determined as the reference point moving path of the vehicle.
[0150] In a short period of time, the vehicle can be considered to be moving in a circular motion. Figure 7 , which is a schematic diagram of a model for determining the moving path of the vehicle reference point in this application, such as Figure 7 As shown, a body coordinate system is established with the direction of vehicle movement as a section and the center of the rear axle of the vehicle as the reference point of the vehicle. By querying the mapping relationship between speed and critical curvature radius, the critical curvature radius R corresponding to the current speed information of the vehicle is obtained. According to the critical curvature radius R corresponding to the current speed information and the current movement direction information of the vehicle, the movement trajectory of the vehicle in a short period of time in the future can be predicted. Specifically, the center of the circular motion of the vehicle in a preset time period can be determined according to the critical curvature radius R corresponding to the current speed information, and the critical curvature radius is used as the radius of the circular motion of the vehicle in the preset time period. Based on the center and radius of the circular motion of the vehicle in the preset time period, combined with the current movement direction information of the vehicle (instantaneous movement direction information), the movement trajectory of the circular motion of the vehicle in the preset time period can be determined. As shown in the figure, the movement trajectory is a circular arc, and the circular arc is the moving path of the vehicle reference point.
[0151] The calculation process can be simplified by considering the vehicle's motion as a circular motion.
[0152] S604: In the vehicle motion simulation system, simulate the movement of the vehicle along the reference point movement path, and collect the position information of the left and right corner points of the vehicle at each position point where the vehicle moves.
[0153] S605. Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained. The predicted safety boundary point sequence information includes the vehicle safety boundary point information of the vehicle in the future time period.
[0154] Reference Figure 8, which is a schematic diagram of a model for determining the predicted safety boundary point sequence information of a vehicle in this application, such as Figure 8 As shown, the reference point of the vehicle moves along the reference point moving path obtained in the previous step, and the whole vehicle moves with the movement of the reference point. Specifically, the whole vehicle moves to a position point in a time period, and moves to the next position point in the next time period. At each position point to which the vehicle moves, the position information of the four corner points of the vehicle body is collected to obtain the predicted safety boundary point sequence information of the vehicle. Among them, the time unit of vehicle movement can be 50ms, and the movement duration (i.e., the preset time period mentioned above, in which the vehicle is regarded as moving in a circle) can be 2s, that is, data collection is performed every 50ms within 2s. The movement duration of the vehicle along the reference point moving path can be determined based on the current speed information of the vehicle and the delay information of the signal transmission between the remote driving end and the vehicle end.
[0155] S606. Sending surrounding environment information and predicted safety boundary point sequence information to the remote driving terminal, so that when the remote driving terminal displays the surrounding environment picture corresponding to the surrounding environment information, the predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0156] In a possible implementation, sending the surrounding environment information and the predicted safety boundary point sequence information to the remote driving terminal may specifically include:
[0157] The surrounding environment information and predicted safety boundary point sequence information are sent to the remote driving end through a real-time transmission channel.
[0158] In this embodiment, data transmission is performed through a real-time transmission channel to achieve low-latency and non-jamming data transmission.
[0159] After the remote driving end obtains the predicted safety boundary point sequence information, it uses the predicted safety boundary point sequence information to draw the predicted vehicle safety boundary line.
[0160] It should be noted that, in the above-mentioned embodiment, the driver is assisted in remote driving operations by superimposing the predicted vehicle safety boundary line on the surrounding environment picture. In addition, the surrounding environment picture with superimposed display of the predicted vehicle safety boundary line provided by the technical solution of the present application can also provide technical support for the automatic driving of the remote driving end. That is, the technical solution of the present application can be applied not only to the scenario where the driver remotely controls the vehicle at the remote driving end, but also to the scenario where the vehicle is remotely controlled by the remote driving end using automatic driving technology.
[0161] The remote driving assistance method provided in the embodiment of the present application can provide auxiliary support for the driver's remote driving by superimposing the predicted vehicle safety boundary line of the vehicle while displaying the surrounding environment information of the remotely driven vehicle on the display screen of the remote driving end, so that the driver can more accurately perceive the current driving safety situation of the remotely driven vehicle. Through the above method, the driver at the remote driving center can intuitively observe the driving safety situation of the remotely driven vehicle, thereby reducing the probability of remote driving errors.
[0162] Fig. 9 A schematic diagram of the structure of a remote driving assistance device provided in this application Figure 1 , applied to remote driving end, such as Fig. 9 As shown, the remote driving assistance device 90 provided in this embodiment includes:
[0163] A first acquisition module 901 is used to acquire the surrounding environment information of the remotely driven vehicle and the predicted safety boundary point sequence information, wherein the predicted safety boundary point sequence information includes the vehicle safety boundary point information within a future time period predicted based on the current driving data of the vehicle;
[0164] The display module 902 is used to display the surrounding environment picture corresponding to the surrounding environment information on the display screen of the remote driving end, and to superimpose the predicted vehicle safety boundary line on the surrounding environment picture. The predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0165] In a possible implementation, the predicted vehicle safety boundary line is a predicted vehicle moving boundary line at which the vehicle does not roll over when turning, and the predicted vehicle safety boundary line includes a first vehicle safety boundary line and a second vehicle safety boundary line, the first vehicle safety boundary line is a moving boundary line of the left body line of the vehicle, and the second vehicle safety boundary line is a moving boundary line of the right body line of the vehicle, and the display module is specifically used for:
[0166] At least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line is drawn on the surrounding environment picture to overlay and display the predicted vehicle safety boundary line on the surrounding environment picture.
[0167] In a possible implementation, the vehicle safety boundary point information includes first vehicle safety boundary point information and second vehicle safety boundary point information, the surrounding environment picture includes the environment picture in front of the vehicle, and the display module is specifically used for:
[0168] Based on the first vehicle safety boundary point information, at least a portion of the first vehicle safety boundary line is drawn on the environment picture in front of the vehicle, and / or, based on the second vehicle safety boundary point information, at least a portion of the second vehicle safety boundary line is drawn on the environment picture in front of the vehicle.
[0169] In a possible implementation, the remote driving assistance device is further used to:
[0170] Acquire predicted trajectory point sequence information of the vehicle, where the predicted trajectory point sequence information includes driving trajectory point information within a future time period predicted based on current driving data of the vehicle;
[0171] The predicted driving trajectory is superimposed on the surrounding environment picture, and the predicted driving trajectory is drawn based on the predicted trajectory point sequence information.
[0172] In a possible implementation, the remote driving assistance device is further used to:
[0173] Obtaining the real-time moving radius of the vehicle and the critical curvature radius corresponding to the current speed information of the vehicle;
[0174] Calculate the difference between the real-time motion radius and the critical curvature radius;
[0175] If the difference is less than the preset difference threshold, a reminder message is generated to remind the user that the vehicle is at risk of rollover.
[0176] A remote driving assistance device provided in this embodiment can execute the method provided in the above-mentioned remote driving method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.
[0177] Fig.10 A schematic diagram of the structure of a remote driving assistance device provided in this application Figure 2 , applied to the vehicle side, such as Fig.10 As shown, the remote driving assistance device 100 provided in this embodiment includes:
[0178] The first acquisition module 1001 is used to acquire the surrounding environment information and current driving data of the vehicle;
[0179] The prediction module 1002 is used to predict the driving safety boundary line of the vehicle based on the current driving data, and obtain the predicted safety boundary point sequence information of the vehicle, where the predicted safety boundary point sequence information includes the vehicle safety boundary point information of the vehicle in the future time period;
[0180] The sending module 1003 is used to send the surrounding environment information and the predicted safety boundary point sequence information to the remote driving end, so that when the remote driving end displays the surrounding environment picture corresponding to the surrounding environment information, the predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0181] In a possible implementation, the current driving data includes current speed information, the vehicle stores a pre-calibrated mapping relationship between the speed and the critical curvature radius, and the prediction module is specifically used to:
[0182] Querying the mapping relationship between the speed and the critical curvature radius, obtaining the critical curvature radius corresponding to the speed information, the critical curvature radius being the minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information;
[0183] Based on the critical curvature radius, determining a reference point moving path of the vehicle; wherein the reference point of the vehicle is the center point of the rear axle of the vehicle;
[0184] In the vehicle motion simulation system, the vehicle is simulated to move along the reference point moving path, and the position information of the left and right corner points of the vehicle is collected at each position point where the vehicle moves;
[0185] Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained.
[0186] In a possible implementation, the prediction module is specifically used for:
[0187] Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period;
[0188] The motion trajectory of the vehicle in circular motion within the preset time period is determined based on the center and radius of the vehicle in circular motion within the preset time period and the current motion direction information of the vehicle, and the motion trajectory is determined as the reference point moving path of the vehicle.
[0189] In a possible implementation, the remote driving assistance device is further used to:
[0190] It is determined that the speed value corresponding to the current speed information of the vehicle is greater than a preset speed threshold.
[0191] A remote driving assistance device provided in this embodiment can execute the method provided in the above-mentioned vehicle-side method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.
[0192] Fig.11 This is a schematic diagram of the structure of a remote driving assistance device provided in this application. Fig.11 As shown, the remote driving assistance device 110 provided in this embodiment includes: at least one processor 1101 and a memory 1102. Optionally, the device 110 also includes a communication component 1103. The processor 1101, the memory 1102 and the communication component 1103 are connected via a bus 1104.
[0193] In a specific implementation process, at least one processor 1101 executes the computer-executable instructions stored in the memory 1102, so that at least one processor 1101 executes the above method.
[0194] The specific implementation process of processor 1101 can be found in the above-mentioned remote driving method embodiment, and its implementation principle and technical effects are similar, so this embodiment will not be repeated here.
[0195] Fig.12 A schematic diagram of the structure of a vehicle provided in this application. Fig.12 As shown, the vehicle 120 provided in this embodiment includes: at least one processor 1201 and a memory 1202. Optionally, the device 120 also includes a communication component 1203. The processor 1201, the memory 1202 and the communication component 1203 are connected via a bus 1204.
[0196] In a specific implementation process, at least one processor 1201 executes the computer-executable instructions stored in the memory 1202, so that at least one processor 1201 executes the above method.
[0197] The specific implementation process of processor 1201 can be found in the above-mentioned vehicle-side method embodiment. Its implementation principle and technical effects are similar, and will not be repeated here in this embodiment.
[0198] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.
[0199] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.
[0200] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.
[0201] The present application also provides a remote driving assistance system, Fig.13 This is a schematic diagram of the structure of a remote driving assistance system provided in the present application. The remote driving assistance system 130 includes a vehicle 1301 and a remote driving assistance device 1302, wherein:
[0202] Vehicle 1301 is used to obtain the surrounding environment information and current driving data of the vehicle; based on the current driving data, predict the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle, and the predicted safety boundary point sequence information includes the vehicle safety boundary point information of the vehicle in the future time period; send the surrounding environment information and the predicted safety boundary point sequence information to the remote driving end;
[0203] The remote driving assistance device 1302 is used to obtain surrounding environment information and predicted safety boundary point sequence information; on the display screen of the remote driving end, the surrounding environment picture corresponding to the surrounding environment information is displayed, and the predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information.
[0204] A remote driving assistance system provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail in this embodiment.
[0205] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0206] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0207] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0208] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0209] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0210] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0211] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0212] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0213] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0214] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A remote driving assistance method, characterized in that: Applied to the remote driving end, the remote driving assistance method includes: Acquiring surrounding environment information and predicted safety boundary point sequence information of a remotely driven vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information within a future time period predicted based on current driving data of the vehicle; On the display screen of the remote driving terminal, a surrounding environment picture corresponding to the surrounding environment information is displayed, and a predicted vehicle safety boundary line is superimposed on the surrounding environment picture, where the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information; Wherein, the current driving data includes the current speed information, the predicted vehicle safety boundary line is a vehicle moving boundary line predicted based on a pre-calibrated speed and a corresponding critical curvature radius for the vehicle to turn without rolling over, and the critical curvature radius is the minimum curvature radius for the vehicle to not roll over when the vehicle moves according to the speed information; the predicted safety boundary point sequence information of the vehicle is obtained by querying the mapping relationship between the speed and the critical curvature radius on the vehicle side to obtain the critical curvature radius corresponding to the speed information, and the reference point moving path of the vehicle is determined based on the critical curvature radius, and the vehicle motion simulation system simulates the movement of the vehicle along the reference point moving path, and after collecting the position information of the left and right corner points of the vehicle at each position point to which the vehicle moves, the reference point is determined based on the position information of the left and right corner points of the vehicle collected at each position point; wherein, the reference point of the vehicle is the center point of the rear axle of the vehicle; the reference point moving path of the vehicle is the center and radius of the circular motion of the vehicle in a preset time period determined by the critical curvature radius and the motion trajectory of the vehicle in a preset time period determined by the current motion direction information of the vehicle; The remote driving assistance method further includes: Acquire a real-time motion radius of the vehicle and a critical curvature radius corresponding to current speed information of the vehicle, wherein the real-time motion radius is calculated based on the acquired current speed information and yaw rate information of the vehicle; Calculating the difference between the real-time motion radius and the critical curvature radius; If the difference is less than a preset difference threshold, a reminder message is generated, and the reminder message is used to remind the user that the vehicle is at risk of rollover.
2. The remote driving assistance method according to claim 1, characterized in that: The predicted vehicle safety boundary line includes a first vehicle safety boundary line and a second vehicle safety boundary line, the first vehicle safety boundary line is a moving boundary line of the left vehicle body line, and the second vehicle safety boundary line is a moving boundary line of the right vehicle body line, and the predicted vehicle safety boundary line is superimposed and displayed on the surrounding environment screen, including: At least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line is drawn on the surrounding environment picture to overlay and display the predicted vehicle safety boundary line on the surrounding environment picture.
3. The remote driving assistance method according to claim 2, characterized in that: The vehicle safety boundary point information includes first vehicle safety boundary point information and second vehicle safety boundary point information, the surrounding environment picture includes a front environment picture of the vehicle, and drawing at least a portion of the first vehicle safety boundary line and / or the second vehicle safety boundary line on the surrounding environment picture includes: Based on the first vehicle safety boundary point information, at least a portion of the first vehicle safety boundary line is drawn on the environment picture in front of the vehicle, and / or, based on the second vehicle safety boundary point information, at least a portion of the second vehicle safety boundary line is drawn on the environment picture in front of the vehicle.
4. The remote driving assistance method according to any one of claims 1 to 3, characterized in that: The remote driving assistance method further includes: Acquire predicted trajectory point sequence information of the vehicle, wherein the predicted trajectory point sequence information includes driving trajectory point information within a future time period predicted based on current driving data of the vehicle; The predicted driving trajectory is superimposed and displayed on the surrounding environment picture, and the predicted driving trajectory is drawn based on the predicted trajectory point sequence information.
5. A remote driving assistance method, characterized in that: Applied to the vehicle side, the remote driving assistance method includes: Obtain the vehicle's surrounding environment information and current driving data; Based on the current driving data, predict the driving safety boundary line of the vehicle to obtain predicted safety boundary point sequence information of the vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information of the vehicle within a future time period; Sending the surrounding environment information and the predicted safety boundary point sequence information to a remote driving terminal, so that when the remote driving terminal displays a surrounding environment picture corresponding to the surrounding environment information, a predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information; The current driving data includes current speed information, the predicted vehicle safety boundary line is a vehicle movement boundary line predicted based on a pre-calibrated speed and a corresponding critical curvature radius at which the vehicle does not roll over when turning, and the critical curvature radius is a minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information; The method further comprises: Acquiring current speed information and yaw rate information of the vehicle, and sending the current speed information and yaw rate information of the vehicle to the remote driving end, so that the remote driving end calculates the current real-time motion radius of the vehicle based on the current speed information and yaw rate information of the vehicle, calculates the difference between the real-time motion radius and the critical curvature radius, and generates a reminder message to remind the user that the vehicle is at risk of rollover if the difference is less than a preset difference threshold; The vehicle stores a mapping relationship between a pre-calibrated speed and a critical curvature radius, and based on the current driving data, predicting the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle includes: Querying the mapping relationship between the speed and the critical curvature radius to obtain the critical curvature radius corresponding to the speed information; Based on the critical curvature radius, determining a reference point moving path of the vehicle; wherein the reference point of the vehicle is a rear axle center point of the vehicle; In the vehicle motion simulation system, the vehicle is simulated to move along the reference point moving path, and the position information of the left and right corner points of the vehicle are collected at each position point to which the vehicle moves; Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained; The step of determining a reference point moving path of the vehicle based on the critical curvature radius includes: Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period; The motion trajectory of the vehicle in circular motion within the preset time period is determined based on the center and radius of the circular motion of the vehicle within the preset time period and the current motion direction information of the vehicle, and the motion trajectory is determined as the reference point moving path of the vehicle.
6. The remote driving assistance method according to claim 5, characterized in that: Before predicting the driving safety boundary line of the vehicle based on the current driving data to obtain the predicted safety boundary point sequence information of the vehicle, the remote driving assistance method further includes: It is determined that a speed value corresponding to the current speed information of the vehicle is greater than a preset speed threshold.
7. A remote driving assistance device, characterized in that: Applied to the remote driving end, the remote driving assistance device includes: A first acquisition module is used to acquire surrounding environment information and predicted safety boundary point sequence information of a remotely driven vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information within a future time period predicted based on current driving data of the vehicle; A display module, configured to display a surrounding environment picture corresponding to the surrounding environment information on a display screen of the remote driving terminal, and to superimpose a predicted vehicle safety boundary line on the surrounding environment picture, wherein the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information; Wherein, the current driving data includes the current speed information, the predicted vehicle safety boundary line is a vehicle moving boundary line predicted based on a pre-calibrated speed and a corresponding critical curvature radius for the vehicle to turn without rolling over, and the critical curvature radius is the minimum curvature radius for the vehicle to not roll over when the vehicle moves according to the speed information; the predicted safety boundary point sequence information of the vehicle is obtained by querying the mapping relationship between the speed and the critical curvature radius on the vehicle side to obtain the critical curvature radius corresponding to the speed information, and the reference point moving path of the vehicle is determined based on the critical curvature radius, and the vehicle motion simulation system simulates the movement of the vehicle along the reference point moving path, and after collecting the position information of the left and right corner points of the vehicle at each position point to which the vehicle moves, the reference point is determined based on the position information of the left and right corner points of the vehicle collected at each position point; wherein, the reference point of the vehicle is the center point of the rear axle of the vehicle; the reference point moving path of the vehicle is the center and radius of the circular motion of the vehicle in a preset time period determined by the critical curvature radius and the motion trajectory of the vehicle in a preset time period determined by the current motion direction information of the vehicle; The remote driving assistance device is also used for: Acquire a real-time motion radius of the vehicle and a critical curvature radius corresponding to current speed information of the vehicle, wherein the real-time motion radius is calculated based on the acquired current speed information and yaw rate information of the vehicle; Calculating the difference between the real-time motion radius and the critical curvature radius; If the difference is less than a preset difference threshold, a reminder message is generated, and the reminder message is used to remind the user that the vehicle is at risk of rollover.
8. A remote driving assistance device, characterized in that: Applied to the vehicle side, the remote driving assistance device includes: A first acquisition module, used to acquire the surrounding environment information and current driving data of the vehicle; A prediction module, configured to predict the driving safety boundary line of the vehicle based on the current driving data, and obtain predicted safety boundary point sequence information of the vehicle, wherein the predicted safety boundary point sequence information includes vehicle safety boundary point information of the vehicle within a future time period; a sending module, configured to send the surrounding environment information and the predicted safety boundary point sequence information to a remote driving terminal, so that when the remote driving terminal displays a surrounding environment picture corresponding to the surrounding environment information, a predicted vehicle safety boundary line is superimposed on the surrounding environment picture, and the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information; The current driving data includes current speed information, the predicted vehicle safety boundary line is a vehicle movement boundary line predicted based on a pre-calibrated speed and a corresponding critical curvature radius at which the vehicle does not roll over when turning, and the critical curvature radius is a minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information; The remote driving assistance device is also used to: obtain the current speed information and yaw rate information of the vehicle, and send the current speed information and yaw rate information of the vehicle to the remote driving end, so that the remote driving end calculates the current real-time motion radius of the vehicle based on the current speed information and yaw rate information of the vehicle, calculates the difference between the real-time motion radius and the critical curvature radius, and generates a reminder message to remind the user that the vehicle is at risk of rollover if the difference is less than a preset difference threshold; The vehicle stores a mapping relationship between a pre-calibrated speed and a critical curvature radius, and based on the current driving data, predicting the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle includes: Querying the mapping relationship between the speed and the critical curvature radius to obtain the critical curvature radius corresponding to the speed information; Based on the critical curvature radius, determining a reference point moving path of the vehicle; wherein the reference point of the vehicle is a rear axle center point of the vehicle; In the vehicle motion simulation system, the vehicle is simulated to move along the reference point moving path, and the position information of the left and right corner points of the vehicle are collected at each position point to which the vehicle moves; Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained; The step of determining a reference point moving path of the vehicle based on the critical curvature radius includes: Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period; The motion trajectory of the vehicle in circular motion within the preset time period is determined based on the center and radius of the circular motion of the vehicle within the preset time period and the current motion direction information of the vehicle, and the motion trajectory is determined as the reference point moving path of the vehicle.
9. A remote driving assistance device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 4.
10. A vehicle, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to claim 5 or 6.
11. A remote driving assistance system, characterized in that: Including vehicles and remote driving assistance equipment; The vehicle is used to obtain the surrounding environment information and current driving data of the vehicle; Based on the current driving data, predict the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle, wherein the predicted safety boundary point sequence information includes the vehicle safety boundary point information of the vehicle in the future time period; send the surrounding environment information and the predicted safety boundary point sequence information to the remote driving terminal; The remote driving assistance device is used to obtain the surrounding environment information and the predicted safety boundary point sequence information; display the surrounding environment picture corresponding to the surrounding environment information on the display screen of the remote driving end, and superimpose the predicted vehicle safety boundary line on the surrounding environment picture, wherein the predicted vehicle safety boundary line is drawn based on the predicted safety boundary point sequence information; The current driving data includes current speed information, the predicted vehicle safety boundary line is a vehicle movement boundary line predicted based on a pre-calibrated speed and a corresponding critical curvature radius at which the vehicle does not roll over when turning, and the critical curvature radius is a minimum curvature radius at which the vehicle does not roll over when the vehicle moves according to the speed information; The vehicle stores a mapping relationship between a pre-calibrated speed and a critical curvature radius, and based on the current driving data, predicting the driving safety boundary line of the vehicle to obtain the predicted safety boundary point sequence information of the vehicle includes: Querying the mapping relationship between the speed and the critical curvature radius to obtain the critical curvature radius corresponding to the speed information; Based on the critical curvature radius, determining a reference point moving path of the vehicle; wherein the reference point of the vehicle is a rear axle center point of the vehicle; In the vehicle motion simulation system, the vehicle is simulated to move along the reference point moving path, and the position information of the left and right corner points of the vehicle are collected at each position point to which the vehicle moves; Based on the position information of the left and right corner points of the vehicle collected at each position point, the predicted safety boundary point sequence information of the vehicle is obtained; The step of determining a reference point moving path of the vehicle based on the critical curvature radius includes: Based on the critical curvature radius, determining the center and radius of the circular motion of the vehicle within a preset time period; Determine the motion trajectory of the circular motion of the vehicle within the preset time period based on the center and radius of the circular motion of the vehicle within the preset time period and the current motion direction information of the vehicle, and determine the motion trajectory as the reference point moving path of the vehicle; The vehicle is also used to obtain current speed information and yaw rate information of the vehicle, and send the current speed information and yaw rate information of the vehicle to a remote driving end, so that the remote driving end calculates the current real-time motion radius of the vehicle based on the current speed information and yaw rate information of the vehicle, calculates the difference between the real-time motion radius and the critical curvature radius, and generates a reminder message to remind the user that the vehicle is at risk of rollover if the difference is less than a preset difference threshold; The remote driving assistance device is also used to obtain the real-time motion radius of the vehicle and the critical curvature radius corresponding to the current speed information of the vehicle, wherein the real-time motion radius is calculated based on the acquired current speed information and yaw angular velocity information of the vehicle; the difference between the real-time motion radius and the critical curvature radius is calculated; if the difference is less than a preset difference threshold, a reminder message is generated, and the reminder message is used to remind the user that the vehicle is at risk of rollover.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method according to any one of claims 1-4 or 5-6.
13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 4 or 5 to 6 is implemented.
