Assisted driving method and apparatus, device and storage medium

Abstract

Embodiments of the present application disclose a kind of auxiliary driving method and device, equipment and storage medium, wherein the method comprises: obtaining the road information where vehicle is located;According to the road information where vehicle is located, generate three-dimensional scene;The environmental information around the vehicle obtained is fused with the three-dimensional scene, to generate the auxiliary driving information for guiding driving behavior;The three-dimensional scene after fusion is displayed.

Description

Technical Field

[0001] This application relates to computer technology, and to, but is not limited to, a driving assistance method, device, equipment, and storage medium. Background Technology

[0002] Currently, more and more vehicles have driver assistance functions, which can be achieved through driver assistance systems, such as lane departure warning systems, automatic emergency braking systems, lane keeping assist systems, forward collision warning systems, and pedestrian protection systems. Summary of the Invention

[0003] In view of the above, embodiments of this application provide an assisted driving method, apparatus, device, and storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] In a first aspect, embodiments of this application provide an assisted driving method, the method comprising:

[0006] Obtain road information where the vehicle is located;

[0007] A three-dimensional scene is generated based on the road information where the vehicle is located;

[0008] The acquired environmental information around the vehicle is fused with the three-dimensional scene to generate assisted driving information to guide driving behavior;

[0009] Display the merged 3D scene.

[0010] In some embodiments, fusing the acquired environmental information surrounding the vehicle with the three-dimensional scene to generate assisted driving information for guiding driving behavior includes: determining the vehicle's driving parameters and, based on the driving parameters, determining the display field of view of the three-dimensional scene; acquiring environmental information matching the driving parameters based on the driving parameters; and fusing the matching environmental information with the three-dimensional scene based on the display field of view to generate assisted driving information for guiding driving behavior.

[0011] In some embodiments, the vehicle's driving parameters include at least driving speed; correspondingly, determining the display field of view of the three-dimensional scene based on the driving parameters includes: if the vehicle's driving speed is greater than a first preset value, determining the three-dimensional scene as a first display field of view; if the vehicle's driving speed is less than the first preset value, determining the three-dimensional scene as a second display field of view; wherein the field of view of the first display field of view is greater than the field of view of the second display field of view.

[0012] In some embodiments, the vehicle's driving parameters include at least driving speed; correspondingly, acquiring environmental information matching the driving parameters includes: if the vehicle's driving speed is greater than a second preset value, acquiring first environmental information around the vehicle; if the vehicle's driving speed is less than the second preset value, acquiring second environmental information around the vehicle; wherein the first environmental information includes at least a first object, the second environmental information includes at least a second object, and the first object is more important to the vehicle than the second object is to the vehicle.

[0013] In some embodiments, displaying the fused 3D scene includes: analyzing the resource occupancy rate of electronic devices within a preset time period to obtain analysis results; adjusting the display parameters of the electronic devices based on the analysis results; and displaying the fused 3D scene based on the adjusted display parameters.

[0014] In some embodiments, adjusting the display parameters of the electronic device based on the analysis results includes at least one of the following: if the analysis results indicate that the resource occupancy rate is greater than a third preset value, adjusting the frequency at which the image continuously appears on the display device of the electronic device; if the analysis results indicate that the resource occupancy rate is greater than a fourth preset value, adjusting the rendering content of the display device.

[0015] In some embodiments, the resource utilization rate of the electronic device includes the resource utilization rate of the processing module of the electronic device; wherein the processing module includes at least one of the following: a central processing unit and a graphics processing unit.

[0016] Secondly, embodiments of this application provide a driver assistance device, the device comprising:

[0017] The acquisition unit is used to acquire road information where the vehicle is located;

[0018] The generation unit is used to generate a three-dimensional scene based on the road information where the vehicle is located;

[0019] The fusion unit is used to fuse the acquired environmental information around the vehicle with the three-dimensional scene to generate assisted driving information for guiding driving behavior.

[0020] The display unit is used to display the merged 3D scene.

[0021] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement the steps in the above-described assisted driving method.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the aforementioned assisted driving method.

[0023] This application provides an assisted driving method, device, equipment, and storage medium. It acquires road information about the vehicle's location; generates a three-dimensional scene based on the road information; fuses the acquired environmental information surrounding the vehicle with the three-dimensional scene to generate assisted driving information to guide driving behavior; and displays the fused three-dimensional scene. This enables a scenario where 3D navigation and assisted driving are integrated, allowing for a more intuitive and convenient understanding of the current navigation status and surrounding traffic conditions through the 3D scene display. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating the implementation process of the assisted driving method in the embodiments of this application. Figure 1 ;

[0025] Figure 2 This is a schematic diagram illustrating the implementation process of the assisted driving method in the embodiments of this application. Figure 2 ;

[0026] Figure 3 This is a schematic diagram illustrating the implementation process of the assisted driving method in the embodiments of this application. Figure 3 ;

[0027] Figure 4 This is a schematic diagram illustrating the implementation framework of the assisted driving method according to an embodiment of this application.

[0028] Figure 5 This is a schematic diagram of the composition of the driver assistance device according to an embodiment of this application;

[0029] Figure 6 This is a schematic diagram of a hardware entity of an electronic device according to an embodiment of this application. Detailed Implementation

[0030] The technical solutions of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0031] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0033] It should be noted that the terms "first, second, and third" used in the embodiments of this application are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0034] Based on this, this application provides an assisted driving method. The function implemented by this method can be achieved by the processor in the electronic device calling program code. Of course, the program code can be stored in the storage medium of the electronic device. Figure 1 This is a schematic diagram illustrating the implementation process of the assisted driving method in the embodiments of this application. Figure 1 ,like Figure 1 As shown, the method includes:

[0035] Step S101: Obtain road information where the vehicle is located;

[0036] Here, the electronic device can be any type of device with information processing capabilities, such as in-vehicle infotainment systems, navigation devices, smartphones, tablets, wearable devices, laptop computers, servers, or server clusters.

[0037] In-vehicle infotainment systems refer to the devices related to in-vehicle infotainment products installed in vehicles. Functionally, these systems enable information communication between people and vehicles, and between vehicles and the outside world (vehicle-to-vehicle communication). Most in-vehicle infotainment systems are installed inside the center console; in some systems, the main unit and screen are combined, while in others they are separate.

[0038] Regarding in-vehicle infotainment systems, the industry generally refers to them as on-board computers, intelligent central control systems, and in-vehicle navigation systems. Currently, in-vehicle infotainment system 3.0, or third-generation products, features driver assistance systems, 360-degree 3D (3-Dimensional) panoramic driving visual assistance systems, and a brand-new information aggregation desktop, which can better meet users' daily driving application needs.

[0039] In this embodiment of the application, road information where the vehicle is located can be obtained based on a high-precision map. For example, the road where the vehicle is located, as well as information such as lane lines, traffic signs, and trees on that road, can be determined using a high-precision map.

[0040] It should be noted that the road information of the vehicle in this embodiment is not limited to the road information corresponding to the lane the vehicle is currently in. The road information of the vehicle in this embodiment can be the road information corresponding to the lane the vehicle is currently in, or it can be the road information corresponding to multiple related lanes around the vehicle. This embodiment does not impose any restrictions on this.

[0041] Step S102: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0042] Here, a corresponding 3D scene can be generated based on the road information where the vehicle is located. The 3D scene includes a 3D scene image or 3D scene video from any viewpoint. For example, the viewpoint can be the field of vision corresponding to the vehicle driver, or it can be a 360-degree 3D panoramic view.

[0043] Step S103: The acquired environmental information around the vehicle is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0044] In this embodiment, environmental information surrounding the vehicle can be acquired through data acquisition devices (such as sensors, cameras, etc.). For example, sensors or cameras can be used to identify pedestrians, other vehicles, road obstacles, traffic congestion / abnormal conditions, etc., around the vehicle. Furthermore, the environmental information surrounding the vehicle can be fused with the three-dimensional scene to generate assisted driving information to guide driving behavior. For example, vehicles identified by sensors (including their actual colors) can be rendered into the three-dimensional scene, and the distances of these vehicles can be monitored in real time. If the distance to the first vehicle is less than a preset value, a warning is issued. The warning can be an audible alert or by changing the first vehicle's position in the three-dimensional scene to a flashing mode. As another example, when a vehicle reaches a designated location, the current traffic light display status identified by sensors can be rendered into the corresponding traffic light model in the three-dimensional scene.

[0045] In related technologies, driver assistance systems do not render the colors of surrounding vehicles; instead, they directly display gray car models on the central control screen of the vehicle's infotainment system. In this solution, vehicle images within a preset range are acquired via sensors, and the body color of each vehicle in the image is identified. The color of the vehicle model is then rendered on the central control screen, allowing the user to intuitively match the vehicles in the actual environment with those on the central control screen. The preset range can be set to the sensor's recognition range or smaller. When the preset range is smaller than the sensor's recognition range, its size can be determined based on the current vehicle speed for driving safety and user experience considerations. For example, the lower the speed, the smaller the preset range; the higher the speed, the larger the preset range. For instance, at a speed of 20 km / h, the preset range is a radius of 10 meters; at 100 km / h, the preset range is a radius of 30 meters. The sensor can be a lidar sensor, which can accurately determine the distance to other vehicles while identifying the vehicle's color. Of course, other types of sensors can also be used; this application does not limit this.

[0046] It should be noted that the tools and methods used to obtain information about the environment around the vehicle are not limited in the embodiments of this application.

[0047] Step S104: Display the fused 3D scene.

[0048] In this embodiment, the fused 3D scene information needs to be displayed for vehicle users (such as drivers) to view and provide assisted driving guidance. For example, the fused 3D scene can be displayed on the display screen of the vehicle's infotainment system for the driver to view, or it can be displayed on the driver's smartphone for the driver to view.

[0049] Here, through the assisted driving methods in steps S101 to S104 above, it is possible to realize the application scenario of integrating 3D navigation and assisted driving. The 3D scene display makes it more intuitive and convenient to understand the current navigation status and the surrounding vehicle conditions.

[0050] Based on the foregoing embodiments, this application further provides an assisted driving method, which is applied to an electronic device. Figure 2 This is a schematic diagram illustrating the implementation process of the assisted driving method in the embodiments of this application. Figure 2 ,like Figure 2 As shown, the method includes:

[0051] Step S201: Obtain road information where the vehicle is located;

[0052] Step S202: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0053] Step S203: Determine the vehicle's driving parameters, and based on the driving parameters, determine the display field of view of the three-dimensional scene;

[0054] In this embodiment, the vehicle's driving parameters include, but are not limited to, the vehicle's speed, direction of travel, and location. The vehicle's location includes the specific city where the vehicle is located, whether it is in an urban or suburban area, and its distance from intersections. Therefore, the display field of view of the 3D scene can be determined based on the vehicle's driving parameters. For example, the display field of view corresponding to a higher vehicle speed can be larger than that corresponding to a lower vehicle speed. Similarly, the display field of view corresponding to a vehicle traveling in a suburban area can be larger than that corresponding to a vehicle traveling in an urban area.

[0055] If the electronic device is a vehicle infotainment system, the vehicle's driving parameters can be obtained directly. Alternatively, the user can send the detected driving parameters to the vehicle infotainment system using a mobile phone. This application embodiment does not impose any restrictions on this.

[0056] Step S204: Based on the driving parameters, obtain environmental information that matches the driving parameters;

[0057] Here, different driving parameters not only correspond to different display fields of view, but also to different environmental information. In this embodiment, it is also necessary to determine and obtain environmental information matching the current driving parameters. This environmental information includes, but is not limited to: pedestrians, vehicles, obstacles, traffic signs, flowers, trees, and buildings.

[0058] Step S205: Based on the display field of view, the matching environmental information is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0059] Here, based on the corresponding display field of view, environmental information matching the current driving parameters can be fused with the three-dimensional scene to generate assisted driving information to guide driving behavior. For example, if the vehicle is currently driving in the suburbs, the view is zoomed out, and the sensors only identify important information such as vehicles and pedestrians, but not detailed information such as greenery. Conversely, if the vehicle is currently driving in the city, the view is zoomed in, and the sensors identify detailed information including trees, flowers, road signs, and obstacles.

[0060] Step S206: Display the fused 3D scene.

[0061] Here, through the assisted driving methods in steps S201 to S206 above, different assisted driving information can be integrated with 3D navigation with different field of view based on the vehicle's driving parameters. The 3D scene display makes it more intuitive and convenient to understand the current navigation status and surrounding vehicle conditions.

[0062] Based on the foregoing embodiments, this application further provides an assisted driving method, which is applied to an electronic device and includes:

[0063] Step S211: Obtain road information where the vehicle is located;

[0064] Step S212: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0065] Step S213: Determine the vehicle's driving parameters; wherein the driving parameters include at least the driving speed;

[0066] Here, the vehicle's speed can be monitored in real time while the vehicle is in motion.

[0067] Step S214: If the vehicle's speed is greater than a first preset value, determine the three-dimensional scene as the first display field of view;

[0068] Step S215: If the vehicle's speed is less than the first preset value, determine the three-dimensional scene as the second display field of view; wherein the field of view of the first display field of view is greater than the field of view of the second display field of view;

[0069] For example, the vehicle's current speed can be obtained from the vehicle's CANbus (vehicle serial bus system) signal. When the speed is greater than a first preset value, the distance between the virtual camera and the vehicle in the 3D scene is increased, widening the viewpoint (e.g., displaying a scene around the vehicle approximately 100 meters away). When the vehicle speed is less than the first preset value, the distance between the camera and the vehicle in the 3D scene is decreased, narrowing the viewpoint (e.g., displaying a scene around the vehicle approximately 40 meters away). When the vehicle speed fluctuates near the critical speed for viewpoint adjustment (the first preset value), an anti-shake algorithm needs to be added to avoid frequent viewpoint switching; for example, the viewpoint switching interval should be no less than 15 seconds.

[0070] Here, if the vehicle's current speed is equal to the first preset value, the display field of view of the three-dimensional scene can be determined as the first field of view or the second field of view according to the actual usage or user habits.

[0071] Step S216: Based on the driving parameters, obtain environmental information that matches the driving parameters;

[0072] Step S217: Based on the display field of view, the matching environmental information is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0073] Step S218: Display the fused 3D scene.

[0074] Here, through the assisted driving methods in steps S211 to S218 above, 3D navigation information from different perspectives can be integrated with assisted driving information based on the vehicle's driving speed. The 3D scene display makes it more intuitive and convenient to understand the current navigation status and surrounding vehicle conditions.

[0075] Based on the foregoing embodiments, this application further provides an assisted driving method, which is applied to an electronic device and includes:

[0076] Step S221: Obtain road information where the vehicle is located;

[0077] Step S222: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0078] Step S223: Determine the vehicle's driving parameters, and based on the driving parameters, determine the display field of view of the three-dimensional scene; wherein, the driving parameters include at least the driving speed;

[0079] Here, the vehicle's speed can be monitored in real time while the vehicle is in motion.

[0080] Step S224: If the vehicle's speed is greater than the second preset value, obtain the first environmental information around the vehicle;

[0081] Step S225: If the vehicle's speed is less than the second preset value, obtain the second environmental information around the vehicle; wherein, the first environmental information includes at least a first object, the second environmental information includes at least a second object, and the first object is more important to the vehicle than the second object is to the vehicle.

[0082] In this embodiment, the first object is more important to the vehicle than the second object is to the vehicle, meaning that the first object has a greater impact on the vehicle's movement than the second object. For example, the first object could be a pedestrian or a vehicle, and the second object could be greenery or an obstacle.

[0083] For example, the vehicle's current speed can be obtained from the vehicle's CANbus signal. When the speed is greater than a second preset value, the sensor only identifies important information such as vehicles and pedestrians. When the speed is less than the first preset value, the sensor identifies not only important information such as vehicles and pedestrians, but also detailed information such as greenery and obstacles.

[0084] Here, if the vehicle's current speed is equal to the second preset value, the first environmental information or the second environmental information can be obtained based on actual usage or user habits. In this embodiment, the second preset value and the first preset value can be the same value, or they can be different values; this embodiment does not impose any restrictions on this.

[0085] Step S226: Based on the display field of view, the matching environmental information is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0086] For example, when the vehicle's speed is greater than 50 km / h, the display field of view of the 3D scene can be adjusted to display the scene around the vehicle body within about 100 meters, and only the recognized vehicles, pedestrians and other important information can be rendered in the 3D scene.

[0087] Step S227: Display the fused 3D scene.

[0088] Here, through the assisted driving methods in steps S221 to S227 above, different assisted driving information and 3D navigation information can be integrated based on the vehicle's driving speed. The 3D scene display makes it more intuitive and convenient to understand the current navigation status and surrounding vehicle conditions.

[0089] Based on the foregoing embodiments, this application further provides an assisted driving method, which is applied to an electronic device. Figure 3 This is a schematic diagram illustrating the implementation process of the assisted driving method in the embodiments of this application. Figure 3 ,like Figure 3 As shown, the method includes:

[0090] Step S301: Obtain road information where the vehicle is located;

[0091] Step S302: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0092] Step S303: The acquired environmental information around the vehicle is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0093] Step S304: Analyze the resource occupancy rate of electronic devices within a preset time period and obtain the analysis results;

[0094] Here, the resource utilization rate of the processing modules of electronic devices can be analyzed within a preset time period to obtain corresponding analysis results. For example, the utilization rate of the CPU (Central Processing Unit) of a mobile phone can be analyzed to obtain corresponding analysis results; similarly, the utilization rate of the GPU (Graphics Processing Unit) of an in-vehicle infotainment device can be analyzed to obtain corresponding analysis results.

[0095] Step S305: Adjust the display parameters of the electronic device based on the analysis results;

[0096] In this embodiment of the application, the display parameters of the electronic device refer to the display parameters of the display device of the electronic device. These display parameters include, but are not limited to: the display content of the display device, the frame rate of the display device, the resolution of the display device, the power consumption of the display device, the refresh rate of the display device, and the pixel pitch of the display device.

[0097] Step S306: Display the fused 3D scene based on the adjusted display parameters.

[0098] Here, through the assisted driving methods in steps S301 to S306 above, the display parameters can be adjusted in real time, and the scene integrating 3D navigation and assisted driving can be displayed based on the adjusted display parameters.

[0099] In some embodiments, step S305, adjusting the display parameters of the electronic device based on the analysis results, includes at least one of the following:

[0100] Firstly, if the analysis results indicate that the resource occupancy rate is greater than a third preset value, the frequency at which the image continuously appears on the display device of the electronic device is adjusted.

[0101] For example, if the CPU utilization rate is greater than 60% and the GPU utilization rate is greater than 80% within a certain period of time, the frequency of images appearing continuously on the display device can be reduced.

[0102] The second method is to adjust the rendering content of the display device if the analysis results show that the resource occupancy rate is greater than the fourth preset value.

[0103] For example, if the CPU utilization is greater than 60% and the GPU utilization is greater than 80% within a certain period of time, the display of scene details can be reduced, and only content such as roads and vehicle models can be rendered.

[0104] Here, the third preset value and the fourth preset value may be the same or different. Furthermore, if the electronic device includes multiple resource utilization rates (such as CPU resource utilization rate and GPU resource utilization rate), the third preset value corresponding to each resource utilization rate may be the same or different; this embodiment does not impose any restrictions on this.

[0105] Based on the foregoing embodiments, this application further provides an assisted driving method, which is applied to an electronic device and includes:

[0106] Step S311: Obtain road information where the vehicle is located;

[0107] Step S312: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0108] Step S313: The acquired environmental information around the vehicle is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0109] Step S314: Analyze the resource occupancy rate of electronic devices within a preset time period and obtain the analysis results;

[0110] Step S315: If the analysis result shows that the resource occupancy rate is greater than the third preset value, adjust the frequency at which the image continuously appears on the display device of the electronic device;

[0111] For example, if the CPU utilization is greater than 60% and the GPU utilization is greater than 80% within a certain period of time, the frame rate of the vehicle's infotainment system display will be reduced.

[0112] Step S316: Display the fused 3D scene based on the adjusted display parameters.

[0113] Based on the foregoing embodiments, this application further provides an assisted driving method, which is applied to an electronic device and includes:

[0114] Step S321: Obtain road information where the vehicle is located;

[0115] Step S322: Generate a three-dimensional scene based on the road information where the vehicle is located;

[0116] Step S323: The acquired environmental information around the vehicle is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior;

[0117] Step S324: Analyze the resource occupancy rate of electronic devices within a preset time period and obtain the analysis results;

[0118] Step S325: If the analysis results indicate that the resource occupancy rate is greater than the fourth preset value, adjust the rendering content of the display device;

[0119] For example, if the CPU utilization is greater than 60% and the GPU utilization is greater than 80% within a certain period of time, the display of scene details can be reduced, and only content such as roads and vehicle models can be rendered. If, after a period of time, the CPU utilization is less than 60% and the GPU utilization is less than 80%, the rendering content of the display device can be increased, rendering not only roads and vehicle models, but also traffic signs, pedestrians, obstacles, greenery, and other information.

[0120] Step S326: Display the fused 3D scene based on the adjusted display parameters.

[0121] In some embodiments, the resource utilization rate of the electronic device includes the resource utilization rate of the processing module of the electronic device;

[0122] The processing module includes at least one of the following: a central processing unit (CPU) and a graphics processing unit (GPU).

[0123] Based on the foregoing embodiments, this application further provides a driving assistance method that addresses the problems in the prior art where "most in-vehicle navigation systems are 2D, which is not intuitive enough, and some 3D navigation and driving assistance solutions lack vehicle details, only displaying vehicle models in grayscale, resulting in monotonous scenes." This driving assistance method integrates 3D navigation and driving assistance information for display, and renders the vehicle model according to the actual usage scenario during the display process.

[0124] Figure 4 This is a framework diagram of the implementation of the assisted driving method according to an embodiment of this application, as shown below. Figure 4 As shown, the vehicle's infotainment system can receive information from sensors, such as information about surrounding vehicles and obstacles, and transmit this information to the system. Then, based on the received sensor information, vehicle status, and a high-precision map (optional), the system can render a 3D scene: displaying roads, surrounding vehicles (e.g., rendering surrounding vehicles in different colors according to their status), obstacles, etc.

[0125] The implementation details of the assisted driving method in the embodiments of this application are described below:

[0126] (1) When the vehicle is in motion, the vehicle's infotainment system receives information about surrounding obstacles, vehicles, pedestrians, etc. from sensors (such as cameras, radar, lasers, etc.).

[0127] (2) The vehicle system generates a 3D scene based on high-precision maps or surrounding road information identified by sensors, and integrates surrounding obstacles, vehicles, pedestrians and other information identified by sensors into the 3D scene.

[0128] (3) Based on the vehicle color information identified by the sensor, combined with information such as weather and light intensity, the 3D scene renders the body color of the vehicle model, and the sensor obtains the body color of the vehicle within a preset distance in real time.

[0129] (4) Based on the traffic light information identified by the sensor, the 3D scene renders the traffic light model and displays the current traffic light status; if the high-precision map contains traffic light location information, the traffic light model can be loaded into the scene when the specified location is reached, and the traffic light status identified by the sensor can be superimposed on the traffic light model.

[0130] (5) The 3D scene adjusts the viewing angle according to the speed. The current vehicle speed is obtained from the vehicle's Canbus signal. When the speed is high (e.g., greater than 50 km / h, the speed can be preset), the distance between the camera and the vehicle in the 3D scene is increased, the viewing angle is widened, and the sensor only recognizes important information such as vehicles and pedestrians, and does not recognize detailed information such as greenery. The 3D scene displays the scene around the vehicle body within about 100 meters, reducing the display of scene details (greenery, obstacles); when the vehicle speed is slow (e.g., less than 50 km / h, the speed can be preset), the distance between the camera and the vehicle in the 3D scene is reduced, the viewing angle is narrowed, and the scene around the vehicle body within about 40 meters is displayed, showing more scene details; when the vehicle speed fluctuates near the critical speed for viewing angle adjustment, an anti-shake algorithm needs to be added to avoid frequent switching of the viewing angle, for example, the viewing angle switching interval should not be less than 15 seconds.

[0131] (6) The 3D scene style changes dynamically according to the season, weather and festival. For example, the autumn style is yellowish and the flowers and trees are yellow; the spring style is fresh green and the flowers and trees are green; and the snow scene is displayed on snowy days.

[0132] (7) Adjust the scene detail display and frame rate according to the vehicle system performance. There are two ways to implement this method:

[0133] The first approach is to pre-test the vehicle's infotainment system performance. If the system's performance is low, reduce 3D scene details and frame rate to achieve a balance between display quality and user experience.

[0134] The second method involves real-time adjustment of the 3D scene. While the 3D scene is running, the frame rate, GPU utilization, and CPU utilization are monitored over a certain period (e.g., within 30 seconds). If the average frame rate is below 20fps, CPU utilization is above 60%, and GPU utilization is above 80%, scene detail is reduced, displaying only road and vehicle models, and the frame rate is also decreased. Note: The critical values ​​for frame rate, CPU, and GPU can all be preset and adjusted.

[0135] (8) When a vehicle is speeding, the driver is alerted by obvious interface changes, such as flashing red light around the scene and changing the color of the vehicle to red.

[0136] (9) When there is a risk of collision with surrounding obstacles, road boundaries, or other vehicles, red light will be displayed on the edges of the obstacles, road boundaries, and other vehicle models to remind the driver to pay attention to driving safety and maintain distance.

[0137] Therefore, the assisted driving method provided in this application embodiment can integrate assisted driving into 3D navigation, making it more intuitive and realistic; it can display the body color of surrounding vehicles in real time based on sensor data; it can adjust the viewing angle according to speed; and it can visualize vehicle speeding and vehicle driving danger warnings.

[0138] Therefore, the assisted driving method in this application embodiment can achieve the following technical effects:

[0139] 1) The integration of navigation and driver assistance, through 3D scene display, makes it more intuitive and convenient for drivers to understand the current navigation status and surrounding vehicle conditions.

[0140] 2) When encountering danger, timely hazard warnings can improve driving safety.

[0141] Based on the foregoing embodiments, this application provides an assisted driving device, which includes various units, modules, and components, and can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor), or FPGA (Field Programmable Gate Array), etc.

[0142] Figure 5 This is a schematic diagram of the composition of the assisted driving device according to an embodiment of this application, such as... Figure 5 As shown, the device 500 includes:

[0143] Acquisition unit 501 is used to acquire road information where the vehicle is located;

[0144] The generation unit 502 is used to generate a three-dimensional scene based on the road information where the vehicle is located;

[0145] The fusion unit 503 is used to fuse the acquired environmental information around the vehicle with the three-dimensional scene to generate assisted driving information for guiding driving behavior.

[0146] Display unit 504 is used to display the fused 3D scene.

[0147] In some embodiments, the fusion unit 503 includes:

[0148] The determination module is used to determine the vehicle's driving parameters and, based on the driving parameters, determine the display field of view of the three-dimensional scene;

[0149] The acquisition module is used to acquire environmental information that matches the driving parameters based on the driving parameters;

[0150] The fusion module is used to fuse the matching environmental information with the three-dimensional scene based on the display field of view to generate assisted driving information for guiding driving behavior.

[0151] In some embodiments, the vehicle's driving parameters include at least driving speed;

[0152] Correspondingly, the determining module includes:

[0153] The first determining component is used to determine the three-dimensional scene as the first display field of view if the vehicle's speed is greater than a first preset value.

[0154] The second determining component is used to determine the three-dimensional scene as the second display field of view if the vehicle's driving speed is less than the first preset value;

[0155] The field of view of the first display field of view is greater than that of the field of view of the second display field of view.

[0156] In some embodiments, the vehicle's driving parameters include at least driving speed;

[0157] Correspondingly, the acquisition module includes:

[0158] The first acquisition component is used to acquire first environmental information around the vehicle if the vehicle's speed is greater than a second preset value.

[0159] The second acquisition component is used to acquire second environmental information around the vehicle if the vehicle's speed is less than the second preset value.

[0160] The first environmental information includes at least a first object, and the second environmental information includes at least a second object, wherein the first object is more important than the second object relative to the vehicle.

[0161] In some embodiments, the display unit 504 includes:

[0162] The analysis module is used to analyze the resource utilization rate of electronic devices within a preset time period and obtain the analysis results;

[0163] An adjustment module is used to adjust the display parameters of the electronic device based on the analysis results;

[0164] The display module is used to display the fused 3D scene based on the adjusted display parameters.

[0165] In some embodiments, the adjustment module includes at least one of the following:

[0166] A first adjustment component is used to adjust the frequency at which the image continuously appears on the display device of the electronic device if the analysis result indicates that the resource occupancy rate is greater than a third preset value.

[0167] The second adjustment component is used to adjust the rendering content of the display device if the analysis results indicate that the resource occupancy rate is greater than a fourth preset value.

[0168] In some embodiments, the resource utilization rate of the electronic device includes the resource utilization rate of the processing module of the electronic device;

[0169] The processing module includes at least one of the following: a central processing unit (CPU) and a graphics processing unit (GPU).

[0170] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0171] It should be noted that, in the embodiments of this application, if the above-mentioned assisted driving method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM (Read Only Memory), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0172] Correspondingly, embodiments of this application provide an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the steps in the assisted driving method provided in the above embodiments.

[0173] Correspondingly, embodiments of this application provide a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the aforementioned assisted driving method.

[0174] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0175] It should be noted that, Figure 6 This is a schematic diagram of a hardware entity of an electronic device according to an embodiment of this application, such as... Figure 6 As shown, the hardware entity of the electronic device 600 includes: a processor 601, a communication interface 602, and a memory 603, wherein...

[0176] The processor 601 typically controls the overall operation of the electronic device 600.

[0177] The communication interface 602 enables the electronic device 600 to communicate with other devices, servers, or platforms via a network.

[0178] The memory 603 is configured to store instructions and applications executable by the processor 601, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 601 and various modules in the electronic device 600. It can be implemented by FLASH (flash memory) or RAM (Random Access Memory).

[0179] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0180] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0181] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0182] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0183] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0184] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0185] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A driving assistance method, characterized in that, The method includes: Obtain road information where the vehicle is located; A three-dimensional scene is generated based on the road information where the vehicle is located; The acquired environmental information around the vehicle is fused with the three-dimensional scene to generate assisted driving information to guide driving behavior; wherein, the fused three-dimensional scene includes at least the actual colors of other vehicles around the vehicle. Display the merged 3D scene; The process of fusing the acquired environmental information surrounding the vehicle with the three-dimensional scene to generate assisted driving information for guiding driving behavior includes: Determine the vehicle's driving parameters, and based on the driving parameters, determine the display field of view of the three-dimensional scene; Based on the driving parameters, obtain environmental information that matches the driving parameters; Based on the display field of view, the matching environmental information is fused with the three-dimensional scene to generate assisted driving information for guiding driving behavior; The vehicle's driving parameters include at least its driving speed; Correspondingly, obtaining environmental information matching the driving parameters based on the driving parameters includes: If the vehicle's speed is greater than the second preset value, obtain the first environmental information around the vehicle; If the vehicle's speed is less than the second preset value, obtain the second environmental information around the vehicle; Wherein, the first environmental information includes at least a first object, the second environmental information includes at least a second object, and the first object is more important than the vehicle; The process of displaying the fused 3D scene includes: The resource utilization rate of electronic devices within a preset time period is analyzed, and the analysis results are obtained. Based on the analysis results, the display parameters of the electronic device are adjusted; wherein the display parameters include at least: rendered content; wherein the rendered content includes at least: at least one of other vehicles, pedestrians, greenery, obstacles, roads, and traffic signs identified by the vehicle sensors; The fused 3D scene is displayed based on the adjusted display parameters.

2. The method according to claim 1, characterized in that, The vehicle's driving parameters include at least its driving speed; Correspondingly, determining the display field of view of the three-dimensional scene based on the driving parameters includes: If the vehicle's speed is greater than a first preset value, the three-dimensional scene is determined to be the first display field of view; If the vehicle's speed is less than the first preset value, the three-dimensional scene is determined to be the second display field of view; The field of view of the first display field of view is greater than that of the field of view of the second display field of view.

3. The method according to claim 1, characterized in that, Adjusting the display parameters of the electronic device based on the analysis results includes at least one of the following: If the analysis results indicate that the resource occupancy rate is greater than a third preset value, the frequency at which the image continuously appears on the display device of the electronic device will be adjusted. If the analysis results indicate that the resource occupancy rate is greater than the fourth preset value, the rendering content of the display device will be adjusted.

4. The method according to claim 1, characterized in that, The resource utilization rate of the electronic device includes the resource utilization rate of the processing module of the electronic device; The processing module includes at least one of the following: a central processing unit (CPU) and a graphics processing unit (GPU).

5. A driver assistance device, characterized in that, The device includes: The acquisition unit is used to acquire road information where the vehicle is located; The generation unit is used to generate a three-dimensional scene based on the road information where the vehicle is located; The fusion unit is used to fuse the acquired environmental information around the vehicle with the three-dimensional scene to generate assisted driving information for guiding driving behavior; wherein, the fused three-dimensional scene includes at least the actual colors of other vehicles around the vehicle. The display unit is used to process the merged 3D scene. The fusion unit includes: The determination module is used to determine the vehicle's driving parameters and, based on the driving parameters, determine the display field of view of the three-dimensional scene; The acquisition module is used to acquire environmental information that matches the driving parameters based on the driving parameters; The fusion module is used to fuse the matching environmental information with the three-dimensional scene based on the display field of view to generate assisted driving information for guiding driving behavior; The acquisition module includes: The first acquisition component is used to acquire first environmental information around the vehicle if the vehicle's speed is greater than a second preset value. The second acquisition component is used to acquire second environmental information around the vehicle if the vehicle's speed is less than the second preset value. Wherein, the first environmental information includes at least a first object, the second environmental information includes at least a second object, and the first object is more important than the vehicle; The analysis module is used to analyze the resource utilization rate of electronic devices within a preset time period and obtain the analysis results; An adjustment module is used to adjust the display parameters of the electronic device based on the analysis results; wherein the display parameters include at least: rendered content; wherein the rendered content includes at least one of other vehicles, pedestrians, greenery, obstacles, roads, and traffic signs identified by the vehicle sensors; The display module is used to display the fused 3D scene based on the adjusted display parameters.

6. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the assisted driving method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the assisted driving method according to any one of claims 1 to 4.

Images