Driving risk prompting method and device and vehicle

Abstract

The invention provides a driving risk prompting method and device and a vehicle, and belongs to the technical field of intelligent vehicles, and the driving risk prompting method comprises the steps: recognizing risk objects around the vehicle; displaying a floating window on a display screen of the vehicle; according to the position of the risk object, a target image of an area where the risk object is located is determined, the target image comprises a rendering image or a live-action image of the area where the risk object is located, and the proportion of the risk object in the target image is in a preset ratio interval; and displaying the target image in the floating window. According to the invention, the probability of dangerous situations caused by warehouse promotion decision making is effectively reduced, and the driving safety is improved. The driving risk prompting method and device are used for prompting the driving risk in the vehicle driving process.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a driving risk prompting method and device, and a vehicle. Background Art

[0002] With the development of intelligent driving technology, the vehicle's vision and radar systems can more accurately identify risk objects around the vehicle and the distance between obstacles and the vehicle, and display risk objects through renderings of the vehicle's surroundings. In this way, for risk objects on the road (such as rolling tires, small plastic bags, bumps or stones on the ground, etc.), users can be prompted through renderings of the vehicle's surroundings, thereby improving driving safety.

[0003] When rendering the rendering of the vehicle's surrounding environment, the rendering will be performed according to the actual size of the object. The actual size of the object matches the size in the rendering of the vehicle's surrounding environment, and the closer the object is to the vehicle, the larger it is in the rendering of the vehicle's surrounding environment. In order to indicate risky objects, in the related art, the identified risky objects are uniformly rendered in blue, that is, the risky objects in the rendering of the vehicle's surrounding environment are all displayed in blue.

[0004] However, the related technology uses color to mark risk objects. For tiny risk objects that are difficult to detect in the user's sight, they are displayed as small blue dots in the rendering of the vehicle's surrounding environment, which is also difficult for users to detect. This may result in users being unable to make decisions on risk objects in a timely manner, and cannot effectively improve driving safety. Summary of the invention

[0005] The present application provides a driving risk prompting method, device, and vehicle, which solves the problem in related technologies that it is difficult for users to perceive risky objects, can effectively reduce the probability of dangerous situations caused by hasty decisions, and improve driving safety.

[0006] In a first aspect, the present application provides a method for prompting driving risks, the method comprising: identifying risk objects around a vehicle; displaying a floating window on a display screen of the vehicle; determining a target image of the area where the risk object is located based on the location of the risk object, the target image comprising a rendering or a real-life image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; and displaying the target image in the floating window.

[0007] Among them, the number of floating windows is at least one, and its display area (also referred to as size) is a preset display area (also referred to as preset size), and the display area refers to the area of ​​the floating window when it is displayed on the display screen. Optionally, for the identified risk objects around the vehicle, the floating window corresponding to the risk object is displayed, that is, each identified risk object corresponds to a floating window. The target image is the image to be displayed in the floating window, and the display area of ​​the target image is the same as the display area of ​​the floating window.

[0008] When the distance between the vehicle and the identified risk object is close, the real-view image has a clearer display effect for the risk object. When the distance between the vehicle and the identified risk object is far, the rendered image has a clearer display effect for the risk object. In the embodiment of the present application, when the distance between the risk object and the vehicle is less than the preset distance, the target image includes a real-view image of the area where the risk object is located. When the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendered image of the area where the risk object is located.

[0009] The method solves the problem in the related art that the risk object is marked by a small blue dot, which makes it difficult for the user to detect it, and the risk object can only be clearly observed when the distance between the vehicle and the risk object is reduced to a certain extent, which may cause the user to be unable to make a decision on the risk object in time. Its beneficial effect is that on the one hand, the display of the floating window will play an obvious prompting role, which will make the user have the cognition that the appearance of the floating window is equivalent to the appearance of the risk object, thereby improving the user's risk awareness. On the other hand, the proportion of the risk object in the floating window is greater than the proportion of the risk object in the rendering of the vehicle's surrounding environment, so that the user can observe and identify the risk object more clearly through the floating window, grasp the information of the risk object, and make the right decision. And even if the distance between the vehicle and the risk object is far, the user can observe and grasp the risk object in time through the floating window, providing the user with sufficient decision-making time, so that the user can make corresponding driving decisions for the risk object in time. Compared with the related technology, it effectively reduces the probability of dangerous situations caused by hasty decisions and improves driving safety.

[0010] In a possible implementation, the target image may be obtained based on a rendering of the vehicle surrounding environment or a real-life image showing a risky object. The process of determining the target image of the area where the risky object is located according to the position of the risky object includes: determining the position of the risky object in an initial image according to the position of the risky object, the initial image including a rendering of the vehicle surrounding environment or a real-life image showing the risky object; determining a target magnification; magnifying the initial image according to the target magnification; and obtaining the target image based on the position of the risky object in the initial image and the magnified initial image.

[0011] The beneficial effect is that a rendering or real-life image of the vehicle's surroundings can be directly used, so that the vehicle consumes less computing resources in the process of determining the target image.

[0012] In a possible implementation, the process of determining the target magnification includes: determining the target magnification according to the distance between the risk object and the vehicle and a preset mapping relationship, wherein the preset mapping relationship includes a mapping relationship between distance and magnification, and in the preset mapping relationship, the greater the distance, the greater the magnification.

[0013] The preset mapping relationship may be a function of the magnification with respect to the distance. The function may be a continuous function, and the magnification changes in real time as the distance changes. The function may also be a piecewise function, and a plurality of magnifications are predefined in the function, and the plurality of magnifications correspond to distance intervals respectively.

[0014] In one possible implementation, the process of determining the target magnification ratio includes: determining the target magnification range based on the proportion of the risk object in the initial image, the display area of ​​the floating window, and a preset ratio range, and when the risk object in the initial image that has been magnified according to the magnification within the target magnification range is displayed in the floating window, the proportion in the floating window is within the preset ratio range; determining the target magnification ratio based on the target magnification range.

[0015] In a possible implementation, the target image may be determined based on the area where the risk object is located in the environmental feature model of the vehicle, and the target image includes a rendering of the area where the risk object is located. The process of determining the target image of the area where the risk object is located according to the position of the risk object includes: obtaining the environmental feature model of the vehicle; determining the target rendering perspective according to the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window; and rendering the environmental feature model according to the target rendering perspective to obtain the target image.

[0016] The environmental feature model can be obtained by modeling environmental information. The environmental information includes at least one of the following: point cloud data of a laser radar, image data of an image sensor, and feedback data of a millimeter wave radar.

[0017] The beneficial effect is that the clarity of the target image obtained by determining the rendering perspective is higher, and the risk object can be displayed more clearly in the subsequent display, making it easier for the user to observe and identify the risk object, grasp the information of the risk object, and make correct decisions.

[0018] In a possible implementation, the target image is acquired based on the target image sensor after adjusting the parameters, and the target image includes a real scene of the area where the risk object is located. The process of determining the target image of the area where the risk object is located according to the position of the risk object includes: determining the target image sensor and adjusting the imaging parameters of the target image sensor according to the position of the risk object; and determining the image captured by the target image sensor according to the adjusted imaging parameters as the target image.

[0019] For example, the image sensor corresponding to the real-life image showing the risky object may be determined as the target image sensor. Alternatively, the target image sensor may be determined based on the location of the risky object to better present the risky object. The target image sensor includes, but is not limited to: a camera close to one side of the risky object, a camera with a greater depth of field, and a macro camera.

[0020] In a possible implementation, the method also includes: determining risk information, the risk information including at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object; and displaying the risk information in a floating window.

[0021] The beneficial effect is that by displaying the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object, the user can directly identify the collision risk of the risk object and take avoidance measures in time.

[0022] In a possible implementation, the method also includes: obtaining a predicted driving trajectory of the vehicle and a predicted motion trajectory of the risk object; generating a driving animation of the vehicle based on the predicted driving trajectory; generating a motion animation of the risk object based on the predicted motion trajectory; and displaying the driving animation of the vehicle and the motion animation of the risk object in a floating window.

[0023] The beneficial effect is that by displaying the driving animation of the vehicle and the motion animation of the risk object, the presentation of the collision animation can more directly prompt the collision risk compared to simply presenting static information.

[0024] In one possible implementation, the vehicle's display screen displays a rendering of the vehicle's surrounding environment, and the process of displaying a floating window on the vehicle's display screen includes: determining the target position of the risk object relative to the vehicle within a preset time in the future; and displaying a floating window at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

[0025] The beneficial effect is that the display orientation of the floating window is the same as the actual orientation of the risk object relative to the vehicle, which helps the user to judge the direction of the risk object through the floating window and facilitates the user to make driving decisions.

[0026] In a possible implementation, the method further includes: looping through a risk warning process, the risk warning process including: identifying risk objects around the vehicle; updating a target image of the area where the risk objects are located based on the latest positions of the risk objects; and displaying the updated target image in a floating window.

[0027] In a second aspect, the present application provides a driving risk prompting device, which includes: an identification module for identifying risk objects around a vehicle; a display module for displaying a floating window on the vehicle's display screen; an image determination module for determining a target image of the area where the risk object is located based on the position of the risk object, the target image including a rendering or a real-life image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; the display module is also used to display the target image in the floating window.

[0028] In one possible implementation, the image determination module is specifically used to: determine the position of the risk object in the initial image according to the position of the risk object, the initial image including a rendering of the vehicle's surrounding environment or a real-life image showing the risk object; determine a target magnification; enlarge the initial image according to the target magnification; and obtain a target image based on the position of the risk object in the initial image and the enlarged initial image.

[0029] In a possible implementation, the image determination module is specifically used to determine the target magnification based on the distance between the risk object and the vehicle and a preset mapping relationship. The preset mapping relationship includes a mapping relationship between distance and magnification. In the preset mapping relationship, the greater the distance, the greater the magnification.

[0030] In one possible implementation, the image determination module is specifically used to: determine a target magnification range based on the proportion of risk objects in the initial image, the display area of ​​the floating window, and a preset ratio range, so that when the risk objects in the initial image that are magnified according to the magnification within the target magnification range are displayed in the floating window, their proportion in the floating window is within the preset ratio range; determine the target magnification ratio based on the target magnification range.

[0031] In one possible implementation, the target image includes a rendering of the area where the risk object is located, and the image determination module is specifically used to: obtain an environmental feature model of the vehicle; determine a target rendering perspective based on the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window; render the environmental feature model according to the target rendering perspective to obtain a target image.

[0032] In a possible implementation, the target image includes a real-life image of the area where the risk object is located, and the image determination module is specifically used to: determine the target image sensor and adjust the camera parameters of the target image sensor according to the location of the risk object; and determine the image captured by the target image sensor according to the adjusted camera parameters as the target image.

[0033] In one possible implementation, when the distance between the risk object and the vehicle is less than a preset distance, the target image includes a real-life image of the area where the risk object is located; when the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendering of the area where the risk object is located.

[0034] In one possible implementation, the device also includes: a risk information determination module, used to determine risk information, the risk information including at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object; the aforementioned display module is also used to display the risk information in a floating window.

[0035] In a possible implementation, the risk information determination module is also used to: obtain the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object; generate a driving animation of the vehicle based on the predicted driving trajectory; generate a motion animation of the risk object based on the predicted motion trajectory; the display module is also used to display the driving animation of the vehicle and the motion animation of the risk object in a floating window.

[0036] In a possible implementation, the display module is specifically used to: determine the target position of the risk object relative to the vehicle within a preset future time; and display a floating window at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

[0037] In one possible implementation, the recognition module is also used to identify risk objects around the vehicle when the risk warning process is cyclically executed; the image determination module is also used to update the target image of the area where the risk object is located according to the latest position of the risk object when the risk warning process is cyclically executed; the display module is also used to display the updated target image in the floating window when the risk warning process is cyclically executed.

[0038] In a third aspect, the present application provides a driving risk reminder device, which includes: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by one or more processors, the one or more processors implement a method as described in any one of the first aspects.

[0039] In a fourth aspect, the present application provides a vehicle, comprising: a driving risk reminder device as described in any one of the second aspect or the third aspect.

[0040] In a fifth aspect, the present application provides a driving risk reminder device, including a processor for executing a method as described in any one of the first aspects.

[0041] In a sixth aspect, the present application provides a driving risk reminder device, comprising: a processing circuit and an interface circuit; wherein the interface circuit is used to couple with a memory outside the driving risk reminder device and provide a communication interface for the processing circuit to access the memory; the processing circuit is used to execute program instructions in the memory to implement a method as described in any one of the first aspects.

[0042] In the specific implementation process, the driving risk reminder device can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit can be, for example, but not limited to, received and input by a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit can be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiment of the present application does not limit the specific implementation of the processor and various circuits.

[0043] In one implementation, the driving risk reminder device may be a wireless communication device, that is, a computer device that supports wireless communication functions. Specifically, the wireless communication device may be a terminal such as a smart phone, or a wireless access network device such as a base station. The system chip may also be referred to as a system on chip (SoC), or simply as an SoC chip. The communication chip may include a baseband processing chip and a radio frequency processing chip. The baseband processing chip is sometimes also referred to as a modem or a baseband chip. The radio frequency processing chip is sometimes also referred to as a radio frequency transceiver or a radio frequency chip. In a physical implementation, some or all of the chips in the communication chip may be integrated inside the SoC chip. For example, the baseband processing chip is integrated in the SoC chip, and the radio frequency processing chip is not integrated with the SoC chip. The interface circuit may be a radio frequency processing chip in a wireless communication device, and the processing circuit may be a baseband processing chip in a wireless communication device.

[0044] In another implementation, the driving risk reminder device may be a part of a wireless communication device, such as an integrated circuit product such as a system chip or a communication chip. The interface circuit may be an input / output interface, an interface circuit, an output circuit, an input circuit, a pin or a related circuit on the chip or chip system. The processor may also be embodied as a processing circuit or a logic circuit.

[0045] In a seventh aspect, the present application provides a computer-readable storage medium, in which a program code is stored. When the program code is executed by a processor, the method as described in any one of the first aspects is implemented.

[0046] In an eighth aspect, the present application provides a chip, comprising: at least one processor. The at least one processor is configured to execute the method as described in any one of the first aspects.

[0047] Optionally, the chip further includes a memory. At least one processor is used to execute the code in the memory, and when the at least one processor executes the code, the chip implements the method as described in any one of the first aspects.

[0048] Optionally, the above chip may also be an integrated circuit.

[0049] In a ninth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to implement the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0051] Figure 2 A functional block diagram of a vehicle provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of a driving risk prompting method provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a rendering of a vehicle surrounding environment provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of a rendering perspective of an object provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of displaying a target image in a floating window provided in an embodiment of the present application;

[0056] Figure 7 A flowchart of another driving risk prompting method provided in an embodiment of the present application;

[0057] Figure 8 A schematic diagram of displaying a floating window provided in an embodiment of the present application;

[0058] Fig. 9 A schematic diagram of a HUD provided in an embodiment of the present application;

[0059] Fig.10 A block diagram of a driving risk reminder device provided in an embodiment of the present application;

[0060] Fig.11 A block diagram of another driving risk reminder device provided in an embodiment of the present application;

[0061] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0062] Fig.13 A schematic diagram of the structure of a driving risk reminder device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] The terms "first", "second", etc. in the specification embodiments, claims, and drawings of the present application are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, including a series of steps or units. The method, system, product, or device is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0065] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0066] The embodiment of the present application provides a driving risk prompting method, which solves the problem in the related art that the user may not be able to make a decision on the risk object in time, and enables the user to observe the risk object more easily and clearly at a longer distance, providing the user with a longer decision time, thereby improving driving safety. The method can be applied to a car (such as a smart car) or a cockpit (such as a smart cockpit).

[0067] Please refer to Figure 1 , Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application, Figure 1 Vehicle 001 and user 002 are shown. Vehicle 001 has a built-in vehicle operating system (referred to as vehicle computer) and is equipped with a variety of sensors. The vehicle computer can automatically determine the driving strategy in the current scenario based on the collected sensor information and internal data, and generate and execute instructions corresponding to the driving strategy. The vehicle computer can also analyze the collected sensor information, identify risk objects around the vehicle that affect the vehicle's driving, and remind the user. User 002 is often a driver or car owner, who can obtain various information around the vehicle through the information displayed on the central control screen or the dashboard, and identify risks around the vehicle based on the displayed information and / or other sound and light information.

[0068] The following is a description of the specific structure of the vehicle. Figure 2 , Figure 2A functional block diagram of a vehicle provided in an embodiment of the present application, the vehicle 100 may include various subsystems, such as a travel system 101, a sensor system 102, a control system 103, one or more peripheral devices 104, a power supply 105, a user interface 106 and a computer system 107. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the vehicle 100 may be interconnected by wire or wireless, for example, by an on-board communication network, and the on-board communication network may include a controller area network bus (Controller Area Network Bus, CAN Bus) or an on-board Ethernet, etc.

[0069] Travel system 101 may include components for providing power to vehicle 100. In one embodiment, travel system 101 may include engine 1011, energy source 1012, transmission 1013, and wheels 1014.

[0070] The engine 1011 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engines or a combination of multiple engines. The combination of multiple engines may include, for example, a hybrid engine consisting of a gasoline engine and an electric motor, and a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 1011 is used to convert the energy source 1012 into mechanical energy.

[0071] Energy source 1012 may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 1012 may also provide energy to other systems of vehicle 100.

[0072] The transmission 1013 can transmit the mechanical power from the engine 1011 to the wheels 1014. The transmission 1013 can include a gearbox, a differential and a drive shaft. In one embodiment, the transmission 1013 can also include other devices, such as a clutch. Among them, the drive shaft can include one or more shafts that can be coupled to one or more wheels 1014.

[0073] The sensor system 102 may include several sensors that sense information about the vehicle 100 itself and the environment around the vehicle 100. For example, the sensor system 102 may include a positioning system 1021 (the positioning system may be a global positioning system (GPS), or a Beidou system or other positioning systems), an inertial measurement unit (IMU) 1022, a radar 1023 (the radar may be a laser radar), a laser rangefinder 1024, a camera 1025, a computer vision system 1026, and a sensor fusion algorithm 1027. The sensor system 102 may also include sensors of the internal systems of the vehicle 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, and an oil temperature gauge, etc.). One or more sensor data from these sensors may be used to detect an object to be detected and its corresponding characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the vehicle 100 to achieve safe operation.

[0074] Among them, the positioning system 1021 can be used to estimate the geographic location of the vehicle 100. The IMU 1022 can be used to sense the position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 1022 can be a combination of an accelerometer and a gyroscope.

[0075] The radar 1023 can use radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 1023 can also be used to sense the speed and / or direction of travel of the object. The radar 1023 includes but is not limited to a laser radar and a millimeter wave radar.

[0076] The laser rangefinder 1024 may utilize laser light to sense objects in the environment of the vehicle 100. In some embodiments, the laser rangefinder 1024 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0077] The camera 1025 may be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 1025 may be a still camera or a video camera.

[0078] The computer vision system 1026 can be used to process and analyze the images captured by the camera 1025 to identify objects and / or features in the environment surrounding the vehicle 100. The objects and / or features may include traffic signs, road boundaries, obstacles, and road conditions. The computer vision system 1026 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 1026 may be used to map the environment, track objects, estimate the speed of objects, etc.

[0079] The control system 103 is used to perform operations to control the vehicle 100 and its components. The control system 103 may include various components. Figure 2 As shown, the control system 103 may include a steering system 1031 , a throttle 1032 , a braking unit 1033 , a route planning system 1034 , and an obstacle avoidance system 1035 .

[0080] The forward direction of the vehicle 100 can be adjusted by operating the steering system 1031 . For example, in one embodiment, it can be a steering wheel system. The throttle 1032 is used to control the operating speed of the engine 1011 and thus control the speed of the vehicle 100 .

[0081] The brake unit 1033 is used to control the deceleration of the vehicle 100. The brake unit 1033 can use friction to slow down the wheel 1014. In other embodiments, the brake unit 1033 can convert the kinetic energy of the wheel 1014 into electric current. The brake unit 1033 can also take other forms to slow down the rotation speed of the wheel 1014 to control the speed of the vehicle 100.

[0082] The route planning system 1034 is used to determine the driving route of the vehicle 100. In some embodiments, the route control system 1034 can combine data from the sensor system 102 (such as the positioning system 1021 and the computer vision system 1026) and one or more predetermined maps to plan a driving route for the vehicle 100 that can avoid potential obstacles in the environment.

[0083] The obstacle avoidance system 1035 is used to generate control values ​​of the accelerator, brake and steering angle according to the driving route or driving trajectory output by the route planning system 1034, so as to control the steering system 1031, the accelerator 1032 and the brake unit 1033.

[0084] Of course, in one example, the control system 103 may include additional or alternative components other than those shown and described, or may reduce some of the components shown above.

[0085] The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through one or more peripheral devices 104. The peripheral devices 104 may include a wireless communication system 1041, an onboard computer 1042, a microphone 1043, and / or a speaker 1044.

[0086] In some embodiments, the peripheral device 104 is used to provide a means for the user of the vehicle 100 to interact with the user interface 106. For example, the onboard computer 1042 can provide information to the user of the vehicle 100. The user interface 106 can also operate the onboard computer 1042 to receive the user's input. In one implementation, the onboard computer 1042 can be operated through a touch screen. In other cases, the peripheral device 104 can provide a means for the vehicle 100 to communicate with other devices located in the vehicle. For example, the microphone 1043 can receive audio (e.g., voice commands or other audio input) from the user of the vehicle 100. Similarly, the speaker 1044 can output audio to the user of the vehicle 100.

[0087] The wireless communication system 1041 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 1041 can use the third generation mobile communication technology (3G) communication, such as code division multiple access (CDMA), global system for mobile communications (GSM), or the fourth generation mobile communication technology (4G) communication, such as LTE (long term evolution). Or the fifth generation mobile communication technology (5G) communication. The wireless communication system 1041 can communicate with a wireless local area network (WLAN) using wireless fidelity (WiFi). In some embodiments, the wireless communication system 1041 can communicate directly with the device using an infrared link, Bluetooth or Zigbee. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system 1041 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communication devices between vehicles and / or roadside stations.

[0088] The power source 105 can provide power to various components of the vehicle 100. In one embodiment, the power source 105 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such batteries can be configured as a power source and provide power to various components of the vehicle 100. In some embodiments, the power source 105 and the energy source 1012 can be implemented together, such as in a fully electric vehicle.

[0089] Some or all functions of the vehicle 100 are controlled by a computer system 107. The computer system 107 may include at least one processor 1071 that executes instructions 1073 stored in a non-transitory computer-readable medium such as a memory 1072. The computer system 107 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0090] The processor 1071 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor 1071 may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Figure 2 The processor 1071, memory 1072 and other elements of the computer system 107 are functionally illustrated, but it should be understood by those skilled in the art that the processor 1071 and memory 1072 may actually include multiple other processors or memories that are not located in the same physical housing. For example, the memory 1072 may be a hard drive or other storage medium located in a housing different from the computer system 107. Therefore, references to the processor 1071 will be understood to include references to a collection of processors or memories that may operate in parallel or not in parallel. Different from using a single processor to perform the steps described herein, some components such as steering components and deceleration components may each have their own processor that performs only calculations related to the functions specific to the component; or subsystems such as the travel system 101, the sensor system 102 and the control system 103 may also have their own processors for implementing the calculations of the relevant tasks of the corresponding subsystems to achieve the corresponding functions.

[0091] In various aspects described herein, the processor 1071 may be located remotely from the vehicle 100 and in wireless communication with the vehicle 100. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle 100, while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0092] In some embodiments, the memory 1072 may include instructions 1073 (e.g., program logic) that may be executed by the processor 1071 to perform various functions of the vehicle 100, including those described above. The memory 1072 may also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the travel system 101, the sensor system 102, the control system 103, and the peripheral device 104.

[0093] In addition to the instructions 1073, the memory 1072 may also store other relevant data, such as map information, route information, the vehicle's location, direction, speed, and other relevant information. Such information may be used by the vehicle 100 or specifically by the computer system 107 during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0094] The user interface 106 is used to provide information to or receive information from a user of the vehicle 100. Optionally, the user interface 106 may include one or more input / output devices within the set of peripherals 104, such as a wireless communication system 1041, an onboard computer 1042, a microphone 1043, and a speaker 1044.

[0095] Computer system 107 may control functions of vehicle 100 based on input received from various subsystems (e.g., travel system 101, sensor system 102, and control system 103) and from user interface 106. In some embodiments, computer system 107 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0096] Alternatively, one or more of the above-mentioned components may be installed or associated separately from the vehicle 100. For example, the memory 1072 may exist partially or completely separately from the vehicle 100. The above-mentioned components may be communicatively coupled together in a wired and / or wireless manner.

[0097] Optionally, the above components are only examples. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 2 It should not be understood as limiting the embodiments of the present application.

[0098] An autonomous vehicle traveling on a road, such as vehicle 100 above, can identify objects in its surrounding environment to determine adjustments to the vehicle's trajectory, including adjustments to the vehicle's speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently and based on the object's respective characteristics, such as its current speed, acceleration, spacing from the vehicle, etc., used to determine the trajectory planning of the autonomous vehicle, including the speed to be adjusted.

[0099] Optionally, a computing device associated with the vehicle 100 (eg, Figure 2The computer system 107, the computer vision system 1026) can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all the identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 100 is able to plan its driving trajectory (including speed) based on the predicted behavior of the identified object. And based on the planning results, it is determined for the autonomous vehicle what state the vehicle will need to be adjusted to (in terms of speed adjustment, such as acceleration, deceleration or stopping). In this process, other factors can also be considered to determine the driving trajectory of the vehicle 100, such as the lateral position of the vehicle 100 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0100] Based on the planning results, in addition to adjusting the speed for the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0101] The vehicle 100 may be a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, a lawn mower, an entertainment vehicle, an amusement park vehicle, construction equipment, a tram, a golf cart, a train, a cart, etc., and the embodiments of the present application are not particularly limited thereto.

[0102] Please refer to Figure 3 , Figure 3 A flowchart of a driving risk prompting method provided in an embodiment of the present application is provided. The method can be applied to a cockpit or a vehicle, for example Figure 1 and Figure 2 The vehicle shown in the figure may include the following process:

[0103] 201. Identify risk objects around the vehicle.

[0104] Risk objects around the vehicle may affect the normal driving of the vehicle. They may be static objects or dynamic objects. For example, they may be objects on the future driving trajectory of the vehicle, or objects that will dynamically move to the future driving trajectory of the vehicle in the future. The types of risk objects include, but are not limited to: other vehicles on the road, rolling tires, floating objects in the air such as plastic bags, bumps and stones on the ground, etc.

[0105] The vehicle analyzes the various information obtained, and then identifies risk objects around the vehicle according to a preset algorithm (also called a risk object identification algorithm).

[0106] As can be seen from the above description, the vehicle includes a sensor system, and the information obtained by the vehicle includes sensor information inside and outside the vehicle collected by several sensors in the sensor system, such as sensor information related to the surrounding environment of the vehicle. The sensor information includes, but is not limited to: images collected by image sensors, point cloud data collected by laser radars, and radar feedback data received by millimeter-wave radars. When identifying risk objects around the vehicle through images collected by image sensors, multiple images can be analyzed simultaneously.

[0107] The information obtained by the vehicle may also include environmental information around the vehicle obtained through the network or communication. For example, a high-precision map of the vehicle's surroundings can be obtained through the network or communication, and then environmental information can be obtained based on the high-precision map. For example, vehicle A updates the local high-precision map based on the point cloud information collected by the laser radar. After that, the vehicle A's car computer compares the local high-precision map with the high-precision map in the cloud. If there are differences between the two high-precision maps, the local high-precision map can be synchronized to the cloud in real time. Vehicle B can directly download the latest high-precision map synchronized by vehicle A from the cloud. For another example, a road device (such as an image sensor) can also scan nearby environmental information in real time and update a high-precision map of the nearby area. The vehicle communicates with the road device, or communicates with the cloud / server (such as a traffic management system) connected to the road device, and then obtains the latest high-precision map. The vehicle analyzes the latest high-precision map obtained to identify risk objects around the vehicle.

[0108] The preset algorithm may be built into the vehicle. The vehicle uses the acquired information or the analysis results of the information as input to the preset algorithm, and the preset algorithm outputs the risk objects around the vehicle. The principle of the preset algorithm may be to determine whether the object in the input information is on the future driving trajectory of the vehicle, or whether it will move to the future driving trajectory of the vehicle in the future.

[0109] For example, the preset algorithm first identifies the object (including dynamic object and static object) based on the input information. Then, it determines whether the object has the possibility of colliding with the vehicle in the future, that is, it determines whether the object has the possibility of appearing on the future driving trajectory of the vehicle. If the object is static, it determines whether the object is in the future driving trajectory of the vehicle. If the object is dynamic, the future motion trajectory of the object is first predicted, and then based on the future motion trajectory of the object and the future driving trajectory of the vehicle, it is determined whether the object has the possibility of colliding with the vehicle in the future.

[0110] This process can be performed by the vehicle's vision and radar systems. It should be noted that the process 201 is always performed during the vehicle's driving process, that is, the risk objects around the vehicle are continuously identified during the vehicle's driving process.

[0111] 202. Display a floating window on a display screen of the vehicle.

[0112] The number of floating windows is at least one, and the display area (also referred to as size) thereof is a preset display area (also referred to as preset size), and the display area refers to the area of ​​the floating window when it is displayed on the display screen. The number of floating windows is related to the risk objects identified in the aforementioned process 201. Optionally, for each risk object identified, a floating window corresponding to the risk object is displayed, that is, each risk object identified corresponds to one floating window.

[0113] The display screen may be a central control screen or an instrument screen of the vehicle, etc., which is not limited in the embodiments of the present application. During driving, a rendering of the vehicle's surrounding environment may be displayed on the display screen. At this time, a floating window may be created on the upper layer of the rendering of the vehicle's surrounding environment, thereby displaying the floating window.

[0114] The rendering of the vehicle's surrounding environment is a two-dimensional rendering that displays the vehicle's surrounding environment with a three-dimensional effect on the vehicle's display screen. It can be obtained based on information acquired by the vehicle (such as at least one of the point cloud data of the lidar, the image data of the image sensor, and the feedback data of the millimeter-wave radar).

[0115] For example, the information obtained by the vehicle can be used to model the environment and obtain an environmental feature model, which is a three-dimensional (3D) model. The environmental feature model is then rendered using a preset rendering perspective to obtain a rendering of the vehicle's surrounding environment. The rendering process can be regarded as simulating a virtual camera, presenting the environmental feature model through the imaging perspective of the virtual camera, thereby displaying the environmental feature model on a two-dimensional plane, and the rendering perspective is the imaging perspective of the virtual camera. It can be seen from this that there may be some areas in the environmental feature model that are not captured by the virtual camera, that is, there will be some areas that are not displayed.

[0116] The preset rendering perspective is related to the vehicle, which uses the vehicle as a reference. When the position of the vehicle changes, the preset rendering perspective will also change accordingly. The preset rendering perspective allows the user to fully observe the environment around the vehicle.

[0117] When creating a rendering of the vehicle's surroundings, for objects that will not affect the vehicle's driving (such as buildings on the street), such objects can be simplified and displayed by filtering the information obtained by the vehicle. The rendering of the vehicle's surroundings can reflect the environmental information around the vehicle in real time, such as surrounding vehicles, road signs, traffic signs, pedestrians, road edges, lane lines, and zebra crossings. During the vehicle's driving, the rendering of the vehicle's surroundings is displayed on the display screen. Users can grasp the environment around the vehicle through the rendering of the vehicle's surroundings, without having to use traditional methods such as rearview mirrors / observing outside the vehicle. The rendering of the vehicle's surroundings is real-time, and the vehicle continuously updates the rendering of the vehicle's surroundings based on the information obtained in real time.

[0118] For example, please refer to Figure 4 , Figure 4 A schematic diagram of a rendering of a vehicle surrounding environment provided in an embodiment of the present application, Figure 4 FIG. 1 shows a rendering of the vehicle's surrounding environment displayed on the display screen during vehicle driving. Figure 4 As shown in FIG. 1 , the rendering of the vehicle's surrounding environment shows the 3D vehicle model, the road / lane where the vehicle is located, and the vehicle's driving route. In addition, objects such as traffic lights and other vehicles on the road are also shown. Figure 4 It can be seen that objects related to vehicle driving, such as roads, vehicles, traffic lights, lane lines, and zebra crossings, are rendered in a refined manner during modeling. Objects that do not affect vehicle driving, such as buildings on both sides of the road, are simplified during modeling.

[0119] The floating window can be displayed at any position of the display screen. Optionally, when the floating window is created on the upper layer of the rendering of the vehicle's surrounding environment, the floating window can be displayed on the left or right side of the vehicle object in the rendering of the vehicle's surrounding environment.

[0120] For example, the display position of the floating window can be determined according to the position of the risk object relative to the vehicle within the future preset time. First, the target position is determined according to the position of the risk object relative to the vehicle within the future preset time, and then the floating window is displayed at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment. For example, the position of the risk object relative to the vehicle within the future preset time can be determined based on the predicted motion trajectory of the risk object within the future preset time and the predicted driving trajectory of the vehicle. If the risk object will be located on the left side of the vehicle within the future preset time, the floating window will be displayed on the left side of the vehicle object in the rendering of the vehicle's surrounding environment. If the risk object will be located on the right side of the vehicle within the future preset time, the floating window will be displayed on the right side of the vehicle object in the rendering of the vehicle's surrounding environment. The display position of the floating window is the same as the actual position of the risk object relative to the vehicle, which helps to form the user's cognition of judging the direction of the risk object through the floating window, and facilitates the user to make driving decisions.

[0121] It should be noted that the information obtained by the vehicle is updated in real time. As this information is updated in real time, the environmental feature model will also be updated in real time, and the rendering of the vehicle's surrounding environment will also be updated in real time accordingly.

[0122] 203. Determine a target image of the area where the risk object is located according to the location of the risk object, the target image includes a rendering or a real scene of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range.

[0123] The target image is the image to be displayed in the floating window, and the display area of ​​the target image is the same as the display area of ​​the floating window. That is, the target image can be fully displayed in the floating window and completely occupies the floating window. After the target image is displayed in the floating window, the proportion of the risk object in the floating window is equal to the proportion of the floating window in the target image.

[0124] When the proportion of risk objects in the target image is within a preset ratio interval, the risk objects can be displayed completely and clearly, with a good display effect. For example, the preset ratio interval is [M1, M2], M1 can be 18%, 19%, 20%, 21% or 22%, etc., and M2 can be 38%, 39%, 40%, 41% or 42%, etc., which is not limited in the embodiments of the present application. The proportion of risk objects in the target image can be regarded as the ratio of the display area of ​​the closed area formed by the closed edge of the risk object in the target image to the display area of ​​the target image. Or it can be regarded as the ratio of the number of pixels in the closed area formed by the closed edge of the risk object in the target image to the total number of pixels included in the target image.

[0125] The vehicle (e.g., the vehicle system) can determine the location of the risk object based on the information obtained by the vehicle (e.g., point cloud data of the laser radar / image data of the image sensor / feedback data of the millimeter wave radar, etc.). The location of the risk object can be the physical location of the risk object relative to the vehicle at the current moment.

[0126] In a first implementation, the target image may be acquired based on a rendering of the vehicle's surroundings or a real-life image showing a risky object. When the target image is acquired based on a rendering of the vehicle's surroundings, the target image includes a rendering of the area where the risky object is located; when the target image is acquired based on a real-life image showing a risky object, the target image includes a real-life image of the area where the risky object is located.

[0127] In the first implementation, in an exemplary manner, the vehicle computer determines the position of the risk object in the initial image according to the position of the risk object. The target magnification is determined, and the initial image is magnified according to the target magnification. Then, based on the position of the risk object in the initial image and the enlarged initial image, the target image is obtained. The initial image includes a rendering of the vehicle's surrounding environment or a real-life image showing the risk object. The vehicle computer can crop the enlarged initial image based on the position of the risk object in the initial image to obtain the target image.

[0128] When the initial image includes a rendering of the vehicle's surroundings, the rendering of the vehicle's surroundings may be obtained first, and then the position of the risk object in the rendering of the vehicle's surroundings may be synchronously located according to the position of the risk object. The vehicle may be located in the rendering of the vehicle's surroundings according to the position of the vehicle and the information obtained by the vehicle.

[0129] When the initial image includes a real-life image showing a risky object, the real-life image showing the risky object may be obtained first, and then the position of the risky object may be located from the real-life image showing the risky object according to the position of the risky object and the image recognition process. The real-life image showing the risky object may be an image captured by an image sensor in the vehicle, and the image sensor includes but is not limited to: a camera close to one side of the risky object, a camera with a greater depth of field, and a macro camera.

[0130] For example, when the vehicle identifies the risk object as a curb when making a sharp turn, the image showing the risk object captured by the camera installed at the rear of the vehicle can be called as the initial image. For another example, the image showing the risk object captured by the corresponding camera can be called as the initial image according to the relative position of the risk object and the vehicle. For example, the image showing the risk object captured by the camera installed in the target direction of the vehicle can be called as the initial image, and the target direction is the direction of the risk object relative to the vehicle. Assuming that the risk object is on the left side of the vehicle, the image showing the risk object captured by the camera installed on the left side of the vehicle can be called as the initial image; assuming that the risk object is on the right side of the vehicle, the image showing the risk object captured by the camera installed on the right side of the vehicle can be called as the initial image.

[0131] In an implementation method for determining a target magnification, the target magnification can be determined according to the distance between the risk object and the vehicle and a preset mapping relationship. The preset mapping relationship includes a mapping relationship between distance and magnification, and the greater the distance in the preset mapping relationship, the greater the magnification. A maximum magnification can be preset, and if the magnification of the distance mapping between the risk object and the vehicle is greater than the maximum magnification, the maximum magnification is determined as the target magnification. If the magnification of the distance mapping between the risk object and the vehicle is less than or equal to the maximum magnification, the mapped magnification is determined as the target magnification. The greater the distance between the risk object and the vehicle, the smaller the proportion of the risk object in the initial image, the greater the mapped magnification, and the maximum magnification is preset. Therefore, the target magnification determined by the preset mapping relationship can effectively ensure that the proportion of the risk object in the target image is within a preset ratio range.

[0132] For example, the vehicle can determine the distance between the risk object and the vehicle based on the acquired information. The preset mapping relationship can be a function N(X) of the magnification N with respect to the distance X. N(X) can be a continuous function, and as the distance X changes continuously, the magnification N also changes in real time. After N reaches the maximum magnification Nmax, when the distance increases again, the magnification N remains unchanged at the maximum magnification Nmax.

[0133] N(X) can also be a piecewise function, and multiple magnifications are pre-defined in N(X). Multiple magnifications correspond to distance intervals, and the distance intervals corresponding to different magnifications may not overlap. When the distance X between the risk object and the vehicle is in different distance intervals, different magnifications N correspond. For example, magnifications N1, N2, N3 and N4 are pre-defined in N(X), and N1≥N2≥N3≥N4. The distance interval corresponding to N1 is [500 meters, +∞], the distance interval corresponding to N2 is [300 meters, 500 meters), the distance interval corresponding to N3 is [100 meters, 300 meters), and the distance interval corresponding to N4 is (0, 100 meters).

[0134] In another implementation method of determining the target magnification ratio, the target magnification ratio range can be determined according to the proportion of the risk object in the initial image, the display area of ​​the floating window, and the preset ratio range, and then the target magnification ratio is determined according to the target magnification ratio range. Among them, when the risk object in the initial image magnified according to the magnification within the target magnification ratio range is displayed in the floating window, the proportion in the floating window is within the preset ratio range. That is, on the same display screen, the ratio of the display area of ​​the risk object in the initial image magnified according to the magnification within the target magnification ratio range to the display area of ​​the floating window is within the preset ratio range.

[0135] The vehicle determines the first magnification ratio required when the proportion of the risk object in the initial image magnified according to the target magnification range in the floating window reaches M1, and determines the second magnification ratio required when the proportion of the risk object in the initial image magnified according to the target magnification range in the floating window reaches M2. Then, the interval composed of the first magnification ratio and the second magnification ratio is determined as the target magnification ratio interval.

[0136] When determining the target magnification based on the target magnification interval, in one example, any magnification value can be taken from the target magnification interval as the target magnification. In another example, the target magnification can be determined in combination with the aforementioned preset mapping relationship. If N(X) is a continuous function, the target magnification interval can be used as the value range of N(X), and then the target magnification can be determined in real time based on the distance between the risk object and the vehicle and N(X). If N(X) is a piecewise function, multiple magnifications can be determined from the target magnification interval as piecewise function values ​​(i.e., the aforementioned multiple pre-defined magnifications), and then the corresponding magnification is determined for the distance interval where the risk object and the vehicle are located, and determined as the target magnification. The specific implementation process of these two examples can refer to the aforementioned first implementation method, and the embodiments of the present application will not be repeated here.

[0137] When the initial image is enlarged according to the target magnification, the initial image may be enlarged in equal proportion according to the target magnification to obtain the target image.

[0138] In another example, the vehicle computer determines a partial image from the initial image according to the position of the risk object. Then, the target magnification is determined. The partial image is enlarged according to the target magnification to obtain the target image. The partial image is an image of the area where the risk object is located in the initial image. The process of determining the target magnification and enlarging the initial image can refer to the above example, and the embodiment of the present application will not be described in detail here.

[0139] The first implementation method can directly use a rendering or a real scene of the vehicle's surrounding environment, so that the vehicle consumes less computing resources in the process of determining the target image.

[0140] In the second implementation, the target image can be determined based on the area where the risk object is located in the environmental feature model of the vehicle, and the target image includes a rendering of the area where the risk object is located. The vehicle computer obtains the environmental feature model of the vehicle, and then determines the target rendering perspective based on the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window. Then, the environmental feature model is rendered according to the target rendering perspective to obtain the target image.

[0141] The environmental feature model can be obtained by modeling environmental information. The environmental information includes at least one of the following: point cloud data of a laser radar, image data of an image sensor, and feedback data of a millimeter wave radar.

[0142] Since the environmental feature model can be a 3D model, the change of the rendering perspective is supported. The rendering process can be regarded as simulating a virtual camera, and the environmental feature model is presented through the imaging perspective of the virtual camera. The imaging perspective of the virtual camera is the rendering perspective. The target rendering perspective can be the same as or different from the preset rendering perspective when creating the rendering image of the vehicle's surrounding environment, and the embodiments of the present application do not limit this.

[0143] Referring to the rendering process of the rendering of the vehicle's surrounding environment, a new virtual camera can be determined according to the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window. The imaging angle of the new virtual camera is the target rendering angle. Then, the environmental feature model is presented through the virtual camera to obtain the target image. The position of the new virtual camera can be the same as or different from the position of the virtual camera corresponding to the rendering of the vehicle's surrounding environment.

[0144] For example, the preset distance can be determined based on the actual size of the risk object and the display area of ​​the floating window. When the distance between the virtual camera and the risk object in the environmental feature model is less than the preset distance, the risk object in the rendered image is displayed in the floating window, and its proportion in the floating window is greater than or equal to the minimum value in the preset ratio interval. Then, a target distance is determined within a range less than the preset distance, and a new virtual camera is determined based on the position of the risk object and the target distance. A mapping relationship between the position of the risk object and the orientation of the virtual camera can be preset in the vehicle computer, and the vehicle computer determines the orientation corresponding to the position of the risk object in the mapping relationship as the target orientation. The new virtual camera is located at the target orientation and the distance from the risk object is the target distance. The presentation angle of the new virtual camera is the target rendering angle.

[0145] For example, assuming the preset distance is 5 meters, the target distance can be determined to be 4 meters. The risk object is located in front of the vehicle, and the target orientation can be determined to be the orientation of 45 degrees in front of the vehicle object in the environmental feature model. The target rendering perspective is: 45 degrees in front of the vehicle object in the environmental feature model, and the distance to the risk object in the environmental feature model is 4 meters.

[0146] This implementation method determines that the target image obtained by the rendering perspective has a higher definition, and can display the risk object more clearly in the subsequent display, making it easier for users to observe and identify the risk object, grasp the information of the risk object, and make correct decisions. Please refer to Figure 5 , Figure 5 A schematic diagram of a rendering perspective of an object provided in an embodiment of the present application, Figure 5 Three objects A, B and C are shown, and the virtual camera is represented by an eye. If the virtual camera is in position 1, the rendering perspectives of the three objects A, B and C are a, b and c respectively. Figure 5 It can be seen that under the perspective of the virtual camera, the observation effect of object A is the best, object B is second, and the observation effect of object C is poor.

[0147] by Figure 5 For example, assuming that object A represents a vehicle, object C represents a risk object identified in process 201, and position 1 represents a preset rendering perspective. When rendering the environmental feature model of vehicle A, if the rendering is performed according to the preset rendering perspective, the rendering perspective of the rendering image of the area where object C is located is c. The rendering effect of the risk object in the rendered image obtained in this way is poor. After the rendering image is displayed in the floating window, it may not be possible for the user to observe object C clearly and intuitively.

[0148] In the second implementation, when rendering the environmental feature model of vehicle A, position 2 can be determined as the position of the virtual camera, that is, perspective d can be determined as the target rendering perspective. Perspective d has a better rendering effect on object C than perspective c, thereby making the presentation effect of object C better. After the rendering image is displayed in the floating window, the user can observe object C clearly and intuitively.

[0149] In the third implementation, the target image is obtained based on the target image sensor after adjusting the parameters, and the target image includes a real-life image of the area where the risk object is located. The vehicle computer determines the target image sensor and adjusts the camera parameters of the target image sensor according to the location of the risk object. Then, the image captured by the target image sensor according to the adjusted camera parameters is determined as the target image.

[0150] In process 201, the vehicle computer can identify risky objects around the vehicle through images. In process 203, the image sensor corresponding to the real-life image showing the risky object can be determined as the target image sensor. Alternatively, the target image sensor can be determined based on the location of the risky object to better present the risky object. The target image sensor includes, but is not limited to: a camera close to one side of the risky object, a camera with a greater depth of field, and a macro camera.

[0151] For example, when the vehicle recognizes that the risk object is a curb when making a sharp turn, the target image sensor may be a camera installed at the rear of the vehicle. For another example, the image captured by the camera installed in the target direction of the vehicle may be called, and the target direction is the direction of the risk object relative to the vehicle. If the risk object is on the left side of the vehicle, the image captured by the camera installed on the left side of the vehicle may be called; if the risk object is on the right side of the vehicle, the image captured by the camera installed on the right side of the vehicle may be called.

[0152] When adjusting the parameters of the target image sensor, the focal length of the target image sensor can be adjusted according to the position of the risk object to ensure that the risk object can be completely and clearly presented in the image captured by the target image sensor. When adjusting the focal length of the target image sensor, the position of the risk object in the real-life image previously displayed with the risk object can also be referred to based on the position of the risk object. For example, the mapping relationship between the position of the risk object (or the position of the risk object and the position of the risk object in the real-life image previously displayed with the risk object) and the position of the image sensor and the focal length can be preset in the vehicle computer. The vehicle computer determines the position of the identified risk object (or the position of the risk object and the position of the risk object in the real-life image previously displayed with the risk object) and the position of the target image sensor, and the corresponding focal length in the mapping relationship. The focal length of the target image sensor is then adjusted to the aforementioned corresponding focal length.

[0153] The third implementation can be combined with the first implementation. For example, after acquiring the image captured by the target image sensor according to the adjusted camera parameters, the target magnification can be determined, and the acquired image can be magnified according to the target magnification to obtain the target image. In this way, when the image captured by the target image sensor is not clear enough, the image can be magnified to improve the presentation effect of the target image.

[0154] When the distance between the vehicle and the identified risk object is close, the real-view image has a clearer display effect for the risk object. When the distance between the vehicle and the identified risk object is far, the rendering has a clearer display effect for the risk object. In an embodiment of the present application, when the distance between the risk object and the vehicle is less than the preset distance, the target image includes a real-view image of the area where the risk object is located, that is, the target image can be determined according to the first implementation method mentioned above, and the initial image is a real-view image showing the risk object, or the target image can be determined according to the third implementation method mentioned above. When the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendering of the area where the risk object is located, that is, the target image can be determined according to the first implementation method mentioned above, and the initial image is a rendering of the surrounding environment of the vehicle, or the target image can be determined according to the second implementation method mentioned above. The preset distance can be, for example, 8 meters, 9 meters, 10 meters, 11 meters or 12 meters, etc., and the embodiment of the present application does not limit this.

[0155] For example, suppose the vehicle is turning and the identified risk object is the edge of the road. The distance between the vehicle and the edge of the road is less than the preset distance, and the target image includes a real-life image of the area where the risk object is located. Suppose the identified risk object is a plastic bag in the distance, and the distance between the vehicle and the plastic bag is greater than the preset distance, and the target image includes a rendering of the area where the risk object is located.

[0156] 204. Display the target image in the floating window.

[0157] Optionally, the target image can be displayed in the center of the floating window. For example, please refer to Figure 6 , Figure 6 A schematic diagram of displaying a target image in a floating window provided in an embodiment of the present application, Figure 6 Take, for example, a target image including a rendering of an area where a risk object is located, and assume that the risk object identified is a stone on the driveway. Figure 6 A rendering of the vehicle's surroundings and a floating window displayed on top of the rendering of the vehicle's surroundings are shown, and both the floating window and the risk object are located on the left side of the vehicle object. Figure 6 It can be seen that in the rendering of the vehicle's surrounding environment, the modeling A01 corresponding to the risk object (stone) is a small black dot. It is difficult for users to perceive the small black dot, and even if they perceive it, they cannot determine what the risk object is. Even if the modeling A01 is marked as a small blue dot according to the prior art, it is still difficult for users to perceive or determine what the risk object is. In an embodiment of the present application, a target image G is displayed in a floating window. Compared with the rendering of the vehicle's surrounding environment, the target image G enlarges the area where the stone is located, and the user can clearly observe the stone and the location of the stone through the target image G.

[0158] In an embodiment of the present application, the risk warning process can be executed cyclically, and the risk warning process includes: identifying risk objects around the vehicle, and updating the target image of the area where the risk objects are located according to the latest position of the risk objects. The updated target image is then displayed in a floating window. This process can refer to the aforementioned processes 201, 203, and 204, and the embodiments of the present application are not described in detail here. When it is identified that the risk object no longer poses a risk to driving, the floating window corresponding to the risk object can be stopped from being displayed, and the subsequent cycle of the risk warning process can be stopped.

[0159] In this way, the target image displayed in the floating window changes in real time. For example, when the vehicle is in a driving state, it is always moving, so the position of the risk object identified in process 201 (referring to the position relative to the vehicle) is also always changing, even if the risk object itself is dynamically moving, its position will also change. The vehicle can update the target image in the floating window in real time. When the position of the risk object changes, the magnification and viewing angle of the target image in the floating window compared to the initial image will also change accordingly, so that the identified risk object is always displayed in the floating window.

[0160] In summary, the driving risk prompting method provided by the embodiment of the present application, when a risk object around the vehicle is identified, a floating window is displayed on the display screen of the vehicle, and then the target image of the area where the risk object is located is determined according to the position of the risk object, and the target image is displayed in the floating window, the target image includes a rendering or a real picture of the area where the risk object is located, and the proportion of the risk object in the target image (equivalent to the proportion of the risk object in the floating window) is in a preset ratio range. On the one hand, the display of the floating window will play an obvious prompting role, which will make the user have the cognition that the appearance of the floating window is equivalent to the appearance of the risk object, thereby improving the user's risk awareness. On the other hand, the proportion of the risk object in the floating window is greater than the proportion of the risk object in the rendering of the vehicle's surrounding environment, so that the user can observe and identify the risk object more clearly through the floating window, grasp the information of the risk object, and make a correct decision. And even if the distance between the vehicle and the risk object is far, the user can observe and grasp the risk object in time through the floating window, providing the user with sufficient decision-making time, so that the user can make corresponding driving decisions for the risk object in time, compared with the relevant technology, effectively reducing the probability of dangerous situations caused by hasty decisions, and improving driving safety.

[0161] Please refer to Figure 7 , Figure 7 This is a flow chart of another driving risk prompting method provided in an embodiment of the present application. The method can be applied to a cockpit or a vehicle, for example Figure 1 and Figure 2 The vehicle shown in the figure may include the following process:

[0162] 301. Identify risk objects around the vehicle.

[0163] This process may refer to the aforementioned process 201, and the embodiment of the present application will not be described in detail here.

[0164] 302. Display a floating window on a display screen of the vehicle.

[0165] This process may refer to the aforementioned process 202, and the embodiment of the present application will not be described in detail here.

[0166] 303. Determine a target image of the area where the risk object is located according to the location of the risk object, where the target image includes a rendering or a real scene of the area where the risk object is located, and a proportion of the risk object in the target image is within a preset ratio range.

[0167] This process may refer to the aforementioned process 203, and the embodiment of the present application will not be described in detail here.

[0168] 304. Display the target image in the floating window.

[0169] This process may refer to the aforementioned process 204, and the embodiment of the present application will not be described in detail here.

[0170] For a dynamic risk object, the following processes 305 and 306 may also be performed.

[0171] 305. Determine risk information, where the risk information includes at least one of the following: a predicted driving trajectory of the vehicle, a predicted motion trajectory of the risk object, a predicted collision point between the vehicle and the risk object, a distance between the predicted collision point and the vehicle, a position where the risk object is closest to the vehicle, and a risk level of the risk object.

[0172] As for the predicted driving trajectory of the vehicle, in one example, it can be obtained through the navigation system of the vehicle. If the vehicle uses the navigation system for route navigation, the future driving trajectory of the vehicle can be directly obtained from the navigation system. In another example, the predicted driving trajectory of the vehicle can be obtained by predicting the lane in which the vehicle is currently traveling. For example, when there is no intersection in front of the current driving lane of the vehicle, the front of the current driving lane is directly used as the predicted driving trajectory of the vehicle. When there is an intersection in front of the current driving lane of the vehicle, if the current driving lane is a straight lane, the front of the current driving lane is used as the predicted driving trajectory of the vehicle. If the current driving lane is a turning lane, the predicted driving trajectory of the vehicle is determined according to the intersection lane corresponding to the turning lane. In another example, the predicted driving trajectory of the vehicle can also be obtained based on the historical driving trajectory of the vehicle, the information obtained by the vehicle, and the driver's behavior and other information.

[0173] The predicted motion trajectory of the risk object can be obtained by the spatial position of the risk object in the historical time period. For example, based on the spatial position of the risk object in the historical time period, the motion trajectory of the risk object is predicted in combination with inertia. The spatial position of the risk object in the historical time period can be obtained by the information obtained by the vehicle.

[0174] Optionally, the risk information may include the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object. By displaying the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object, the user can directly identify the collision risk of the risk object and take avoidance measures in time.

[0175] Furthermore, it is also possible to determine whether there is a spatial intersection between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object, thereby determining whether the vehicle and the risk object will collide. If there is a spatial intersection, the vehicle and the risk object will collide; if there is no spatial intersection, the vehicle and the risk object will not collide.

[0176] If the vehicle and the risk object will not collide, but the vehicle and the risk object will come into close contact in the future, the risk information may also include the position where the risk object is closest to the vehicle, so as to remind the user to drive carefully when driving to the position to avoid collision with the risk object.

[0177] If a collision occurs, the risk information may also include the predicted collision point between the vehicle and the risk object, which is the spatial intersection of the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object. Further, the risk information may also include the distance between the predicted collision point and the vehicle.

[0178] For the risk level of a risk object, the greater the minimum distance between the risk object and the vehicle, the lower the risk level of the risk object. For example, if the risk object and the vehicle will directly collide, the risk level of the risk object is level one (the highest level). If the minimum distance between the risk object and the vehicle is greater than 0 and less than 1 meter, the risk level of the risk object is level two. If the minimum distance between the risk object and the vehicle is greater than 1 meter, the risk level of the risk object is level three. The specific division of risk levels is not limited in the embodiments of the present application.

[0179] 306. Display risk information in a floating window.

[0180] When the risk information includes the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object, if the target image is a rendering of the area where the risk object is located, the environmental feature model can be directly called to generate and display the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object in the floating window. If the target image is a real-life image of the area where the risk object is located, a user interface (UI) layer can be added to the floating window to display the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object.

[0181] When the risk information includes the predicted collision point between the vehicle and the risky object, the distance between the predicted collision point and the vehicle, or the position where the risky object is closest to the vehicle, the predicted collision point identifier, the distance between the predicted collision point and the vehicle, or the position where the risky object is closest to the vehicle can be superimposed and displayed in the floating window in the form of a UI layer or UI control.

[0182] When the risk information includes the risk level of the risk object, different types of colors or the same color at different depths can be displayed on the edge of the floating window to indicate different risk levels. Alternatively, a risk icon (such as an exclamation mark) can be displayed in the floating window, and different types of color shading can be displayed in the floating window to indicate different risk levels. The embodiment of the present application does not limit the form of displaying the risk level. Taking the risk level of the aforementioned process 305 as an example, when the risk level is level one, a dark red light band can be displayed on the edge of the floating window. When the risk level is level two, a yellow light band can be displayed on the edge of the floating window. When the risk level is level three, a green light band can be displayed on the edge of the floating window.

[0183] 307. Obtain the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object.

[0184] This process may refer to the aforementioned process 305, and the embodiment of the present application will not be described in detail here.

[0185] 308. Generate a driving animation of the vehicle according to the predicted driving trajectory.

[0186] The vehicle can simulate the movement of the vehicle based on the predicted driving trajectory of the vehicle, and then obtain the driving animation of the vehicle.

[0187] 309. Generate a motion animation of the risk object based on the predicted motion trajectory.

[0188] The vehicle can simulate the motion of the object based on the predicted motion trajectory of the risk object, and then obtain the motion animation of the risk object.

[0189] 310. Display the driving animation of the vehicle and the movement animation of the risk object in the floating window.

[0190] If the target image is a rendering of the area where the risk object is located, the environmental feature model can be directly called to simulate the movement of the vehicle and the risk object, and the vehicle's driving animation and the risk object's movement animation can be rendered and displayed in the floating window. If the target image is a real-life image of the area where the risk object is located, a UI layer can be added to the floating window to display the vehicle and risk object logos, and the vehicle and risk object logos move on their respective predicted trajectories, thereby displaying the vehicle's driving animation and the risk object's movement animation.

[0191] When a vehicle and a risk object are about to collide in the future, the vehicle's driving animation and the risk object's motion animation form a collision animation. Compared with simply presenting static information, the presentation of animation can more directly indicate the risk of collision.

[0192] For example, please refer to Figure 8 , Figure 8 A schematic diagram of a floating window display provided in an embodiment of the present application is shown. Figure 8 The risk information includes the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, and the predicted collision point between the risk object and the vehicle as an example for explanation. Figure 8 The risk object identified in the figure is a motorcycle, and both the motorcycle and the floating window are located on the right side of the vehicle object. Figure 8 The predicted driving trajectory B01 of the vehicle, the predicted motion trajectory B02 of the motorcycle, and the predicted collision point B03 between the vehicle and the motorcycle are shown. Figure 8 A warning sign is also displayed at the predicted collision point B03. Figure 8 The floating window can show users the locations where there is a risk of collision.

[0193] Optionally, the above is explained by taking the floating window prompting driving risks as an example. On the basis of the aforementioned floating window, risk objects can also be prompted by voice and head up display (HUD) devices. HUD, also known as the head-up display system, refers to a driver-centric blind-operated, multi-function instrument panel. HUD is a projection on the front windshield of the vehicle, which can display some information, such as current speed and navigation information. In this way, the driver can see the driving information from the projection without lowering or turning his head, which reduces the probability of safety accidents to a certain extent.

[0194] Taking HUD as an example, after determining the direction of the risk object relative to the vehicle, a risk icon can be displayed on the HUD. The risk icon is used to identify the direction of the risk object relative to the lane the vehicle is currently in. For example, please refer to Fig. 9 , Fig. 9 A schematic diagram of a HUD provided in an embodiment of the present application, Fig. 9The black dot displayed on the HUD is a risk icon, indicating that the risk object is on the left side of the vehicle's current lane.

[0195] It should be noted that when the distance between the risk object identified in process 301 and the vehicle is 0, there is no need to perform subsequent processes 302 to 308. When the distance between the risk object identified in process 301 and the vehicle is greater than 0, subsequent processes 302 to 308 may be performed.

[0196] The aforementioned processes 301 to 310 may be executed cyclically. During the driving process of the vehicle, risk objects around the vehicle may be continuously identified. And each time a risk object is identified and the distance between the risk object and the vehicle is greater than 0, the aforementioned processes 302 to 310 are executed.

[0197] In summary, the driving risk prompting method provided by the embodiment of the present application, when a risk object around the vehicle is identified, a floating window is displayed on the display screen of the vehicle, and then the target image of the area where the risk object is located is determined according to the position of the risk object, and the target image is displayed in the floating window, the target image includes a rendering or a real picture of the area where the risk object is located, and the proportion of the risk object in the target image (equivalent to the proportion of the risk object in the floating window) is in a preset ratio range. On the one hand, the display of the floating window will play an obvious prompting role, which will make the user have the cognition that the appearance of the floating window is equivalent to the appearance of the risk object, thereby improving the user's risk awareness. On the other hand, the proportion of the risk object in the floating window is greater than the proportion of the risk object in the rendering of the vehicle's surrounding environment, so that the user can observe and identify the risk object more clearly through the floating window, grasp the information of the risk object, and make a correct decision. And even if the distance between the vehicle and the risk object is far, the user can observe and grasp the risk object in time through the floating window, providing the user with sufficient decision-making time, so that the user can make corresponding driving decisions for the risk object in time, compared with the relevant technology, effectively reducing the probability of dangerous situations caused by hasty decisions, and improving driving safety.

[0198] It can also determine risk information and display it in a floating window. The risk information includes at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object. By displaying the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object, the user can directly identify the collision risk of the risk object and take avoidance measures in a timely manner.

[0199] In addition, the vehicle driving animation can be generated based on the predicted driving trajectory of the vehicle, and the motion animation of the risk object can be generated based on the predicted motion trajectory, and the vehicle driving animation and the motion animation of the risk object can be displayed in the floating window. By displaying the collision animation of the vehicle and the risk object, compared with only presenting static information, the presentation of the collision animation can more directly indicate the collision risk.

[0200] The sequence of the methods provided in the embodiments of the present application can be adjusted appropriately, and the process can be increased or decreased accordingly according to the situation. Any technician familiar with the technical field can easily think of a method of change within the technical scope disclosed in this application, which should be included in the protection scope of this application, and the embodiments of the present application do not limit this.

[0201] The above mainly introduces the driving risk prompt method provided by the embodiment of the present application from the perspective of the device. It can be understood that in order to realize the above functions, the device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0202] The embodiment of the present application can divide the functional modules of the device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0203] Fig.10 A block diagram of a driving risk reminder device provided in an embodiment of the present application, in the case of dividing each functional module according to each function, the driving risk reminder device 400 may include an identification module 401, a display module 402, and an image determination module 403. Exemplarily, the driving risk reminder device may be a vehicle or a vehicle system, or a chip therein or other combination devices, components, etc. having the functions of the above-mentioned driving risk reminder device. The functions of each module of the driving risk reminder device are as follows:

[0204] Identification module 401, used to identify risk objects around the vehicle;

[0205] A display module 402, used to display a floating window on a display screen of the vehicle;

[0206] An image determination module 403 is used to determine a target image of the area where the risk object is located according to the location of the risk object, wherein the target image includes a rendering or a real image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range;

[0207] The display module 402 is also used to display the target image in the floating window.

[0208] In combination with the above scheme, the image determination module 403 is specifically used to: determine the position of the risk object in the initial image according to the position of the risk object, the initial image including a rendering of the vehicle's surrounding environment or a real-life image showing the risk object; determine the target magnification; enlarge the initial image according to the target magnification; and obtain the target image based on the position of the risk object in the initial image and the enlarged initial image.

[0209] In combination with the above scheme, the image determination module 403 is specifically used to determine the target magnification according to the distance between the risk object and the vehicle and a preset mapping relationship. The preset mapping relationship includes a mapping relationship between distance and magnification. In the preset mapping relationship, the greater the distance, the greater the magnification.

[0210] In combination with the above scheme, the image determination module 403 is specifically used to: determine the target magnification range according to the proportion of the risk object in the initial image, the display area of ​​the floating window and the preset ratio range, so that when the risk object in the initial image that is magnified according to the magnification within the target magnification range is displayed in the floating window, the proportion in the floating window is within the preset ratio range; determine the target magnification ratio according to the target magnification range.

[0211] In combination with the above scheme, the target image includes a rendering of the area where the risk object is located, and the image determination module is specifically used to: obtain the environmental feature model of the vehicle; determine the target rendering perspective according to the position of the risk object, the actual size of the risk object and the display area of ​​the floating window; render the environmental feature model according to the target rendering perspective to obtain the target image.

[0212] In combination with the above scheme, the target image includes a real-life image of the area where the risk object is located, and the image determination module 403 is specifically used to: determine the target image sensor and adjust the camera parameters of the target image sensor according to the location of the risk object; and determine the image captured by the target image sensor according to the adjusted camera parameters as the target image.

[0213] In combination with the above scheme, when the distance between the risk object and the vehicle is less than the preset distance, the target image includes a real picture of the area where the risk object is located; when the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendering of the area where the risk object is located.

[0214] Fig.11 A block diagram of another driving risk reminder device provided in an embodiment of the present application, Fig.10 On the basis of, the device also includes: a risk information determination module 404, used to determine risk information, the risk information includes at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object; the aforementioned display module 402 is also used to display the risk information in a floating window.

[0215] In combination with the above scheme, the risk information determination module 404 is also used to: obtain the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object; generate a driving animation of the vehicle based on the predicted driving trajectory; generate a motion animation of the risk object based on the predicted motion trajectory; the aforementioned display module 402 is also used to display the driving animation of the vehicle and the motion animation of the risk object in a floating window.

[0216] In combination with the above solution, the display module 402 is specifically used to: determine the target position of the risk object relative to the vehicle within a preset time in the future; and display a floating window at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

[0217] In combination with the above scheme, the identification module 401 is also used to identify risk objects around the vehicle when the risk warning process is cyclically executed; the image determination module 403 is also used to update the target image of the area where the risk object is located according to the latest position of the risk object when the risk warning process is cyclically executed; the display module 402 is also used to display the updated target image in the floating window when the risk warning process is cyclically executed.

[0218] Fig.12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 500 may be a vehicle or a chip or a functional module in a vehicle, or may be a vehicle system or a chip or a functional module in a vehicle system. Fig.12 As shown, the electronic device 500 includes a processor 501 , a transceiver 502 and a communication circuit 503 .

[0219] The processor 501 is used to execute the following Figure 3 and Figure 7In any step of the method embodiment shown, when executing a process such as obtaining environmental information, the transceiver 502 and the communication line 503 may be selectively called to complete the corresponding operation.

[0220] Furthermore, the electronic device 500 may further include a memory 504. The processor 501, the memory 504 and the transceiver 502 may be connected via a communication line 503.

[0221] The transceiver 502 is used to communicate with other devices or other communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The transceiver 502 can be a module, a circuit, a transceiver or any device capable of achieving communication.

[0222] The transceiver 502 is mainly used for sending and receiving information, etc., and may include a transmitter and a receiver, which respectively send and receive information, etc.; operations other than sending and receiving information, etc. are implemented by the processor, such as determining the target image of the area where the risk object is located based on the position of the risk object.

[0223] The communication line 503 is used to transmit information between the components included in the electronic device 500.

[0224] In one design, the processor may be considered as the logic circuit and the transceiver may be considered as the interface circuit.

[0225] The memory 504 is used to store instructions, where the instructions may be computer programs.

[0226] It should be noted that the memory 504 can exist independently of the processor 501, or can be integrated with the processor 501. The memory 504 can be used to store instructions or program codes or some data, etc. The memory 504 can be located inside the electronic device 500, or outside the electronic device 500, without limitation. The processor 501 is used to execute the instructions stored in the memory 504 to implement the method provided in the above embodiment of the present application.

[0227] In one example, the processor 501 may include one or more processors, such as Fig.12 Processor 0 and processor 1 in.

[0228] As an optional implementation, the electronic device 500 includes multiple processors, for example, Fig.12 In addition to the processor 501, a processor 507 may also be included.

[0229] As an optional implementation, the electronic device 500 further includes an output device 505 and an input device 506. Exemplarily, the input device 506 is a device such as a keyboard, a mouse, a microphone or a joystick, and the output device 505 is a device such as a display screen and a speaker.

[0230] It should be noted that the electronic device 500 may be a chip system or a Fig.12 Devices with similar structures in the chip system. Among them, the chip system can be composed of chips, or it can include chips and other discrete devices. The actions, terms, etc. involved in the various embodiments of this application can refer to each other without limitation. The message name or parameter name in the message exchanged between the various devices in the embodiment of this application is only an example, and other names can also be used in the specific implementation without limitation. In addition, Fig.12 The components shown in the figure do not constitute a limitation on the electronic device 500. Fig.12 In addition to the components shown, the electronic device 500 may include Fig.12 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0231] The processor and transceiver described in the present application can be implemented in an integrated circuit (IC), an analog IC, a radio frequency integrated circuit, a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductor (CMOS), N-type metal oxide semiconductor (NMOS), P-type metal oxide semiconductor (positive channel metal oxide semiconductor, PMOS), bipolar junction transistor (BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0232] Fig.13 This is a schematic diagram of the structure of a driving risk reminder device provided in an embodiment of the present application. The driving risk reminder device can be applied to the scenario shown in the above method embodiment. For the convenience of explanation, Fig.13Only the main components of the driving risk reminder device are shown, including a processor, a memory, a control circuit, and an input-output device. The processor is mainly used to process the communication protocol and communication data, execute the software program, and process the data of the software program. The memory is mainly used to store the software program and data. The control circuit is mainly used for power supply and transmission of various electrical signals. The input-output device is mainly used to receive data input by the user and output data to the user.

[0233] When the driving risk reminder device is a vehicle or a vehicle computer, the control circuit can be a mainboard, the memory includes a hard disk, RAM, ROM and other media with storage functions, the processor can include a baseband processor and a central processing unit, the baseband processor is mainly used to process the communication protocol and communication data, the central processing unit is mainly used to control the entire driving risk reminder device, execute software programs, and process the data of the software programs, and the input and output devices include a display screen, a keyboard and a mouse, etc.; the control circuit can further include or be connected to a transceiver circuit or a transceiver, such as: a network cable interface, etc., for sending or receiving data or signals, such as data transmission and communication with other devices. Furthermore, it can also include an antenna for sending and receiving wireless signals, and for data / signal transmission with other devices.

[0234] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes any of the methods described in the embodiments of the present application.

[0235] The embodiment of the present application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by a computer or a device with driving risk prompting capability to control the relevant hardware to complete, and the computer program or the group of instructions can be stored in the above computer-readable storage medium. When the computer program or the group of instructions is executed, it may include the processes of the above method embodiments. The computer-readable storage medium can be an internal storage unit of the vehicle or vehicle system of any of the above embodiments, such as a hard disk or memory of the vehicle or vehicle system. The above computer-readable storage medium can also be an external storage device of the above vehicle or vehicle system, such as a plug-in hard disk equipped on the above vehicle or vehicle system, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. Further, the above computer-readable storage medium can also include both the internal storage unit of the above vehicle or vehicle system and an external storage device. The above computer-readable storage medium is used to store the above computer program or instruction and other programs and data required by the above vehicle or vehicle system. The above computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0236] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0237] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0238] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0239] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0240] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0241] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.

[0242] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for prompting driving risks, characterized in that: The method comprises: Identify risk objects around the vehicle; Displaying a floating window on a display screen of the vehicle; Determine, according to the location of the risk object, a target image of the area where the risk object is located, wherein the target image includes a rendering or a real image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; The target image is displayed in the floating window.

2. The method according to claim 1, characterized in that The step of determining a target image of an area where the risk object is located according to the position of the risk object includes: Determining the position of the risk object in an initial image according to the position of the risk object, wherein the initial image includes a rendering of the surrounding environment of the vehicle or a real scene image showing the risk object; Determine the target magnification; Enlarging the initial image according to the target magnification; The target image is obtained based on the position of the risk object in the initial image and the enlarged initial image.

3. The method according to claim 2, characterized in that Determining the target magnification includes: The target magnification is determined according to the distance between the risk object and the vehicle and a preset mapping relationship, wherein the preset mapping relationship includes a mapping relationship between distance and magnification, and the larger the distance in the preset mapping relationship, the larger the magnification.

4. The method according to claim 2, characterized in that: Determining the target magnification includes: Determine a target magnification range according to the proportion of the risk object in the initial image, the display area of ​​the floating window, and the preset ratio range, so that when the risk object in the initial image magnified according to the magnification within the target magnification range is displayed in the floating window, the proportion in the floating window is within the preset ratio range; The target magnification is determined according to the target magnification range.

5. The method according to claim 1, characterized in that The target image includes a rendering of the area where the risk object is located, and determining the target image of the area where the risk object is located according to the position of the risk object includes: Acquire an environmental feature model of the vehicle; Determining a target rendering viewing angle according to the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window; The environmental feature model is rendered according to the target rendering perspective to obtain the target image.

6. The method according to claim 1, characterized in that The target image includes a real picture of the area where the risk object is located, and determining the target image of the area where the risk object is located according to the position of the risk object includes: According to the position of the risk object, determining a target image sensor and adjusting the imaging parameters of the target image sensor; An image captured by the target image sensor according to the adjusted imaging parameters is determined as the target image.

7. The method according to any one of claims 1 to 6, characterized in that: When the distance between the risk object and the vehicle is less than a preset distance, the target image includes a real picture of the area where the risk object is located; when the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendering of the area where the risk object is located.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Determine risk information, the risk information including at least one of the following: a predicted driving trajectory of the vehicle, a predicted motion trajectory of the risk object, a predicted collision point between the vehicle and the risk object, a distance between the predicted collision point and the vehicle, a position at which the risk object is closest to the vehicle, and a risk level of the risk object; The risk information is displayed in the floating window.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Acquiring a predicted driving trajectory of the vehicle and a predicted motion trajectory of the risk object; generating a driving animation of the vehicle according to the predicted driving trajectory; generating a motion animation of the risk object according to the predicted motion trajectory; The driving animation of the vehicle and the movement animation of the risk object are displayed in the floating window.

10. The method according to any one of claims 1 to 9, characterized in that: The display screen of the vehicle displays a rendering of the surrounding environment of the vehicle, and the floating window is displayed on the display screen of the vehicle, including: Determine the target position of the risk object relative to the vehicle within a preset time in the future; The floating window is displayed at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: The risk warning process is executed cyclically, and the risk warning process includes: identifying the risk objects around the vehicle; According to the latest position of the risk object, update the target image of the area where the risk object is located; The updated target image is displayed in the floating window.

12. A driving risk reminder device, characterized in that: The device comprises: Identification module, used to identify risk objects around the vehicle; A display module, used for displaying a floating window on a display screen of the vehicle; An image determination module, configured to determine, according to the position of the risk object, a target image of the area where the risk object is located, wherein the target image includes a rendering or a real image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; The display module is further used to display the target image in the floating window.

13. A driving risk reminder device, characterized in that: The device comprises: one or more processors; a memory for storing one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A vehicle, characterized in that: The vehicle comprises: A driving risk reminder device as claimed in claim 12 or 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and when the program codes are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images