Driving risk warning method and apparatus, and vehicle

Abstract

A driving risk warning method and apparatus and a vehicle, which are used for giving warnings about driving risks during the travelling of a vehicle so as to effectively reduce the probability of occurrence of dangerous situations caused by hasty decisions, thus improving driving safety. The driving risk warning method comprises: identifying a risky object around a vehicle (step 201); displaying a floating window on a display screen of the vehicle (step 202); according to the position of the risky object, determining a target image of a region where the risky object is located, the target image comprising a rendered image or real-scene image of the region where the risky object is located, and the proportion of the risky object in the target image being within a preset proportion range (step 203); and displaying the target image in the floating window (step 204).

Description

Driving risk prompting method and device, and vehicle

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 25, 2023, with application number 202310922673.1 and application name “Driving Risk Prompt Method, Device, and Vehicle”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of intelligent vehicle technology, and in particular to a driving risk prompting method and device, and a vehicle. Background Art

[0003] With the development of intelligent driving technology, the vehicle's vision and radar systems can more accurately identify dangerous objects around the vehicle and the distance between obstacles and the vehicle. These dangerous objects can be displayed through a rendering of the vehicle's surroundings. This can improve driving safety by alerting users to dangerous objects on the road (such as rolling tires, small plastic bags, bumps or rocks on the ground).

[0004] When rendering the vehicle's surroundings, objects are rendered based on their actual size. The actual size of the object matches the size in the rendered image, and objects closer to the vehicle appear larger in the rendered image. To highlight risky objects, related technologies typically render identified risky objects in a uniform blue color. This means that all risky objects appear blue in the rendered image.

[0005] However, related technologies use color to mark risky objects. For tiny, hard-to-see risky objects, these are displayed as small blue dots in the rendered image of the vehicle's surroundings, making them equally difficult for users to detect. This can hinder users from making timely decisions about risky objects and fails to effectively improve driving safety.

[0006] Summary of the Invention

[0007] The present application provides a driving risk prompting method, device, and vehicle, which solves the problem in related technologies that users have difficulty in detecting risky objects, can effectively reduce the probability of dangerous situations caused by hasty decisions, and improve driving safety.

[0008] In a first aspect, the present application provides a driving risk prompting method, which includes: identifying risky objects around a vehicle; displaying a floating window on the vehicle's display screen; determining a target image of the area where the risky object is located based on the location of the risky object, the target image including a rendering or a real-life image of the area where the risky object is located, and the proportion of the risky object in the target image is within a preset ratio range; and displaying the target image in the floating window.

[0009] There is at least one floating window, and its display area (also referred to as size) is a preset display area (also referred to as preset size). The display area refers to the area of ​​the floating window when displayed on the display screen. Optionally, for each identified risk object around the vehicle, a floating window corresponding to the risk object is displayed, i.e., each identified risk object corresponds to a floating window. The target image is the image to be displayed in the floating window, and the display area of ​​the target image is the same as the display area of ​​the floating window.

[0010] When the vehicle is close to an identified risky object, the real-life image provides a clearer display of the risky object. When the vehicle is farther away from the identified risky object, the rendered image provides a clearer display of the risky object. In this embodiment of the present application, when the distance between the risky object and the vehicle is less than a preset distance, the target image includes a real-life image of the area where the risky object is located. When the distance between the risky object and the vehicle is greater than or equal to the preset distance, the target image includes a rendered image of the area where the risky object is located.

[0011] This method solves the problem in related technologies where risk objects are marked with small blue dots, making them difficult for users to detect. Consequently, the risk objects can only be clearly observed when the distance between the vehicle and the risk object decreases to a certain level, potentially preventing users from making timely decisions about the risk objects. The beneficial effect is that, on the one hand, the display of the floating window provides a clear reminder, allowing users to perceive the appearance of the floating window as equivalent to the presence of a risk object, thereby enhancing their risk awareness. On the other hand, the proportion of the risk object in the floating window is greater than that of the risk object in the rendered image of the vehicle's surroundings, allowing users to more clearly observe and identify the risk object through the floating window, gaining information about it and making informed decisions. Furthermore, even at a distance from the vehicle, users can still observe and grasp the risk object through the floating window, providing ample time for decision-making and enabling them to make timely decisions regarding the risk object. Compared to related technologies, this method effectively reduces the probability of dangerous situations caused by hasty decisions, thereby improving driving safety.

[0012] In one possible implementation, the target image can be obtained based on a rendered image of the vehicle's surroundings or a real-life image showing a risky object. The process of determining the target image of the area where the risky object is located based on the location of the risky object includes: determining the location of the risky object in an initial image based on the location of the risky object, the initial image comprising a rendered image of the vehicle's surroundings or a real-life image showing the risky object; determining a target magnification; magnifying the initial image according to the target magnification; and obtaining the target image based on the location of the risky object in the initial image and the magnified initial image.

[0013] The beneficial effect is that a rendering or real-life image of the vehicle's surroundings can be directly used, so that the vehicle consumes fewer computing resources in the process of determining the target image.

[0014] In one possible implementation, the process of determining the target magnification includes: determining the target magnification based on the distance between the risk object and the vehicle and a preset mapping relationship, wherein the preset mapping relationship includes a mapping relationship between distance and magnification, and the greater the distance in the preset mapping relationship, the greater the magnification.

[0015] The preset mapping relationship can be a function of magnification with respect to distance. The function can be a continuous function, where the magnification changes in real time as the distance changes. The function can also be a piecewise function, where multiple magnifications are predefined, each corresponding to a distance interval.

[0016] In one possible implementation, the process of determining the target magnification includes: determining the target magnification range based on the proportion of the risk object in the initial image, the display area of ​​the floating window, and a preset ratio range, so that when the risk object in the initial image that has been magnified according to the magnification within the target magnification range is displayed in the floating window, the proportion in the floating window is within the preset ratio range; and determining the target magnification based on the target magnification range.

[0017] In one possible implementation, a target image can be determined based on the area where the risk object is located in the vehicle's environmental feature model. The target image includes a rendering of the area where the risk object is located. The process of determining the target image for the area where the risk object is located based on the location of the risk object includes: obtaining the vehicle's environmental feature model; determining a target rendering perspective based on the location of the risk object, its actual size, and the display area of ​​the floating window; and rendering the environmental feature model according to the target rendering perspective to obtain the target image.

[0018] The environmental feature model can be obtained by modeling environmental information. The environmental information includes at least one of the following: point cloud data from a laser radar, image data from an image sensor, and feedback data from a millimeter-wave radar.

[0019] The beneficial effect is that the target image obtained by determining the rendering perspective has higher clarity, and the risk object can be displayed more clearly in the subsequent display, making it easier for users to observe and identify the risk object, grasp the information of the risk object, and thus make correct decisions.

[0020] In one possible implementation, a target image is acquired based on a target image sensor with adjusted parameters, and the target image includes a real-world view of the area where the risky object is located. Determining the target image of the area where the risky object is located based on the location of the risky object includes: determining a target image sensor based on the location of the risky object and adjusting the imaging parameters of the target image sensor; and determining an image captured by the target image sensor using the adjusted imaging parameters as the target image.

[0021] For example, the image sensor corresponding to the real-life image displaying the risky object can be determined as the target image sensor. Alternatively, the target image sensor can be determined based on the location of the risky object to better present the risky object. Target image sensors include, but are not limited to, cameras located close to the risky object, cameras with a greater depth of field, and macro cameras.

[0022] In one possible implementation, the method also includes: determining risk information, the risk information including at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object; and displaying the risk information in a floating window.

[0023] The beneficial effect is that by displaying the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object, the user can directly identify the collision risk of the risk object and take avoidance measures in time.

[0024] In one possible implementation, the method further includes: obtaining a predicted driving trajectory of the vehicle and a predicted motion trajectory of the risk object; generating a driving animation of the vehicle based on the predicted driving trajectory; generating a motion animation of the risk object based on the predicted motion trajectory; and displaying the driving animation of the vehicle and the motion animation of the risk object in a floating window.

[0025] The beneficial effect is that by displaying the driving animation of the vehicle and the motion animation of the risk object, the presentation of the collision animation can more directly prompt the collision risk compared to simply presenting static information.

[0026] In one possible implementation, the vehicle's display screen displays a rendering of the vehicle's surrounding environment. The process of displaying a floating window on the vehicle's display screen includes: determining the target position of the risk object relative to the vehicle within a preset time in the future; and displaying a floating window at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

[0027] The beneficial effect is that the display orientation of the floating window is the same as the actual orientation of the risk object relative to the vehicle, which helps the user to judge the direction of the risk object through the floating window and facilitates the user to make driving decisions.

[0028] In one possible implementation, the method further includes: looping through a risk warning process, the risk warning process including: identifying risk objects around the vehicle; updating a target image of the area where the risk objects are located based on their latest positions; and displaying the updated target image in a floating window.

[0029] In the second aspect, the present application provides a driving risk prompt device, which includes: an identification module for identifying risk objects around a vehicle; a display module for displaying a floating window on the vehicle's display screen; an image determination module for determining a target image of the area where the risk object is located based on the position of the risk object, the target image including a rendering or a real-life image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; the display module is also used to display the target image in the floating window.

[0030] In one possible implementation, the image determination module is specifically configured to: determine the position of the risk object in an initial image based on the position of the risk object, where the initial image includes a rendering of the vehicle's surroundings or a real-life image showing the risk object; determine a target magnification; enlarge the initial image according to the target magnification; and obtain a target image based on the position of the risk object in the initial image and the enlarged initial image.

[0031] In one possible implementation, the image determination module is specifically used to determine the target magnification based on the distance between the risk object and the vehicle and a preset mapping relationship. The preset mapping relationship includes a mapping relationship between distance and magnification. In the preset mapping relationship, the greater the distance, the greater the magnification.

[0032] In one possible implementation, the image determination module is specifically used to: determine a target magnification range based on the proportion of risk objects in the initial image, the display area of ​​the floating window, and a preset ratio range, so that when the risk objects in the initial image that are magnified according to the magnification within the target magnification range are displayed in the floating window, the proportion of the risk objects in the floating window is within the preset ratio range; and determine the target magnification ratio based on the target magnification range.

[0033] In one possible implementation, the target image includes a rendering of the area where the risk object is located, and the image determination module is specifically used to: obtain the vehicle's environmental feature model; determine the target rendering perspective based on the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window; render the environmental feature model according to the target rendering perspective to obtain the target image.

[0034] In one possible implementation, the target image includes a real-life image of the area where the risk object is located, and the image determination module is specifically configured to: determine a target image sensor and adjust imaging parameters of the target image sensor based on the location of the risk object; and determine an image captured by the target image sensor according to the adjusted imaging parameters as the target image.

[0035] In one possible implementation, when the distance between the risk object and the vehicle is less than a preset distance, the target image includes a real-life image of the area where the risk object is located; when the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendered image of the area where the risk object is located.

[0036] In one possible implementation, the device also includes: a risk information determination module, used to determine risk information, the risk information including at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position at which the risk object is closest to the vehicle, and the risk level of the risk object; the aforementioned display module is also used to display the risk information in a floating window.

[0037] In one possible implementation, the risk information determination module is also used to: obtain the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object; generate a driving animation of the vehicle based on the predicted driving trajectory; generate a motion animation of the risk object based on the predicted motion trajectory; the display module is also used to display the driving animation of the vehicle and the motion animation of the risk object in a floating window.

[0038] In a possible implementation, the display module is specifically configured to: determine the target position of the risk object relative to the vehicle within a preset future time; and display a floating window at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

[0039] In one possible implementation, the recognition module is also used to identify risk objects around the vehicle when the risk warning process is cyclically executed; the image determination module is also used to update the target image of the area where the risk object is located according to the latest position of the risk object when the risk warning process is cyclically executed; the display module is also used to display the updated target image in the floating window when the risk warning process is cyclically executed.

[0040] In a third aspect, the present application provides a driving risk reminder device, which includes: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by one or more processors, the one or more processors implement the method as described in any one of the first aspects.

[0041] In a fourth aspect, the present application provides a vehicle, comprising: a driving risk reminder device as described in any one of the second aspect or the third aspect.

[0042] In a fifth aspect, the present application provides a driving risk reminder device, comprising a processor for executing the method as described in any one of the first aspects.

[0043] In a sixth aspect, the present application provides a driving risk reminder device, comprising: a processing circuit and an interface circuit; wherein the interface circuit is used to couple with a memory outside the driving risk reminder device and provide a communication interface for the processing circuit to access the memory; the processing circuit is used to execute program instructions in the memory to implement a method as described in any one of the first aspects.

[0044] In a specific implementation, the driving risk warning device can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit can be, for example, but not limited to, received and input by a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to and transmitted by a transmitter. The input circuit and the output circuit can be the same circuit, which functions as an input circuit and an output circuit at different times. The embodiments of this application do not limit the specific implementation of the processor and various circuits.

[0045] In one implementation, the driving risk reminder device may be a wireless communication device, that is, a computer device that supports wireless communication functions. Specifically, the wireless communication device may be a terminal such as a smartphone, or a wireless access network device such as a base station. The system chip may also be referred to as a system on chip (SoC), or simply as an SoC chip. The communication chip may include a baseband processing chip and a radio frequency processing chip. The baseband processing chip is sometimes also referred to as a modem or baseband chip. The radio frequency processing chip is sometimes also referred to as a radio frequency transceiver or radio frequency chip. In physical implementation, some or all of the chips in the communication chip may be integrated inside the SoC chip. For example, the baseband processing chip is integrated into the SoC chip, and the radio frequency processing chip is not integrated with the SoC chip. The interface circuit may be the radio frequency processing chip in the wireless communication device, and the processing circuit may be the baseband processing chip in the wireless communication device.

[0046] In another implementation, the driving risk warning device may be a component within a wireless communication device, such as an integrated circuit product such as a system-on-chip (SoC) or a communication chip. The interface circuit may include an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuitry on the chip or chip system. The processor may also be embodied as a processing circuit or logic circuit.

[0047] In a seventh aspect, the present application provides a computer-readable storage medium, in which program code is stored. When the program code is executed by a processor, the method as described in any one of the first aspects is implemented.

[0048] In an eighth aspect, the present application provides a chip, comprising: at least one processor. The at least one processor is configured to execute the method according to any one of the first aspects.

[0049] Optionally, the chip further includes a memory, and at least one processor is configured to execute code in the memory. When the at least one processor executes the code, the chip implements the method as described in any one of the first aspects.

[0050] Optionally, the chip may also be an integrated circuit.

[0051] In a ninth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to implement the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0053] FIG2 is a functional block diagram of a vehicle provided in an embodiment of the present application;

[0054] FIG3 is a flow chart of a method for prompting driving risks provided in an embodiment of the present application;

[0055] FIG4 is a schematic diagram of a rendering of a vehicle's surrounding environment provided in an embodiment of the present application;

[0056] FIG5 is a schematic diagram of a rendering perspective of an object provided in an embodiment of the present application;

[0057] FIG6 is a schematic diagram of displaying a target image in a floating window provided by an embodiment of the present application;

[0058] FIG7 is a flow chart of another driving risk prompting method provided in an embodiment of the present application;

[0059] FIG8 is a schematic diagram showing a floating window provided in an embodiment of the present application;

[0060] FIG9 is a schematic diagram of a HUD provided in an embodiment of the present application;

[0061] FIG10 is a block diagram of a driving risk reminder device provided by an embodiment of the present application;

[0062] FIG11 is a block diagram of another driving risk reminder device provided by an embodiment of the present application;

[0063] FIG12 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0064] FIG13 is a schematic structural diagram of a driving risk reminder device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0066] The terms "first," "second," and the like in the description, embodiments, claims, and drawings of this application are used solely for descriptive purposes and are not to be construed as indicating or implying relative importance or order. Furthermore, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, inclusion of a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0068] The present invention provides a method for indicating driving risks, resolving the issue in related technologies where users may not be able to make timely decisions regarding risky objects. This method allows users to more easily observe risky objects, even from a distance, giving them more time to make decisions and thus improving driving safety. This method can be applied to vehicles (e.g., smart cars) or cockpits (e.g., smart cockpits).

[0069] Please refer to Figure 1, which is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 shows vehicle 001 and user 002. Vehicle 001 has a built-in car operating system (referred to as car computer) and is equipped with a variety of sensors. The car computer can automatically determine the driving strategy in the current scenario based on the collected sensor information and internal data, and generate and execute instructions corresponding to the driving strategy. The car computer can also analyze the collected sensor information, identify risk objects around the vehicle that affect the vehicle's driving, and remind the user. User 002 is often the driver or car owner, who can obtain various information around the vehicle through the information displayed on the central control screen or the instrument panel, and identify risks around the vehicle based on the displayed information and / or other sound and light information.

[0070] The specific structure of the vehicle is described below. Please refer to Figure 2, which is a functional block diagram of a vehicle provided in an embodiment of the present application. The vehicle 100 may include various subsystems, such as a travel system 101, a sensor system 102, a control system 103, one or more peripheral devices 104, a power supply 105, a user interface 106, and a computer system 107. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the vehicle 100 can be interconnected by wire or wireless, for example, through an on-board communication network, which may include a controller area network bus (CAN Bus) or an on-board Ethernet, etc.

[0071] Travel system 101 may include components for providing power to vehicle 100. In one embodiment, travel system 101 may include an engine 1011, an energy source 1012, a transmission 1013, and wheels 1014.

[0072] The engine 1011 can be an internal combustion engine, an electric motor, an air compression engine, or another type of engine or a combination of multiple engines. Examples of the combination of multiple engines include a hybrid engine consisting of a gasoline engine and an electric motor, and a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 1011 is configured to convert the energy source 1012 into mechanical energy.

[0073] Energy source 1012 may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 1012 may also provide energy to other systems of vehicle 100.

[0074] The transmission 1013 can transmit mechanical power from the engine 1011 to the wheels 1014. The transmission 1013 can include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 1013 can also include other components, such as a clutch. The drive shaft can include one or more shafts that can be coupled to one or more wheels 1014.

[0075] The sensor system 102 may include several sensors that sense information about the vehicle 100 itself and the environment surrounding the vehicle 100. For example, the sensor system 102 may include a positioning system 1021 (the positioning system may be a global positioning system (GPS), a BeiDou system, or other positioning systems), an inertial measurement unit (IMU) 1022, a radar 1023 (the radar may be a lidar), a laser rangefinder 1024, a camera 1025, a computer vision system 1026, and a sensor fusion algorithm 1027. The sensor system 102 may also include sensors for the vehicle 100's internal systems (e.g., an in-vehicle air quality monitor, a fuel gauge, and an oil temperature gauge). Sensor data from one or more of these sensors may be used to detect an object and its corresponding characteristics (position, shape, direction, speed, etc.). This detection and recognition is a key function for the vehicle 100 to achieve safe operation.

[0076] Among them, the positioning system 1021 can be used to estimate the geographic location of the vehicle 100. The IMU 1022 can be used to sense the position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 1022 can be a combination of an accelerometer and a gyroscope.

[0077] Radar 1023 can use radio signals to sense objects in the surrounding environment of vehicle 100. In some embodiments, in addition to sensing objects, radar 1023 can also be used to sense the speed and / or direction of travel of objects. Radar 1023 includes but is not limited to laser radar and millimeter wave radar.

[0078] The laser rangefinder 1024 may utilize laser light to sense objects in the environment of the vehicle 100. In some embodiments, the laser rangefinder 1024 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0079] The camera 1025 may be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 1025 may be a still camera or a video camera.

[0080] The computer vision system 1026 can be used to process and analyze images captured by the camera 1025 to identify objects and / or features in the environment surrounding the vehicle 100. These objects and / or features may include traffic signs, road boundaries, obstacles, and road surface conditions. The computer vision system 1026 can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 1026 can be used to map the environment, track objects, estimate the speed of objects, and so on.

[0081] The control system 103 is used to perform operations to control the vehicle 100 and its components. The control system 103 may include various components. As shown in FIG2 , the control system 103 may include a steering system 1031 , a throttle 1032 , a braking unit 1033 , a route planning system 1034 , and an obstacle avoidance system 1035 .

[0082] The forward direction of the vehicle 100 can be adjusted by operating the steering system 1031. For example, in one embodiment, it can be a steering wheel system. The throttle 1032 is used to control the operating speed of the engine 1011 and thus the speed of the vehicle 100.

[0083] Braking unit 1033 is used to control the deceleration of vehicle 100. Braking unit 1033 can use friction to slow down wheels 1014. In other embodiments, braking unit 1033 can convert the kinetic energy of wheels 1014 into electric current. Braking unit 1033 can also take other forms to slow the rotation speed of wheels 1014 and thus control the speed of vehicle 100.

[0084] The route planning system 1034 is used to determine the driving route of the vehicle 100. In some embodiments, the route planning system 1034 can combine data from the sensor system 102 (e.g., the positioning system 1021 and the computer vision system 1026) and one or more predetermined maps to plan a driving route for the vehicle 100 that avoids potential obstacles in the environment.

[0085] The obstacle avoidance system 1035 is used to generate control values ​​of the accelerator, brake, and steering angle according to the driving route or driving trajectory output by the route planning system 1034, thereby controlling the steering system 1031, the accelerator 1032, and the brake unit 1033.

[0086] Of course, in one embodiment, the control system 103 may include additional or alternative components other than those shown and described, or may also include a portion of the components shown above.

[0087] The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users via one or more peripheral devices 104. The peripheral devices 104 may include a wireless communication system 1041, an onboard computer 1042, a microphone 1043, and / or a speaker 1044.

[0088] In some embodiments, the peripheral devices 104 are used to provide a means for the user of the vehicle 100 to interact with the user interface 106. For example, the onboard computer 1042 can provide information to the user of the vehicle 100. The user interface 106 can also operate the onboard computer 1042 to receive user input. In one implementation, the onboard computer 1042 can be operated via a touch screen. In other cases, the peripheral devices 104 can provide a means for the vehicle 100 to communicate with other devices located within the vehicle. For example, the microphone 1043 can receive audio (e.g., voice commands or other audio input) from the user of the vehicle 100. Similarly, the speaker 1044 can output audio to the user of the vehicle 100.

[0089] The wireless communication system 1041 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 1041 can use third-generation mobile communication technology (3G) communication, such as code division multiple access (CDMA), global system for mobile communications (GSM), or fourth-generation mobile communication technology (4G) communication, such as LTE (long term evolution). Or fifth-generation mobile communication technology (5G) communication. The wireless communication system 1041 can communicate with a wireless local area network (WLAN) using wireless fidelity (WiFi). In some embodiments, the wireless communication system 1041 can communicate directly with the device using an infrared link, Bluetooth, or Zigbee. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system 1041 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communication devices between vehicles and / or roadside stations.

[0090] The power source 105 can provide power to the various components of the vehicle 100. In one embodiment, the power source 105 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such batteries can be configured as a power source and provide power to the various components of the vehicle 100. In some embodiments, the power source 105 and the energy source 1012 can be implemented together, such as in an all-electric vehicle.

[0091] Some or all functions of the vehicle 100 are controlled by a computer system 107. The computer system 107 may include at least one processor 1071 that executes instructions 1073 stored in a non-transitory computer-readable medium, such as a memory 1072. The computer system 107 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0092] The processor 1071 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor 1071 can be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although FIG. 2 functionally illustrates the processor 1071, memory 1072, and other elements of the computer system 107, it should be understood by those skilled in the art that the processor 1071 and memory 1072 can actually include multiple other processors or memories that are not located in the same physical housing. For example, the memory 1072 can be a hard drive or other storage medium located in a housing different from the computer system 107. Therefore, references to the processor 1071 will be understood to include references to a collection of processors or memories that can operate in parallel or not in parallel. Instead of using a single processor to perform the steps described herein, some components, such as steering components and deceleration components, can each have their own processor, which only performs calculations related to the functions specific to the component; or subsystems such as the travel system 101, sensor system 102 and control system 103 can also have their own processors to perform calculations for related tasks of the corresponding subsystems to achieve corresponding functions.

[0093] In various aspects described herein, the processor 1071 can be located remotely from the vehicle 100 and in wireless communication with the vehicle 100. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle 100, while others are performed by a remote processor, including taking the necessary steps to execute a single maneuver.

[0094] In some embodiments, the memory 1072 may include instructions 1073 (e.g., program logic) that may be executed by the processor 1071 to perform various functions of the vehicle 100, including those described above. The memory 1072 may also include additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the travel system 101, the sensor system 102, the control system 103, and the peripheral devices 104.

[0095] In addition to the instructions 1073, the memory 1072 may also store other relevant data, such as map information, route information, the vehicle's location, direction, speed, and other relevant information. This information may be used by the vehicle 100, or specifically by the computer system 107, during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0096] User interface 106 is used to provide information to or receive information from a user of vehicle 100. Optionally, user interface 106 may include one or more input / output devices within the set of peripherals 104, such as wireless communication system 1041, onboard computer 1042, microphone 1043, and speaker 1044.

[0097] Computer system 107 may control functions of vehicle 100 based on input received from various subsystems (e.g., travel system 101, sensor system 102, and control system 103) and from user interface 106. In some embodiments, computer system 107 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0098] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 100. For example, the memory 1072 may be partially or completely separate from the vehicle 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0099] Optionally, the above components are only an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 2 should not be understood as a limitation to the embodiments of the present application.

[0100] An autonomous vehicle traveling on a road, such as vehicle 100 above, can identify objects in its surroundings to determine adjustments to the vehicle's trajectory, including adjustments to the vehicle's speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently and used to determine the autonomous vehicle's trajectory plan, including the speed adjustment, based on its respective characteristics, such as its current speed, acceleration, and distance from the vehicle.

[0101] Optionally, a computing device associated with the vehicle 100 (such as the computer system 107 and the computer vision system 1026 of Figure 2) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can be considered together to predict the behavior of a single identified object. The vehicle 100 can plan its driving trajectory (including speed) based on the predicted behavior of the identified objects. Based on the planning results, it is determined for the autonomous vehicle to what state the vehicle will need to adjust to (in terms of speed adjustment, such as acceleration, deceleration or stopping). In this process, other factors can also be considered to determine the driving trajectory of the vehicle 100, such as the lateral position of the vehicle 100 on the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0102] Based on the planning results, in addition to adjusting the speed for the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0103] The vehicle 100 may be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawn mower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, cart, etc., and the embodiments of the present application do not impose any particular limitation.

[0104] Please refer to FIG3 , which is a flowchart of a driving risk prompting method provided in an embodiment of the present application. The method can be applied to a cockpit or a vehicle, such as the vehicle shown in FIG1 and FIG2 . The method may include the following steps:

[0105] 201. Identify dangerous objects around the vehicle.

[0106] Risky objects around the vehicle can affect its normal operation. They can be static or dynamic. For example, they can be objects in the vehicle's future trajectory, or objects that are expected to dynamically move into the vehicle's future trajectory. Risky objects include, but are not limited to, other vehicles on the road, rolling tires, floating objects such as plastic bags, bumps on the ground, and rocks.

[0107] The vehicle analyzes the various information obtained, and then identifies risk objects around the vehicle according to a preset algorithm (also known as a risk object recognition algorithm).

[0108] As can be seen from the preceding description, a vehicle includes a sensor system. The information acquired by the vehicle includes sensor information collected by multiple sensors within the sensor system, both inside and outside the vehicle, such as sensor information related to the vehicle's surrounding environment. This sensor information includes, but is not limited to, images captured by image sensors, point cloud data collected by lidar, and radar feedback data received by millimeter-wave radar. When identifying risky objects around the vehicle using images captured by the image sensor, multiple images can be analyzed simultaneously.

[0109] The information obtained by the vehicle may also include environmental information around the vehicle obtained through the network or communication. For example, a high-precision map of the vehicle's surroundings can be obtained through the network or communication, and then environmental information can be obtained based on the high-precision map. For example, vehicle A updates the local high-precision map based on the point cloud information collected by the lidar. Afterwards, the vehicle A's onboard computer compares the local high-precision map with the high-precision map on the cloud. If there are differences between the two high-precision maps, the local high-precision map can be synchronized to the cloud in real time. Vehicle B can directly download the latest high-precision map synchronized by vehicle A from the cloud. For another example, a road device (such as an image sensor) can also scan nearby environmental information in real time and update the high-precision map of the nearby area. The vehicle communicates with the road device, or communicates with the cloud / server (such as a traffic management system) connected to the road device, to obtain the latest high-precision map. The vehicle analyzes the latest high-precision map obtained to identify risk objects around the vehicle.

[0110] The preset algorithm can be built into the vehicle. The vehicle uses various information acquired or the results of analyzing such information as input to the preset algorithm, which then outputs risky objects in the vehicle's vicinity. The algorithm can determine whether the objects in the input information are within the vehicle's future trajectory or are expected to move into the vehicle's future trajectory.

[0111] For example, the preset algorithm first identifies an object (including dynamic and static objects) based on the input information. It then determines whether the object is likely to collide with the vehicle in the future, that is, whether the object is likely to appear on the vehicle's future driving trajectory. If the object is static, it determines whether the object is within the vehicle's future driving trajectory. If the object is dynamic, it first predicts the object's future motion trajectory, and then, based on the object's future motion trajectory and the vehicle's future driving trajectory, determines whether the object is likely to collide with the vehicle in the future.

[0112] This process can be performed by the vehicle's vision and radar systems. It should be noted that the process 201 is always performed during the vehicle's driving process, that is, the identification of risk objects around the vehicle is continuously performed during the vehicle's driving process.

[0113] 202. Display a floating window on a display screen of the vehicle.

[0114] The number of floating windows is at least one, and its display area (also referred to as size) is a preset display area (also referred to as preset size). The display area refers to the area of ​​the floating window when displayed on the display screen. The number of floating windows is related to the risk objects identified in the aforementioned process 201. Optionally, for each identified risk object, a corresponding floating window is displayed, that is, each identified risk object corresponds to one floating window.

[0115] The display screen can be a central control screen or instrument screen of the vehicle, etc., and is not limited to this embodiment of the present application. While the vehicle is driving, a rendering of the vehicle's surrounding environment can be displayed on the display screen. At this time, a floating window can be created on top of the rendering of the vehicle's surrounding environment, thereby displaying the floating window.

[0116] The rendering of the vehicle's surrounding environment is a two-dimensional rendering that displays the vehicle's surrounding environment in a three-dimensional effect on the vehicle's display screen. It can be obtained based on information acquired by the vehicle (such as at least one of the point cloud data of the lidar, the image data of the image sensor, and the feedback data of the millimeter-wave radar).

[0117] For example, the information obtained by the vehicle can be used to model the environment and obtain an environmental feature model, which is a three-dimensional (3D) model. The environmental feature model is then rendered using a preset rendering perspective to obtain a rendering of the vehicle's surrounding environment. The rendering process can be regarded as simulating a virtual camera, presenting the environmental feature model through the imaging perspective of the virtual camera, thereby displaying the environmental feature model on a two-dimensional plane, and the rendering perspective is the imaging perspective of the virtual camera. It can be seen that there may be some areas in the environmental feature model that are not captured by the virtual camera, that is, there will be some areas that are not displayed.

[0118] The preset rendering perspective is vehicle-dependent, using the vehicle as a reference point. As the vehicle's position changes, the preset rendering perspective also changes. This allows users to fully observe the vehicle's surroundings.

[0119] When creating a rendering of the vehicle's surroundings, for objects that will not affect the vehicle's driving (such as buildings on the street), such objects can be simplified by filtering the information obtained from the vehicle. The rendering of the vehicle's surroundings can reflect the environmental information around the vehicle in real time, such as surrounding vehicles, road signs, traffic signs, pedestrians, road edges, lane lines, and zebra crossings. While the vehicle is driving, the rendering of the vehicle's surroundings is displayed on the display screen. Users can grasp the environment around the vehicle through the rendering of the vehicle's surroundings, without having to use traditional methods such as rearview mirrors / observing outside the car. The rendering of the vehicle's surroundings is real-time, and the vehicle continuously updates the rendering of the vehicle's surroundings based on the information obtained in real time.

[0120] For example, please refer to Figure 4, which is a schematic diagram of a rendering of a vehicle's surrounding environment provided in an embodiment of the present application. Figure 4 shows a rendering of the vehicle's surrounding environment displayed on a display screen during vehicle driving. As shown in Figure 4, the rendering of the vehicle's surrounding environment shows a 3D vehicle model, the road / lane where the vehicle is located, and the vehicle's driving route. In addition, objects such as traffic lights and other vehicles on the road are also shown. As can be seen from Figure 4, objects related to vehicle driving, such as roads, vehicles, traffic lights, lane lines, and zebra crossings, are rendered in a refined manner during modeling. Objects that do not affect vehicle driving, such as buildings on both sides of the road, are displayed in a simplified manner during modeling.

[0121] The floating window can be displayed at any position on the display screen. Optionally, when creating a floating window on top of the rendering of the vehicle's surrounding environment, the floating window can be displayed on the left or right side of the vehicle object in the rendering of the vehicle's surrounding environment.

[0122] For example, the display position of a floating window can be determined based on the position of the risky object relative to the vehicle within a preset future timeframe. First, a target position is determined based on the position of the risky object relative to the vehicle within the preset future timeframe. Then, the floating window is displayed at the target position of the vehicle object in the upper layer of a rendered image of the vehicle's surroundings. For example, the position of the risky object relative to the vehicle within the preset future timeframe can be determined based on the predicted motion trajectory of the risky object and the predicted driving trajectory of the vehicle. If the risky object is expected to be located to the left of the vehicle within the preset future timeframe, the floating window is displayed to the left of the vehicle object in the rendered image of the vehicle's surroundings. If the risky object is expected to be located to the right of the vehicle within the preset future timeframe, the floating window is displayed to the right of the vehicle object in the rendered image of the vehicle's surroundings. The display position of the floating window is consistent with the actual position of the risky object relative to the vehicle, helping users understand the direction of the risky object through the floating window and facilitating driving decisions.

[0123] It should be noted that the information obtained by the vehicle is updated in real time. As this information is updated in real time, the environmental feature model will also be updated in real time, and accordingly, the rendering of the vehicle's surrounding environment will also be updated in real time.

[0124] 203. Determine a target image of the area where the risk object is located based on the location of the risk object. The target image includes a rendering or a real-life image of the area where the risk object is located. The proportion of the risk object in the target image is within a preset ratio range.

[0125] The target image is the image to be displayed in the floating window. The target image's display area is the same as the floating window's. This means the target image can be fully displayed within the floating window and completely occupies it. After the target image is displayed in the floating window, the proportion of the risk object in the floating window is equal to the proportion of the floating window in the target image.

[0126] When the proportion of the risk object in the target image is within a preset ratio interval, the risk object can be displayed completely and clearly with a better display effect. For example, the preset ratio interval is [M1, M2], M1 can be 18%, 19%, 20%, 21% or 22%, etc., and M2 can be 38%, 39%, 40%, 41% or 42%, etc., and the embodiments of the present application are not limited to this. The proportion of the risk object in the target image can be regarded as the ratio of the display area of ​​the closed area formed by the closed edge of the risk object in the target image to the display area of ​​the target image. Or it can be regarded as the ratio of the number of pixels in the closed area formed by the closed edge of the risk object in the target image to the total number of pixels included in the target image.

[0127] The vehicle (e.g., the vehicle-mounted system) can determine the location of a risky object based on information acquired by the vehicle (e.g., point cloud data from a lidar, image data from an image sensor, feedback data from a millimeter-wave radar, etc.). The location of the risky object can be its current physical location relative to the vehicle.

[0128] In a first implementation, the target image can be acquired based on a rendered image of the vehicle's surroundings or a real-life image showing a risky object. When the target image is acquired based on a rendered image of the vehicle's surroundings, the target image includes a rendered image of the area where the risky object is located. When the target image is acquired based on a real-life image showing a risky object, the target image includes a real-life image of the area where the risky object is located.

[0129] In this first implementation, for example, the vehicle computer determines the location of the risky object in the initial image based on the location of the risky object. A target magnification factor is determined, and the initial image is magnified according to the target magnification factor. A target image is then obtained based on the location of the risky object in the initial image and the magnified initial image. The initial image may include a rendering of the vehicle's surroundings or a real-life image showing the risky object. The vehicle computer may crop the magnified initial image based on the location of the risky object in the initial image to obtain the target image.

[0130] When the initial image includes a rendering of the vehicle's surroundings, the rendering of the vehicle's surroundings can be obtained first, and then, based on the location of the risky object, the location of the risky object in the rendering of the vehicle's surroundings can be synchronously located. The vehicle can be located in the rendering of the vehicle's surroundings based on the vehicle's location and the information obtained by the vehicle.

[0131] When the initial image includes a real-life image of a risky object, the real-life image of the risky object may be first acquired, and then, based on the location of the risky object and the image recognition process, the location of the risky object may be located from the real-life image. The real-life image of the risky object may be an image captured by an image sensor in the vehicle, including but not limited to a camera positioned near the risky object, a camera with a greater depth of field, or a macro camera.

[0132] For example, if a vehicle identifies a risky object as a curb during a sharp turn, an image showing the risky object captured by a camera mounted at the rear of the vehicle can be used as the initial image. In another example, an image showing the risky object captured by a corresponding camera can be used as the initial image based on the relative position of the risky object to the vehicle. For example, an image showing the risky object captured by a camera mounted in the vehicle's target direction can be used as the initial image, where the target direction is the direction of the risky object relative to the vehicle. If the risky object is on the left side of the vehicle, an image showing the risky object captured by a camera mounted on the left side of the vehicle can be used as the initial image; if the risky object is on the right side of the vehicle, an image showing the risky object captured by a camera mounted on the right side of the vehicle can be used as the initial image.

[0133] In one implementation for determining a target magnification, the target magnification can be determined based on the distance between the risk object and the vehicle and a preset mapping relationship. The preset mapping relationship includes a mapping relationship between distance and magnification, where the greater the distance, the greater the magnification. A maximum magnification can be preset. If the magnification of the distance mapping between the risk object and the vehicle is greater than the maximum magnification, the maximum magnification is determined as the target magnification. If the magnification of the distance mapping between the risk object and the vehicle is less than or equal to the maximum magnification, the magnification of the mapping is determined as the target magnification. As the distance between the risk object and the vehicle increases, the proportion of the risk object in the initial image decreases, and the mapped magnification increases. A maximum magnification is preset. Therefore, the target magnification determined by the preset mapping relationship can effectively ensure that the proportion of the risk object in the target image is within a preset ratio range.

[0134] For example, the vehicle can determine the distance between the risk object and the vehicle based on the acquired information. The preset mapping relationship can be a function N(X) of the magnification N with respect to the distance X. N(X) can be a continuous function, and as the distance X changes, the magnification N also changes in real time. After N reaches the maximum magnification Nmax, if the distance increases further, the magnification N remains unchanged at the maximum magnification Nmax.

[0135] N(X) can also be a piecewise function. Multiple magnification factors are predefined in N(X), each corresponding to a different range. The ranges corresponding to different magnification factors may not overlap. When the distance X between the risk object and the vehicle is in different ranges, different magnification factors N are used. For example, N(X) predefines magnification factors N1, N2, N3, and N4, where N1 ≥ N2 ≥ N3 ≥ N4. N1 corresponds to the range [500 meters, +∞], N2 to [300 meters, 500 meters], N3 to [100 meters, 300 meters], and N4 to (0, 100 meters).

[0136] In another implementation method for determining the target magnification ratio, a target magnification range can be determined based on the proportion of the risk object in the initial image, the display area of ​​the floating window, and a preset ratio range, and the target magnification ratio is then determined based on the target magnification range. Specifically, when the risk object in the initial image, after being magnified by a factor within the target magnification range, is displayed in the floating window, its proportion in the floating window falls within the preset ratio range. That is, on the same display screen, the ratio of the display area of ​​the risk object in the initial image, after being magnified by a factor within the target magnification range, to the display area of ​​the floating window falls within the preset ratio range.

[0137] The vehicle determines a first magnification required when the proportion of the risk object in the floating window reaches M1, after the initial image is magnified within a target magnification range. The vehicle also determines a second magnification required when the proportion of the risk object in the floating window reaches M2, after the initial image is magnified within the target magnification range. The range consisting of the first and second magnifications is then determined as the target magnification range.

[0138] When determining the target magnification based on the target magnification range, as an example, any magnification value can be taken from the target magnification range as the target magnification. As another example, the target magnification can be determined in combination with the aforementioned preset mapping relationship. If N(X) is a continuous function, the target magnification range can be used as the value range of N(X), and then the target magnification is determined in real time based on the distance between the risk object and the vehicle and N(X). If N(X) is a piecewise function, multiple magnifications can be determined from the target magnification range as piecewise function values ​​(i.e., the aforementioned predefined multiple magnifications), and then the corresponding magnification is determined for the distance interval where the risk object and the vehicle are located, and determined as the target magnification. The specific implementation process of these two examples can refer to the first implementation method mentioned above, and the embodiments of this application will not be described here.

[0139] When the initial image is magnified according to the target magnification, the initial image may be magnified in equal proportion according to the target magnification to obtain the target image.

[0140] In another example, the vehicle computer determines a partial image from the initial image based on the location of the risk object. A target magnification factor is then determined. The partial image is magnified according to the target magnification factor to obtain a target image. The partial image is an image of the area in the initial image where the risk object is located. The process of determining the target magnification factor and magnifying the initial image can be referenced to the aforementioned example and will not be further described in detail in this embodiment of the present application.

[0141] The first implementation method can directly use a rendering or a real scene image of the vehicle's surrounding environment, so that the vehicle consumes less computing resources in the process of determining the target image.

[0142] In the second implementation, the target image can be determined based on the area where the risk object is located in the vehicle's environmental feature model. The target image includes a rendering of the area where the risk object is located. The vehicle computer obtains the vehicle's environmental feature model and then determines the target rendering perspective based on the risk object's location, actual size, and the display area of ​​the floating window. The environmental feature model is then rendered according to the target rendering perspective to produce the target image.

[0143] The environmental feature model can be obtained by modeling environmental information. The environmental information includes at least one of the following: point cloud data from a laser radar, image data from an image sensor, and feedback data from a millimeter-wave radar.

[0144] Because the environmental feature model can be a 3D model, the rendering perspective can be changed. The rendering process can be considered to simulate a virtual camera, presenting the environmental feature model through the imaging perspective of this virtual camera. The imaging perspective of this virtual camera is the rendering perspective. The target rendering perspective can be the same as or different from the preset rendering perspective used when creating the rendering image of the vehicle's surrounding environment, and this is not limited in this embodiment of the application.

[0145] Referring to the rendering process of the vehicle's surrounding environment, a new virtual camera can be determined based on the location of the risk object, its actual size, and the display area of ​​the floating window. The imaging perspective of this new virtual camera is the target rendering perspective. The environmental feature model is then rendered using this virtual camera to obtain the target image. The position of this new virtual camera can be the same as or different from the position of the virtual camera corresponding to the rendering of the vehicle's surrounding environment.

[0146] For example, the preset distance can be determined based on the actual size of the risk object and the display area of ​​the floating window. When the distance between the virtual camera and the risk object in the environmental feature model is less than the preset distance, the risk object in the rendered image is displayed in the floating window, and its proportion in the floating window is greater than or equal to the minimum value in the preset ratio interval. Then, a target distance is determined within a range less than the preset distance, and a new virtual camera is determined based on the position of the risk object and the target distance. A mapping relationship between the position of the risk object and the orientation of the virtual camera can be preset in the vehicle computer, and the vehicle computer determines the orientation corresponding to the position of the risk object in the mapping relationship as the target orientation. The new virtual camera is located at the target orientation and its distance from the risk object is the target distance. The presentation angle of the new virtual camera is the target rendering angle.

[0147] For example, assuming the preset distance is 5 meters, the target distance can be determined to be 4 meters. If the risk object is located in front of the vehicle, the target orientation can be determined to be a 45-degree angle in front of the vehicle object in the environmental feature model. The target rendering angle is: a 45-degree angle in front of the vehicle object in the environmental feature model, and a distance of 4 meters from the risk object in the environmental feature model.

[0148] This implementation method determines that the clarity of the target image obtained by the rendering perspective is high, and the risk object can be displayed more clearly in the subsequent display, making it easier for users to observe and identify risk objects, grasp the information of risk objects, and make correct decisions. Please refer to Figure 5, which is a schematic diagram of the rendering perspective of an object provided in an embodiment of the present application. Figure 5 shows three objects A, B and C, and the virtual camera is represented by an eye. If the virtual camera is in position 1, the rendering perspectives of the three objects A, B and C are a, b and c respectively. As can be seen from Figure 5, under the perspective of the virtual camera, the observation effect of object A is the best, object B is second, and the observation effect of object C is poor.

[0149] Taking Figure 5 as an example, assume that object A represents a vehicle, object C represents the risky object identified in process 201, and position 1 represents the preset rendering perspective. When rendering the environmental feature model of vehicle A, if the preset rendering perspective is used, the rendering perspective of the area where object C is located will be c. This will result in a poor rendering effect of the risky object in the rendered image. Displaying the rendered image in a floating window may prevent the user from clearly and intuitively observing object C.

[0150] In this second implementation, when rendering the environmental feature model of vehicle A, position 2 can be determined as the position of the virtual camera, that is, perspective d can be determined as the target rendering perspective. Perspective d provides a better rendering effect for object C than perspective c, thereby improving the presentation of object C. After the rendering is displayed in a floating window, the user can clearly and intuitively observe object C.

[0151] In the third implementation, the target image is captured by a target image sensor with adjusted parameters. The target image includes a realistic view of the area where the risk object is located. Based on the location of the risk object, the vehicle computer determines the target image sensor and adjusts its imaging parameters. The image captured by the target image sensor using the adjusted imaging parameters is then designated as the target image.

[0152] In process 201, the vehicle computer can identify risky objects around the vehicle through images. In process 203, the image sensor corresponding to the real-life image displaying the risky object can be identified as the target image sensor. Alternatively, the target image sensor can be determined based on the location of the risky object to better visualize the risky object. Target image sensors include, but are not limited to, cameras located close to the risky object, cameras with a greater depth of field, and macro cameras.

[0153] For example, if a vehicle identifies a hazardous object as a curb during a sharp turn, the target image sensor can be a camera mounted on the rear of the vehicle. As another example, images captured by a camera mounted on the vehicle in the target direction can be used. The target direction is the direction of the hazardous object relative to the vehicle. If the hazardous object is on the left side of the vehicle, images captured by the left camera can be used; if the hazardous object is on the right side of the vehicle, images captured by the right camera can be used.

[0154] When adjusting the parameters of the target image sensor, the focal length of the target image sensor can be adjusted according to the position of the risk object to ensure that the risk object can be fully and clearly presented in the image captured by the target image sensor. When adjusting the focal length of the target image sensor, the position of the risk object in the real-life image that previously displayed the risk object can also be referred to based on the position of the risk object. For example, the vehicle computer can preset a mapping relationship between the position of the risk object (or the position of the risk object and the position of the risk object in the real-life image that previously displayed the risk object) and the position of the image sensor and the focal length. The vehicle computer determines the position of the identified risk object (or the position of the risk object and the position of the risk object in the real-life image that previously displayed the risk object) and the position of the target image sensor, and the corresponding focal length in the mapping relationship. The focal length of the target image sensor is then adjusted to the aforementioned corresponding focal length.

[0155] This third implementation can be combined with the first implementation. For example, after acquiring an image captured by the target image sensor according to the adjusted camera parameters, a target magnification factor can be determined, and the acquired image can be magnified according to the target magnification factor to obtain the target image. This allows the image to be magnified when the image captured by the target image sensor is not clear enough, thereby improving the presentation of the target image.

[0156] When the distance between the vehicle and the identified risk object is close, the real-life image has a clearer display effect for the risk object. When the distance between the vehicle and the identified risk object is far, the rendered image has a clearer display effect for the risk object. In an embodiment of the present application, when the distance between the risk object and the vehicle is less than a preset distance, the target image includes a real-life image of the area where the risk object is located, that is, the target image can be determined according to the first implementation method mentioned above, and the initial image is a real-life image showing the risk object, or the target image can be determined according to the third implementation method mentioned above. When the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendered image of the area where the risk object is located, that is, the target image can be determined according to the first implementation method mentioned above, and the initial image is a rendered image of the vehicle's surrounding environment, or the target image can be determined according to the second implementation method mentioned above. The preset distance can be, for example, 8 meters, 9 meters, 10 meters, 11 meters or 12 meters, etc., and this embodiment of the present application does not limit this.

[0157] For example, suppose a vehicle is turning and the identified risky object is the edge of the road. If the distance between the vehicle and the edge is less than a preset distance, the target image will include a real-life image of the area where the risky object is located. If the identified risky object is a plastic bag in the distance and the distance between the vehicle and the plastic bag is greater than a preset distance, the target image will include a rendered image of the area where the risky object is located.

[0158] 204. Display the target image in the floating window.

[0159] Optionally, the target image can be centered in the floating window. For example, please refer to Figure 6, which is a schematic diagram of displaying a target image in a floating window according to an embodiment of the present application. Figure 6 uses a rendering of the target image including the area where a risk object is located as an example, and assumes that the risk object is identified as a rock on the roadway. Figure 6 shows a rendering of the vehicle's surroundings and a floating window displayed on top of the rendering. Both the floating window and the risk object are located to the left of the vehicle object. As can be seen in Figure 6, in the rendering of the vehicle's surroundings, the model A01 corresponding to the risk object (rock) is a small black dot. This small black dot is difficult for the user to perceive, and even if they do perceive it, they cannot determine the specific nature of the risk object. Even if the model A01 is marked as a small blue dot according to existing techniques, it is still difficult for the user to perceive or determine the risk object. In this embodiment of the present application, a target image G is displayed in the floating window. Compared to the rendering of the vehicle's surroundings, target image G magnifies the area where the rock is located. Through target image G, the user can clearly see the rock and its location.

[0160] In an embodiment of the present application, a risk warning process can be executed in a loop. The risk warning process includes: identifying risk objects around the vehicle, and updating the target image of the area where the risk objects are located based on the latest location of the risk objects. The updated target image is then displayed in a floating window. This process can refer to the aforementioned processes 201, 203, and 204, and will not be described in detail in this embodiment of the present application. When it is determined that the risk object no longer poses a risk to driving, the floating window corresponding to the risk object can be stopped from being displayed, and the subsequent loop of the risk warning process can be stopped.

[0161] In this way, the target image displayed in the floating window changes in real time. For example, when a vehicle is in motion, it is constantly moving, so the position of the risk object identified in process 201 (relative to the vehicle) is also constantly changing. Even if the risk object itself is dynamically moving, its position will change. The vehicle can update the target image in the floating window in real time. When the position of the risk object changes, the magnification and viewing angle of the target image in the floating window will also change accordingly compared to the initial image, so that the identified risk object is always displayed in the floating window.

[0162] In summary, the driving risk warning method provided by the embodiments of the present application displays a floating window on the vehicle's display screen when a risky object is identified near the vehicle. Based on the risky object's location, a target image of the area where the risky object is located is then determined and displayed in the floating window. The target image includes a rendered or real-life image of the area where the risky object is located, and the proportion of the risky object in the target image (equivalent to the proportion of the risky object in the floating window) is within a preset ratio range. On the one hand, the display of the floating window provides a clear warning, allowing users to perceive the appearance of the floating window as equivalent to the presence of a risky object, thereby enhancing their risk awareness. On the other hand, the proportion of the risky object in the floating window is greater than the proportion of the risky object in the rendered image of the vehicle's surroundings, allowing users to more clearly observe and identify the risky object through the floating window, grasp its information, and make appropriate decisions. Furthermore, even if the vehicle is far away from the risky object, the user can still observe and grasp the risky object in a timely manner through the floating window, providing the user with ample time to make appropriate driving decisions regarding the risky object. Compared to related technologies, this effectively reduces the probability of dangerous situations caused by hasty decisions and improves driving safety.

[0163] Please refer to FIG7 , which is a flowchart of another driving risk prompting method provided by an embodiment of the present application. This method can be applied to a cockpit or a vehicle, such as the vehicle shown in FIG1 and FIG2 . The method may include the following steps:

[0164] 301. Identify risky objects around the vehicle.

[0165] This process can refer to the aforementioned process 201, and will not be described in detail in this embodiment of the present application.

[0166] 302. Display a floating window on a display screen of the vehicle.

[0167] This process can refer to the aforementioned process 202, and will not be described in detail in this embodiment of the present application.

[0168] 303. Determine a target image of the area where the risk object is located based on the location of the risk object. The target image includes a rendering or a real-life image of the area where the risk object is located. The proportion of the risk object in the target image is within a preset ratio range.

[0169] This process can refer to the aforementioned process 203, and will not be described in detail in this embodiment of the present application.

[0170] 304. Display the target image in the floating window.

[0171] This process can refer to the aforementioned process 204, and will not be described in detail in this embodiment of the present application.

[0172] For dynamic risk objects, the following processes 305 and 306 may also be performed.

[0173] 305. Determine risk information, where the risk information includes at least one of the following: a predicted driving trajectory of the vehicle, a predicted motion trajectory of the risk object, a predicted collision point between the vehicle and the risk object, a distance between the predicted collision point and the vehicle, a position at which the risk object is closest to the vehicle, and a risk level of the risk object.

[0174] As for the predicted driving trajectory of the vehicle, one example can be obtained through the vehicle's navigation system. If the vehicle uses the navigation system for route navigation, the future driving trajectory of the vehicle can be directly obtained from the navigation system. As another example, the predicted driving trajectory of the vehicle can be obtained by predicting the lane in which the vehicle is currently traveling. For example, when there is no intersection in front of the vehicle's current driving lane, the front of the current driving lane is directly used as the predicted driving trajectory of the vehicle. When there is an intersection in front of the vehicle's current driving lane, if the current driving lane is a straight lane, the front of the current driving lane is used as the predicted driving trajectory of the vehicle. If the current driving lane is a turning lane, the predicted driving trajectory of the vehicle is determined based on the intersection lane corresponding to the turning lane. As another example, the predicted driving trajectory of the vehicle can also be obtained based on the vehicle's historical driving trajectory, information obtained from the vehicle, and information such as driver behavior.

[0175] The predicted trajectory of a risky object can be derived from its spatial position over a historical period. For example, the risky object's spatial position over a historical period can be combined with inertia to predict its trajectory. The spatial position of the risky object over a historical period can be derived from information acquired by the vehicle.

[0176] Optionally, the risk information may include the vehicle's predicted driving trajectory and the predicted motion trajectory of the risky object. By displaying the vehicle's predicted driving trajectory and the predicted motion trajectory of the risky object, the user can directly identify the collision risk of the risky object and take timely evasive measures.

[0177] Furthermore, it is possible to determine whether there is a spatial intersection between the vehicle's predicted driving trajectory and the risk object's predicted motion trajectory, thereby determining whether the vehicle and the risk object will collide. If there is a spatial intersection, the vehicle and the risk object will collide; if there is no spatial intersection, the vehicle and the risk object will not collide.

[0178] If the vehicle and the risk object will not collide, but the vehicle and the risk object may come into close contact in the future, the risk information may also include the location where the risk object is closest to the vehicle. This can remind the user to drive carefully when approaching this location to avoid a collision with the risk object.

[0179] If a collision is expected, the risk information may also include the predicted collision point between the vehicle and the risky object. The predicted collision point is the spatial intersection of the vehicle's predicted driving trajectory and the risky object's predicted motion trajectory. Furthermore, the risk information may also include the distance between the predicted collision point and the vehicle.

[0180] Regarding the risk level of a risky object, the greater the minimum distance between the risky object and the vehicle, the lower the risk level of the risky object. For example, if the risky object and the vehicle are likely to collide directly, the risk level of the risky object is level one (the highest level). If the minimum distance between the risky object and the vehicle is greater than 0 and less than 1 meter, the risk level of the risky object is level two. If the minimum distance between the risky object and the vehicle is greater than 1 meter, the risk level of the risky object is level three. The embodiments of this application do not limit the specific classification of risk levels.

[0181] 306. Display risk information in a floating window.

[0182] When risk information includes the vehicle's predicted trajectory and the risk object's predicted motion trajectory, if the target image is a rendered image of the area where the risk object is located, the environmental feature model can be directly invoked to generate and display the vehicle's predicted trajectory and the risk object's predicted motion trajectory in a floating window. If the target image is a real-world image of the area where the risk object is located, a user interface (UI) layer can be added to the floating window to display the vehicle's predicted trajectory and the risk object's predicted motion trajectory.

[0183] When the risk information includes the predicted collision point between the vehicle and the risky object, the distance between the predicted collision point and the vehicle, or the position at which the risky object is closest to the vehicle, the predicted collision point identifier, the distance between the predicted collision point and the vehicle, or the position at which the risky object is closest to the vehicle can be superimposed and displayed in the floating window as a UI layer or UI control.

[0184] When the risk information includes the risk level of the risk object, different types of colors or the same color at different depths can be displayed on the edge of the floating window to indicate different risk levels. Alternatively, a risk icon (such as an exclamation mark) can be displayed in the floating window, and different types of color shading can be displayed in the floating window to indicate different risk levels. The embodiment of the present application does not limit the form of displaying the risk level. Taking the risk level of the aforementioned process 305 as an example, when the risk level is level one, a dark red light band can be displayed on the edge of the floating window. When the risk level is level two, a yellow light band can be displayed on the edge of the floating window. When the risk level is level three, a green light band can be displayed on the edge of the floating window.

[0185] 307. Obtain the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object.

[0186] This process can refer to the aforementioned process 305, and will not be described in detail in this embodiment of the present application.

[0187] 308. Generate a driving animation of the vehicle based on the predicted driving trajectory.

[0188] The vehicle can simulate the movement of the vehicle based on the predicted driving trajectory of the vehicle, and then obtain the driving animation of the vehicle.

[0189] 309. Generate motion animation of the risk object based on the predicted motion trajectory.

[0190] The vehicle can simulate the motion of the object based on the predicted motion trajectory of the risk object, and then obtain the motion animation of the risk object.

[0191] 310. Display the vehicle's driving animation and the dangerous object's movement animation in the floating window.

[0192] If the target image is a rendering of the area where the risk object is located, the environmental feature model can be directly used to simulate the movement of the vehicle and the risk object. The rendered animations of the vehicle and the risk object can be displayed in a floating window. If the target image is a real-life image of the area where the risk object is located, a UI layer can be added to the floating window to display the vehicle and risk object logos. The vehicle and risk object logos move along their respective predicted trajectories, thus displaying the vehicle and risk object animations.

[0193] When a collision between a vehicle and a dangerous object occurs in the future, the vehicle's driving animation and the dangerous object's motion animation form a collision animation. Compared to simply presenting static information, the animation can more directly indicate the collision risk.

[0194] For example, please refer to Figure 8, which is a schematic display diagram of a floating window provided in an embodiment of the present application. Figure 8 takes the risk information including the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, and the predicted collision point between the risk object and the vehicle as an example for explanation. The risk object identified in Figure 8 is a motorcycle, and the motorcycle and the floating window are both located to the right of the vehicle object. Figure 8 shows the predicted driving trajectory B01 of the vehicle, the predicted motion trajectory B02 of the motorcycle, and the predicted collision point B03 between the vehicle and the motorcycle. Figure 8 also displays a warning sign at the predicted collision point B03. Through the floating window of Figure 8, the user can be shown locations where collision risks will occur.

[0195] Optionally, the above is explained using the example of a floating window prompting driving risks. On the basis of the aforementioned floating window, risk objects can also be prompted through voice and head up display (HUD) devices. HUD, also known as a head-up display system, refers to a driver-centered blind operation, multi-function instrument panel. HUD is a projection on the front windshield of the vehicle, which can display some information, such as current speed and navigation information. This allows the driver to see driving information from the projection without having to lower or turn his head, which reduces the probability of safety accidents to a certain extent.

[0196] Taking the HUD as an example, after determining the direction of a risk object relative to the vehicle, a risk icon can be displayed on the HUD. The risk icon is used to identify the direction of the risk object relative to the vehicle's current lane. For example, please refer to Figure 9, which is a schematic diagram of a HUD provided in an embodiment of the present application. The black dot displayed on the HUD in Figure 9 is a risk icon, indicating that the risk object is to the left of the vehicle's current lane.

[0197] It should be noted that when the distance between the risk object identified in process 301 and the vehicle is 0, there is no need to execute subsequent processes 302 to 308. When the distance between the risk object identified in process 301 and the vehicle is greater than 0, subsequent processes 302 to 308 can be executed.

[0198] The aforementioned processes 301 to 310 may be executed in a loop. While the vehicle is traveling, risky objects around the vehicle may be continuously identified. Each time a risky object is identified and the distance between the risky object and the vehicle is greater than 0, the aforementioned processes 302 to 310 are executed.

[0199] In summary, the driving risk warning method provided by the embodiments of the present application displays a floating window on the vehicle's display screen when a risky object is identified near the vehicle. Based on the risky object's location, a target image of the area where the risky object is located is then determined and displayed in the floating window. The target image includes a rendered or real-life image of the area where the risky object is located, and the proportion of the risky object in the target image (equivalent to the proportion of the risky object in the floating window) is within a preset ratio range. On the one hand, the display of the floating window provides a clear warning, allowing users to perceive the appearance of the floating window as equivalent to the presence of a risky object, thereby enhancing their risk awareness. On the other hand, the proportion of the risky object in the floating window is greater than the proportion of the risky object in the rendered image of the vehicle's surroundings, allowing users to more clearly observe and identify the risky object through the floating window, grasp its information, and make appropriate decisions. Furthermore, even if the vehicle is far away from the risky object, the user can still observe and grasp the risky object in a timely manner through the floating window, providing the user with ample time to make appropriate driving decisions regarding the risky object. Compared to related technologies, this effectively reduces the probability of dangerous situations caused by hasty decisions and improves driving safety.

[0200] It can also determine risk information and display it in a floating window. The risk information includes at least one of the following: the vehicle's predicted driving trajectory, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object. By displaying the vehicle's predicted driving trajectory and the predicted motion trajectory of the risk object, users can directly identify the collision risk of the risk object and take avoidance measures in a timely manner.

[0201] Furthermore, the system can generate a driving animation of the vehicle based on the predicted trajectory, and a motion animation of the risky object based on the predicted trajectory, and display both the vehicle's driving animation and the risky object's motion animation in a floating window. By displaying an animation of the collision between the vehicle and the risky object, the collision animation can more directly indicate the collision risk than simply presenting static information.

[0202] The order of the methods provided in the embodiments of the present application can be adjusted appropriately, and the process can be increased or decreased accordingly. Any method that can be easily thought of by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application, and the embodiments of the present application do not limit this.

[0203] The above mainly introduces the driving risk prompting method provided by the embodiment of the present application from the perspective of the device. It can be understood that in order to realize the above functions, the device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0204] The embodiment of the present application can divide the functional modules of the device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0205] Figure 10 is a block diagram of a driving risk reminder device provided in an embodiment of the present application. When each functional module is divided according to its function, the driving risk reminder device 400 may include an identification module 401, a display module 402, and an image determination module 403. For example, the driving risk reminder device may be a vehicle or vehicle-mounted system, or a chip therein, or other combination device or component having the functions of the aforementioned driving risk reminder device. The functions of each module of the driving risk reminder device are as follows:

[0206] Identification module 401, used to identify risk objects around the vehicle;

[0207] Display module 402, for displaying a floating window on the display screen of the vehicle;

[0208] An image determination module 403 is configured to determine a target image of the area where the risky object is located based on the location of the risky object. The target image includes a rendering or a real image of the area where the risky object is located, and the proportion of the risky object in the target image is within a preset ratio range.

[0209] The display module 402 is further configured to display the target image in the floating window.

[0210] In combination with the above scheme, the image determination module 403 is specifically used to: determine the position of the risk object in the initial image based on the position of the risk object, the initial image including a rendering of the vehicle's surrounding environment or a real-life image showing the risk object; determine the target magnification; enlarge the initial image according to the target magnification; and obtain the target image based on the position of the risk object in the initial image and the enlarged initial image.

[0211] In combination with the above solution, the image determination module 403 is specifically used to: determine the target magnification based on the distance between the risk object and the vehicle and a preset mapping relationship. The preset mapping relationship includes a mapping relationship between distance and magnification. In the preset mapping relationship, the greater the distance, the greater the magnification.

[0212] In combination with the above scheme, the image determination module 403 is specifically used to: determine the target magnification range based on the proportion of the risk object in the initial image, the display area of ​​the floating window and the preset ratio range, so that when the risk object in the initial image that is magnified according to the magnification within the target magnification range is displayed in the floating window, the proportion of the risk object in the floating window is within the preset ratio range; determine the target magnification ratio based on the target magnification range.

[0213] In combination with the above scheme, the target image includes a rendering of the area where the risk object is located, and the image determination module is specifically used to: obtain the vehicle's environmental feature model; determine the target rendering perspective based on the position of the risk object, the actual size of the risk object and the display area of ​​the floating window; render the environmental feature model according to the target rendering perspective to obtain the target image.

[0214] In combination with the above scheme, the target image includes a real-life image of the area where the risk object is located. The image determination module 403 is specifically used to: determine the target image sensor and adjust the camera parameters of the target image sensor based on the location of the risk object; and determine the image captured by the target image sensor according to the adjusted camera parameters as the target image.

[0215] In combination with the above scheme, when the distance between the risk object and the vehicle is less than the preset distance, the target image includes a real-life image of the area where the risk object is located; when the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendered image of the area where the risk object is located.

[0216] Figure 11 is a block diagram of another driving risk prompt device provided in an embodiment of the present application. Based on Figure 10, the device also includes: a risk information determination module 404, which is used to determine risk information, and the risk information includes at least one of the following: the predicted driving trajectory of the vehicle, the predicted motion trajectory of the risk object, the predicted collision point between the vehicle and the risk object, the distance between the predicted collision point and the vehicle, the position where the risk object is closest to the vehicle, and the risk level of the risk object; the aforementioned display module 402 is also used to display risk information in a floating window.

[0217] In combination with the above scheme, the risk information determination module 404 is also used to: obtain the predicted driving trajectory of the vehicle and the predicted motion trajectory of the risk object; generate the vehicle's driving animation based on the predicted driving trajectory; generate the risk object's motion animation based on the predicted motion trajectory; the aforementioned display module 402 is also used to display the vehicle's driving animation and the risk object's motion animation in a floating window.

[0218] In combination with the above solution, the display module 402 is specifically used to: determine the target position of the risk object relative to the vehicle within a preset time in the future; and display a floating window at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

[0219] In combination with the above scheme, the identification module 401 is also used to identify risk objects around the vehicle when the risk warning process is cyclically executed; the image determination module 403 is also used to update the target image of the area where the risk object is located according to the latest position of the risk object when the risk warning process is cyclically executed; the display module 402 is also used to display the updated target image in the floating window when the risk warning process is cyclically executed.

[0220] FIG12 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 500 can be a vehicle or a chip or functional module in a vehicle, or can be a vehicle-mounted system or a chip or functional module in a vehicle-mounted system. As shown in FIG12 , the electronic device 500 includes a processor 501, a transceiver 502, and a communication circuit 503.

[0221] The processor 501 is used to execute any step in the method embodiments shown in Figures 3 and 7, and when executing processes such as obtaining environmental information, it can choose to call the transceiver 502 and the communication line 503 to complete the corresponding operation.

[0222] Furthermore, the electronic device 500 may further include a memory 504 , wherein the processor 501 , the memory 504 and the transceiver 502 may be connected via a communication line 503 .

[0223] The transceiver 502 is used to communicate with other devices or other communication networks, such as Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc. The transceiver 502 can be a module, a circuit, a transceiver, or any device capable of implementing communication.

[0224] The transceiver 502 is mainly used for sending and receiving information, etc., and may include a transmitter and a receiver, which respectively send and receive information, etc.; operations other than sending and receiving information, etc. are implemented by the processor, such as determining the target image of the area where the risk object is located based on the location of the risk object.

[0225] The communication line 503 is used to transmit information between the components included in the electronic device 500.

[0226] In one design, the processor can be considered as the logic circuit and the transceiver as the interface circuit.

[0227] The memory 504 is used to store instructions, where the instructions may be computer programs.

[0228] It should be noted that the memory 504 can exist independently of the processor 501 or can be integrated with the processor 501. The memory 504 can be used to store instructions, program code, or some data. The memory 504 can be located inside the electronic device 500 or outside the electronic device 500, without limitation. The processor 501 is configured to execute the instructions stored in the memory 504 to implement the methods provided in the above embodiments of the present application.

[0229] In one example, processor 501 may include one or more processors, such as processor 0 and processor 1 in FIG. 12 .

[0230] As an optional implementation, the electronic device 500 includes multiple processors. For example, in addition to the processor 501 in FIG. 12 , it may also include a processor 507 .

[0231] As an optional implementation, the electronic device 500 further includes an output device 505 and an input device 506. For example, the input device 506 is a keyboard, a mouse, a microphone, a joystick, or the like, and the output device 505 is a display screen, a speaker, or the like.

[0232] It should be pointed out that the electronic device 500 can be a chip system or a device with a similar structure as shown in Figure 12. Among them, the chip system can be composed of chips, or it can include chips and other discrete devices. The actions, terms, etc. involved in the various embodiments of this application can refer to each other without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are only an example, and other names can also be used in the specific implementation without limitation. In addition, the component structure shown in Figure 12 does not constitute a limitation on the electronic device 500. In addition to the components shown in Figure 12, the electronic device 500 may include more or fewer components than those shown in Figure 12, or combine certain components, or arrange the components differently.

[0233] The processor and transceiver described in this application can be implemented on an integrated circuit (IC), an analog IC, a radio frequency integrated circuit, a mixed-signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductor (CMOS), N-type metal oxide semiconductor (NMOS), P-type metal oxide semiconductor (positive channel metal oxide semiconductor, PMOS), bipolar junction transistor (BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0234] Figure 13 is a schematic diagram of the structure of a driving risk reminder device provided in an embodiment of the present application. This driving risk reminder device can be applied to the scenarios described in the above method embodiments. For ease of illustration, Figure 13 only shows the main components of the driving risk reminder device, including a processor, memory, control circuitry, and input / output devices. The processor is primarily used to process communication protocols and communication data, execute software programs, and process software program data. The memory is primarily used to store software programs and data. The control circuitry is primarily used to provide power and transmit various electrical signals. The input / output devices are primarily used to receive user input and output data to the user.

[0235] When the driving risk warning device is a vehicle or a vehicle computer, the control circuit may be a motherboard. The memory may include a hard disk, RAM, ROM, or other storage media. The processor may include a baseband processor and a central processing unit. The baseband processor is primarily responsible for processing communication protocols and communication data, while the central processing unit is primarily responsible for controlling the entire driving risk warning device, executing software programs, and processing software program data. Input and output devices may include a display screen, keyboard, and mouse. The control circuit may further include or be connected to a transceiver circuit or transceiver, such as a network cable interface, for sending or receiving data or signals, such as for data transmission and communication with other devices. Furthermore, the control circuit may include an antenna for transmitting and receiving wireless signals for data / signal transmission with other devices.

[0236] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes computer program code. When the computer program code runs on a computer, it enables the computer to execute any of the methods described in the embodiments of the present application.

[0237] The present application also provides a computer-readable storage medium. All or part of the processes in the above-mentioned method embodiments can be completed by a computer or a device with driving risk warning capabilities executing a computer program or instructions to control the relevant hardware. The computer program or the set of instructions can be stored in the above-mentioned computer-readable storage medium. When executed, the computer program or the set of instructions can include the processes of the above-mentioned method embodiments. The computer-readable storage medium can be an internal storage unit of the vehicle or vehicle system in any of the above-mentioned embodiments, such as the hard disk or memory of the vehicle or vehicle system. The above-mentioned computer-readable storage medium can also be an external storage device of the above-mentioned vehicle or vehicle system, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the above-mentioned vehicle or vehicle system. Furthermore, the above-mentioned computer-readable storage medium can also include both the internal storage unit of the above-mentioned vehicle or vehicle system and an external storage device. The above-mentioned computer-readable storage medium is used to store the above-mentioned computer program or instructions and other programs and data required by the above-mentioned vehicle or vehicle system. The above-mentioned computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0238] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0239] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0240] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0241] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0242] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0243] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0244] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for prompting driving risks, characterized in that: The method comprises: Identify risk objects around the vehicle; Displaying a floating window on a display screen of the vehicle; Determine, according to the location of the risk object, a target image of the area where the risk object is located, wherein the target image includes a rendering or a real image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; The target image is displayed in the floating window.

2. The method according to claim 1, characterized in that The step of determining a target image of an area where the risk object is located according to the position of the risk object includes: Determining the position of the risk object in an initial image according to the position of the risk object, wherein the initial image includes a rendering of the surrounding environment of the vehicle or a real scene image showing the risk object; Determine the target magnification; Enlarging the initial image according to the target magnification; The target image is obtained based on the position of the risk object in the initial image and the enlarged initial image.

3. The method according to claim 2, characterized in that Determining the target magnification includes: The target magnification is determined according to the distance between the risk object and the vehicle and a preset mapping relationship, wherein the preset mapping relationship includes a mapping relationship between distance and magnification, and the larger the distance in the preset mapping relationship, the larger the magnification.

4. The method according to claim 2, characterized in that: Determining the target magnification includes: Determine a target magnification range according to the proportion of the risk object in the initial image, the display area of ​​the floating window, and the preset ratio range, so that when the risk object in the initial image magnified according to the magnification within the target magnification range is displayed in the floating window, the proportion in the floating window is within the preset ratio range; The target magnification is determined according to the target magnification range.

5. The method according to claim 1, characterized in that: The target image includes a rendering of the area where the risk object is located, and determining the target image of the area where the risk object is located according to the position of the risk object includes: Acquire an environmental feature model of the vehicle; Determining a target rendering viewing angle according to the position of the risk object, the actual size of the risk object, and the display area of ​​the floating window; The environmental feature model is rendered according to the target rendering perspective to obtain the target image.

6. The method according to claim 1, characterized in that The target image includes a real picture of the area where the risk object is located, and determining the target image of the area where the risk object is located according to the position of the risk object includes: According to the position of the risk object, determining a target image sensor and adjusting the imaging parameters of the target image sensor; An image captured by the target image sensor according to the adjusted imaging parameters is determined as the target image.

7. The method according to any one of claims 1 to 6, characterized in that: When the distance between the risk object and the vehicle is less than a preset distance, the target image includes a real picture of the area where the risk object is located; when the distance between the risk object and the vehicle is greater than or equal to the preset distance, the target image includes a rendering of the area where the risk object is located.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Determine risk information, the risk information including at least one of the following: a predicted driving trajectory of the vehicle, a predicted motion trajectory of the risk object, a predicted collision point between the vehicle and the risk object, a distance between the predicted collision point and the vehicle, a position at which the risk object is closest to the vehicle, and a risk level of the risk object; The risk information is displayed in the floating window.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Acquiring a predicted driving trajectory of the vehicle and a predicted motion trajectory of the risk object; generating a driving animation of the vehicle according to the predicted driving trajectory; generating a motion animation of the risk object according to the predicted motion trajectory; The driving animation of the vehicle and the movement animation of the risk object are displayed in the floating window.

10. The method according to any one of claims 1 to 9, characterized in that: The display screen of the vehicle displays a rendering of the surrounding environment of the vehicle, and the floating window is displayed on the display screen of the vehicle, including: Determine the target position of the risk object relative to the vehicle within a preset time in the future; The floating window is displayed at the target position of the vehicle object in the upper layer of the rendering of the vehicle's surrounding environment.

11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: The risk warning process is executed cyclically, and the risk warning process includes: identifying the risk objects around the vehicle; According to the latest position of the risk object, update the target image of the area where the risk object is located; The updated target image is displayed in the floating window.

12. A driving risk reminder device, characterized in that: The device comprises: Identification module, used to identify risk objects around the vehicle; A display module, used for displaying a floating window on a display screen of the vehicle; An image determination module, configured to determine, according to the position of the risk object, a target image of the area where the risk object is located, wherein the target image includes a rendering or a real image of the area where the risk object is located, and the proportion of the risk object in the target image is within a preset ratio range; The display module is further used to display the target image in the floating window.

13. A driving risk reminder device, characterized in that: The device comprises: one or more processors; a memory for storing one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A vehicle, characterized in that: The vehicle comprises: A driving risk reminder device as claimed in claim 12 or 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and when the program codes are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images