Obstacle rendering method and device based on perception confidence coefficient

Abstract

The invention provides an obstacle rendering method based on perception confidence, and relates to the technical field of artificial intelligence, automatic driving and the like. The method comprises the following steps: acquiring environment sensing data of a vehicle; determining perception confidence of one or more obstacles around the vehicle according to the environment perception data; determining the display transparency of a first obstacle according to the perception confidence of the first obstacle in the one or more obstacles; and rendering and displaying the animation in combination with the display transparency of the first obstacle. According to the method and the device, the environmental perception data can be comprehensively utilized, noise and errors of obstacle display are reduced, the problem of HMI layer visual flash caused by unstable obstacle perception and detection at a bottom layer is reduced, and the driving safety is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to technical fields such as autonomous driving. Background Art

[0002] In the related technology, when a vehicle is driving, a perception display animation will be displayed based on the human-machine interface HMI device (such as the central control display screen). Among them, obstacles around the vehicle (which may be moving obstacles, such as other vehicles, or stationary obstacles, such as cones) become unstable and may jump or flicker.

[0003] Therefore, how to reduce the visual flicker problem at the HMI layer caused by the instability of the underlying perception and detection of obstacles, thereby improving driving safety, has become one of the important research directions. Summary of the invention

[0004] The present disclosure provides an obstacle rendering method based on perception confidence.

[0005] According to one aspect of the present disclosure, a method for rendering an obstacle based on perception confidence is provided, the method comprising:

[0006] Acquire the vehicle’s environmental perception data;

[0007] determining a perception confidence level of one or more obstacles located around the vehicle based on the environmental perception data;

[0008] Determining a display transparency of a first obstacle among the one or more obstacles based on a perceived confidence of the first obstacle;

[0009] Combined with the display transparency of the first obstacle, a display animation is rendered.

[0010] The present disclosure can comprehensively utilize environmental perception data, reduce noise and errors in obstacle display, reduce the visual flickering problem of the HMI layer caused by unstable obstacle detection in the underlying perception, and improve driving safety.

[0011] According to another aspect of the present disclosure, there is provided an obstacle rendering device based on perception confidence, comprising:

[0012] An acquisition module, used to acquire environmental perception data of the vehicle;

[0013] A first determination module, configured to determine a perception confidence of one or more obstacles located around the vehicle based on the environmental perception data;

[0014] A second determination module, configured to determine a display transparency of a first obstacle among the one or more obstacles based on a perceived confidence level of the first obstacle;

[0015] The rendering module is used to render and display the animation in combination with the display transparency of the first obstacle.

[0016] According to another aspect of the present disclosure, there is provided an electronic device, including at least one processor, and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the obstacle rendering method based on perception confidence of the embodiment of the first aspect of the present disclosure.

[0019] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the obstacle rendering method based on perception confidence according to the embodiment of the first aspect of the present disclosure.

[0020] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the obstacle rendering method based on perception confidence of the embodiment of the first aspect of the present disclosure.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flow chart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure;

[0024] Figure 2 is a flow chart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure;

[0025] Figure 3 is a flow chart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure;

[0028] Figure 6 is a flow chart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure;

[0029] Figure 7 is a structural diagram of an obstacle rendering device based on perception confidence according to an embodiment of the present disclosure;

[0030] Figure 8 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] The embodiments of the present disclosure relate to the fields of artificial intelligence technologies such as computer vision and deep learning.

[0033] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0034] Automatic driving generally refers to an automatic driving system, which uses advanced communication, computer, network and control technologies to achieve real-time and continuous control of trains. Modern communication means are used to directly face the train, which can realize two-way data communication between the train and the ground, with fast transmission rate and large amount of information. Subsequent tracking trains and control centers can know the exact location of the preceding train in time, making operation management more flexible, control more effective, and more adapted to the needs of automatic train driving.

[0035] The obstacle rendering method and device based on perception confidence of the present disclosure are described below with reference to the accompanying drawings.

[0036] Figure 1 is a flowchart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure, such as Figure 1 As shown, the method comprises the following steps:

[0037] S101, obtaining environmental perception data of the vehicle.

[0038] In some implementations, the vehicle's environmental perception data may be visual data, such as image data collected by a camera or sensor.

[0039] In some implementations, the vehicle's environmental perception data may be lidar data, such as lidar scanning the surrounding environment and collecting information on the location, shape, and distance of objects.

[0040] In other implementations, the vehicle's environmental perception data may also include global positioning GPS data. The global positioning system may provide the vehicle's location and speed information, or radar data, etc., which is not limited in this disclosure.

[0041] S102: Determine perception confidence of one or more obstacles around the vehicle based on the environmental perception data.

[0042] Perception confidence is a measure of the credibility or certainty of the existence of obstacles around the vehicle based on environmental perception data. It reflects the degree of trust the vehicle has in the perceived obstacle information. Perception confidence can be calculated based on environmental perception data.

[0043] For example, in some embodiments, when the vehicle's environmental perception data is image data, obstacles in the image are detected based on deep learning methods (such as convolutional neural networks), and feature matching and tracking algorithms are used to track the position and movement of obstacles between consecutive frames. Based on the results of target detection and tracking, the perception confidence of the obstacle is calculated.

[0044] For another example, in some implementations, when the vehicle's environmental perception data is multiple types of environmental perception data, a sensor fusion algorithm is used to fuse the environmental perception data from different sensors, and image processing, lidar processing and other technologies are used to extract obstacle-related features from the environmental perception data. Based on these features, machine learning or deep learning methods can be used to classify obstacles and assign one or more perception confidence levels to each obstacle.

[0045] S103: Determine a display transparency of a first obstacle among the one or more obstacles based on a perceived confidence level of the first obstacle.

[0046] In some implementations, the perceived confidence of an obstacle is one, that is, the perceived confidence only includes the first confidence. At this time, the first confidence can indicate the certainty of the existence of the obstacle. The greater the first confidence of the first obstacle, the lower the display transparency in the display animation, that is, the higher the opacity, the more obvious the first obstacle.

[0047] In some implementations, there are two perception confidence levels for an obstacle, that is, the perception confidence level includes a second confidence level and a third confidence level. In this case, the second confidence level represents the probability that the distance between the obstacle and the vehicle gradually decreases, and the third confidence level represents the probability that the distance between the obstacle and the vehicle gradually increases.

[0048] Optionally, if the second confidence level of the first obstacle is greater than the third confidence level, it means that the distance between the first obstacle and the vehicle is gradually decreasing. In order to improve the perception of the first obstacle, the display transparency of the first obstacle in the display animation needs to be gradually reduced so that the first obstacle gradually appears.

[0049] Optionally, if the second confidence level of the first obstacle is less than the third confidence level, it means that the distance between the first obstacle and the vehicle is gradually increasing. In order to reduce the visual flickering problem of the HMI layer caused by the instability of the underlying perception and detection of obstacles, the transparency of the first obstacle in the display animation needs to be gradually increased so that the first obstacle gradually disappears.

[0050] S104, rendering and displaying an animation in combination with the display transparency of the first obstacle.

[0051] In some implementations, the display animation may be an animation for navigation. In some implementations, the display animation may be an animation for displaying obstacles. The embodiments of the present disclosure are not limited to this.

[0052] The display animation is rendered according to the display transparency of the first obstacle in the display animation, and is displayed on a human-machine interface HMI device.

[0053] Optionally, in the embodiment of the present disclosure, the HMI device may be a display or a central control screen, etc.

[0054] In the embodiment of the present disclosure, environmental perception data of the vehicle is obtained; the perception confidence of one or more obstacles around the vehicle is determined according to the environmental perception data; the display transparency of the first obstacle is determined according to the perception confidence of the first obstacle among the one or more obstacles; and the display animation is rendered in combination with the display transparency of the first obstacle. The present disclosure can comprehensively utilize environmental perception data to reduce noise and errors in obstacle display, reduce the visual flicker problem of the HMI layer caused by the instability of the underlying perception detection of obstacles, and improve driving safety.

[0055] Figure 2 is a flowchart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure, such as Figure 2 As shown, the method comprises the following steps:

[0056] S201, obtaining environmental perception data of the vehicle.

[0057] S202: Determine perception confidence of one or more obstacles around the vehicle based on the environmental perception data.

[0058] For the introduction of step S201 to step S202, reference may be made to the relevant contents in the above embodiment, which will not be repeated here.

[0059] S203: Acquire a first mapping relationship between a first confidence level and transparency, where the first confidence level is used to indicate a probability of existence of an obstacle.

[0060] In some embodiments, when the perception confidence includes only the first confidence, the first confidence is used to indicate the probability of the existence of an obstacle, and the greater the first confidence, the greater the certainty of the existence of the obstacle.

[0061] Optionally, there is a first mapping relationship between the first confidence and the transparency, and the greater the first confidence, the lower the transparency in the perceived display animation, that is, the higher the opacity, and the more obvious the obstacle.

[0062] S204: Determine, based on the first mapping relationship, the target transparency corresponding to the first confidence level of the first obstacle.

[0063] In some implementations, the transparency corresponding to any first confidence level is a fixed value. For example, the first confidence level of the first obstacle is 80, and the corresponding transparency is 20; the first confidence level of the first obstacle is 20, and the corresponding transparency is 80.

[0064] In some implementations, the transparency corresponding to any first confidence level is a dynamic sequence that changes over time. For example, the display animation is 200 ms. The first confidence level of the first obstacle is 80, and the corresponding transparency is shown in Table 1.

[0065] Table 1

[0066] time 1-50ms 50-100ms 100-150ms 150-200ms transparency 20 30 20 30

[0067] Taking the display animation as 200ms as an example, the first confidence of the first obstacle is 20, and the corresponding transparency is shown in Table 2.

[0068] Table 2

[0069] time 1-50ms 50-100ms 100-150ms 150-200ms transparency 80 60 80 60

[0070] S205: Determine the target transparency as the display transparency of the first obstacle.

[0071] S206, rendering and displaying an animation in combination with the display transparency of the first obstacle.

[0072] For the introduction of step S205 to step S206, reference may be made to the relevant contents in the above embodiment, which will not be repeated here.

[0073] In the embodiment of the present disclosure, a first mapping relationship between a first confidence level and transparency is obtained, the first confidence level is used to indicate the probability of the existence of an obstacle, and the target transparency corresponding to the first confidence level of the first obstacle is determined based on the first mapping relationship. The present disclosure can reduce noise and errors in obstacle display, reduce the visual flickering problem of the HMI layer caused by the instability of the underlying perception and detection of obstacles, and improve driving safety.

[0074] Figure 3 is a flowchart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure, such as Figure 3 As shown, the method comprises the following steps:

[0075] S301, obtaining environmental perception data of the vehicle.

[0076] S302: Determine perception confidence of one or more obstacles around the vehicle based on the environmental perception data.

[0077] For the introduction of step S301 to step S302, reference may be made to the relevant contents in the above embodiment, which will not be repeated here.

[0078] It should be noted that, in the embodiment of the present disclosure, the perception confidence includes two confidences.

[0079] S303: Determine the existence state of the first obstacle according to the two perception confidences of the first obstacle.

[0080] In the embodiment of the present application, the perception confidence includes a second confidence and a third confidence, the second confidence represents the probability that the distance between the obstacle and the vehicle gradually decreases, and the third confidence represents the probability that the distance between the obstacle and the vehicle gradually increases.

[0081] In some embodiments, in response to the second confidence of the first obstacle being less than a preset first confidence threshold and the third confidence of the first obstacle being greater than the preset second confidence threshold, the existence state of the first obstacle is determined to be a second state in which the distance from the vehicle gradually increases.

[0082] In some embodiments, in response to the third confidence of the first obstacle being less than a preset third confidence threshold and the second confidence of the first obstacle being greater than a preset fourth confidence threshold, the existence state of the first obstacle is determined to be a first state in which the distance to the vehicle gradually decreases.

[0083] The first confidence threshold is smaller than the second confidence threshold, and the third confidence threshold is smaller than the fourth confidence threshold.

[0084] In some embodiments, in response to the second confidence level of the first obstacle being greater than the third confidence level, the existence state of the first obstacle is determined to be a first state in which the distance to the vehicle is gradually decreasing; in response to the second confidence level of the first obstacle being less than the third confidence level, the existence state of the first obstacle is determined to be a second state in which the distance to the vehicle is gradually increasing.

[0085] S304: Determine the display transparency of the first obstacle according to the existence status.

[0086] In some implementations, the transparency of the first obstacle at each moment in the display animation may be determined according to the existence state and the timing array, wherein the timing array includes the transparency at each moment in the display animation.

[0087] Optionally, in response to the existence state being the first state, a first time series array whose transparency decreases with time is determined from the transparency time series set, and the transparency of the first obstacle at each moment in the display animation is determined based on the first time series array. Taking the display animation as 200ms as an example, the first time series array can be as shown in Table 3.

[0088] Table 3

[0089] time 1-50ms 50-100ms 100-150ms 150-200ms transparency 60 50 40 30

[0090] Optionally, in response to the existence state being the second state, a second time series array in which transparency increases with time is determined from the transparency time series set, and the transparency of the obstacle at each moment in the perception display animation is determined based on the second time series array. Taking the perception display animation as 200ms as an example, the first time series array can be shown in Table 4.

[0091] Table 4

[0092] time 1-50ms 50-100ms 100-150ms 150-200ms transparency 80 85 90 95

[0093] In some implementations, the transparency of the first obstacle at each moment in the perception display animation may be determined according to the existence state and the dynamic effect curve, wherein the horizontal axis of the dynamic effect curve is time and the vertical axis is transparency.

[0094] Optionally, according to the existence state, a target animation curve matching the existence state may be determined from a set of candidate animation curves, and the transparency of the first obstacle at each moment in the perceived display animation may be determined based on the target animation curve.

[0095] In some embodiments, based on the existence state, a first motion curve of target monotonicity is determined from a set of candidate motion curves. Since of the two confidence levels, the higher confidence level has a greater impact on transparency, the maximum confidence level can be determined from the two confidence levels, and a second mapping relationship between the maximum confidence level and the first motion curve is obtained. Based on the second mapping relationship, a target motion curve corresponding to the maximum confidence level of the first obstacle is determined from the first motion curve.

[0096] For example, the display animation is 200ms, and the animation curve can indicate the display transparency of the first obstacle within 200ms.

[0097] like Figure 4 As shown, in response to the existence state being the first state, a first animation curve with monotonically decreasing transparency is determined from the candidate animation curve set. Figure 5 As shown, in response to the existence state being the second state, a first animation curve with monotonically increasing transparency is determined from the candidate animation curve set.

[0098] Among them, when the maximum confidence is the second confidence, the lower the second confidence, the smaller the curvature of the first motion effect curve; when the maximum confidence is the third confidence, the lower the third confidence, the greater the curvature of the first motion effect curve.

[0099] S305 , rendering and displaying an animation in combination with the display transparency of the first obstacle.

[0100] For the introduction of step S305, please refer to the content of the above embodiment, which will not be repeated here.

[0101] In the embodiment of the present disclosure, the existence state of the obstacle is determined according to the two perception confidences of the obstacle, and the transparency of the obstacle at each moment in the perception display animation is determined according to the existence state. The present disclosure can comprehensively utilize environmental perception data, reduce noise and errors in obstacle display, reduce the visual flicker problem of the HMI layer caused by the instability of the underlying perception detection of obstacles, and improve the perception accuracy and driving safety.

[0102] Figure 6 is a flowchart of an obstacle rendering method based on perception confidence according to an embodiment of the present disclosure, such as Figure 6 As shown, the method comprises the following steps:

[0103] S601, obtaining environmental perception data of the vehicle.

[0104] S602: Determine perception confidence of one or more obstacles around the vehicle according to the environmental perception data.

[0105] S603: Determine a display transparency of a first obstacle among the one or more obstacles based on a perceived confidence level of the first obstacle.

[0106] For the introduction of steps S601 to S603, reference may be made to the contents of the above embodiment, which will not be repeated here.

[0107] S604: Acquire a first in-vehicle position of the driver and a second in-vehicle position of the human-machine interface device.

[0108] Taking the central control screen as an example of the human-machine interface device, in some implementations, the driver's position in the car is on the left side, and the central control screen is on the right side in the car.

[0109] S605: Determine an edge display area for displaying the animation according to the first in-vehicle position and the second in-vehicle position.

[0110] In the embodiment of the present application, a target direction from the first position to the second position is obtained, and a display area with a preset proportion of the target direction in the display animation is used as an edge display area.

[0111] Taking the example that the driver is on the left side of the car and the central control screen is on the right side of the car, the display area of ​​the right 1 / 4 of the central control screen can be used as the edge display area for displaying animation.

[0112] S606: Overlay a canvas mask on the edge display area.

[0113] In some implementations, a canvas mask layer that matches the shape of the edge display area may be created according to the boundary of the edge display area, and then the canvas mask layer may be overlaid and rendered on the edge display area.

[0114] S607 , rendering and displaying an animation in combination with the display transparency of the first obstacle.

[0115] For the introduction of step S607, please refer to the content of the above embodiment, which will not be repeated here.

[0116] Since the peripheral vision of the human eye is more sensitive to moving objects, and the edge display area happens to be far away from the vehicle, the perception accuracy will be relatively low. Therefore, the obstacle display in the edge display area is weakened. By rendering the edge display area and adding a semi-transparent mask, information noise reduction can be performed to avoid visual flickering in the distance that may attract the driver's attention.

[0117] In the embodiment of the present disclosure, the first in-vehicle position of the driver and the second in-vehicle position of the human-machine interface device are obtained, and the edge display area for displaying the animation is determined according to the first in-vehicle position and the second in-vehicle position, and a canvas mask is superimposed on the edge display area. The present disclosure can further reduce the visual flickering problem of the HMI layer caused by the instability of the underlying perception and detection of obstacles, thereby improving driving safety.

[0118] Figure 7is a structural diagram of an obstacle rendering device based on perception confidence according to an embodiment of the present disclosure, such as Figure 7 As shown, the obstacle rendering device 700 based on perception confidence includes:

[0119] An acquisition module 710 is used to acquire environmental perception data of the vehicle;

[0120] A first determination module 720, configured to determine a perception confidence of one or more obstacles located around the vehicle according to the environmental perception data;

[0121] A second determination module 730, configured to determine a display transparency of a first obstacle among the one or more obstacles based on a perceived confidence level of the first obstacle;

[0122] The rendering module 740 is used to render and display the animation in combination with the display transparency of the first obstacle.

[0123] In some implementations, when the perception confidence includes only the first confidence, the second determination module 730 is further configured to:

[0124] Acquire a first mapping relationship between a first confidence level and transparency, where the first confidence level is used to indicate a probability of existence of an obstacle;

[0125] Determine, based on the first mapping relationship, a target transparency corresponding to a first confidence level of the first obstacle;

[0126] The target transparency is determined as the display transparency of the first obstacle.

[0127] In some implementations, when the perception confidence includes two confidences, the second determination module 730 is further configured to:

[0128] Determining the existence state of the first obstacle according to the two perception confidences of the first obstacle;

[0129] The display transparency of the first obstacle is determined according to the existence status.

[0130] In some embodiments, the perception confidence includes a second confidence and a third confidence, the second confidence represents the probability that the distance between the obstacle and the vehicle gradually decreases, and the third confidence represents the probability that the distance between the obstacle and the vehicle gradually increases. The second determination module 730 is further used to:

[0131] In response to the second confidence level of the first obstacle being less than the preset first confidence level threshold and the third confidence level of the first obstacle being greater than the preset second confidence level threshold, determining that the existence state of the first obstacle is the second state in which the distance from the vehicle is gradually increasing; or

[0132] In response to the third confidence of the first obstacle being less than a preset third confidence threshold and the second confidence of the first obstacle being greater than a preset fourth confidence threshold, determining that the existence state of the first obstacle is a first state in which the distance between the first obstacle and the vehicle is gradually decreasing;

[0133] The first confidence threshold is smaller than the second confidence threshold, and the third confidence threshold is smaller than the fourth confidence threshold.

[0134] In some implementations, the second determining module 730 is further configured to:

[0135] In response to the existence state being the first state, determining a first time series array in which transparency decreases over time from the transparency time series set, and determining the display transparency of the first obstacle based on the first time series array; or

[0136] In response to the existence state being the second state, determining a second time series array in which transparency increases with time from the transparency time series set, and determining the display transparency of the first obstacle based on the second time series array;

[0137] The time series array includes the transparency of the obstacle at each moment in the display animation.

[0138] In some implementations, the second determining module 730 is further configured to:

[0139] According to the existence state, determine a target animation curve matching the existence state from a set of candidate animation curves;

[0140] Determine the transparency of the obstacle at each moment in the obstacle perception display animation based on the target motion effect curve;

[0141] Among them, the horizontal axis of the target motion effect curve is time, and the vertical axis is transparency.

[0142] In some implementations, the second determining module 730 is further configured to:

[0143] According to the existence state, determine the first motion curve of the target monotonicity from the candidate motion curve set, and determine the maximum confidence from the two confidences;

[0144] Obtaining a second mapping relationship between the maximum confidence and the first motion effect curve;

[0145] Based on the second mapping relationship, determine the target dynamic effect curve corresponding to the maximum confidence level of the obstacle in the first dynamic effect curve;

[0146] Among them, when the maximum confidence is the second confidence, the lower the second confidence, the smaller the curvature of the first motion effect curve; when the maximum confidence is the third confidence, the lower the third confidence, the greater the curvature of the first motion effect curve.

[0147] In some implementations, the second determining module 730 is further configured to:

[0148] In response to the existence state being the first state, determining a first animation curve with monotonically decreasing transparency from the candidate animation curve set; or

[0149] In response to the existence state being the second state, a first animation curve with monotonically increasing transparency is determined from the candidate animation curve set.

[0150] In some implementations, the rendering module 740 is further configured to:

[0151] Acquire a first in-vehicle position of a driver and a second in-vehicle position of a human-machine interface device;

[0152] Determine an edge display area for displaying the animation according to the first in-vehicle position and the second in-vehicle position;

[0153] Overlay a canvas mask on the edge display area.

[0154] In some implementations, the rendering module 740 is further configured to:

[0155] Obtain a target direction from the first position to the second position;

[0156] A display area of ​​a preset proportion in the target direction in the display animation is used as the edge display area.

[0157] The present disclosure can comprehensively utilize environmental perception data, reduce noise and errors in obstacle display, reduce the visual flickering problem of the HMI layer caused by unstable obstacle detection in the underlying perception, and improve driving safety.

[0158] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0159] Figure 8 It is a block diagram of an electronic device for implementing an embodiment of the present disclosure. The electronic device can implement the obstacle rendering method based on perception confidence of the embodiment of the present disclosure, and the electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0160] like Figure 8As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0161] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0162] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the obstacle rendering method based on perception confidence. For example, in some embodiments, the obstacle rendering method based on perception confidence may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the obstacle rendering method based on perception confidence described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the obstacle rendering method based on perception confidence in any other appropriate manner (eg, by means of firmware).

[0163] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0165] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0167] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0168] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0169] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0170] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An obstacle rendering method based on perception confidence, wherein: include: Acquire the vehicle’s environmental perception data; determining a perception confidence level of one or more obstacles located around the vehicle based on the environmental perception data; determining, based on a perceived confidence level of a first obstacle among the one or more obstacles, a display transparency of the first obstacle; In combination with the display transparency of the first obstacle, an animation is rendered and displayed.

2. The method according to claim 1, wherein: In a case where the perception confidence includes only the first confidence, determining the display transparency of a first obstacle among the one or more obstacles based on the perception confidence of the first obstacle includes: Acquire a first mapping relationship between the first confidence and transparency, where the first confidence is used to indicate a probability that the obstacle exists; Determine, based on the first mapping relationship, a target transparency corresponding to a first confidence level of the first obstacle; The target transparency is determined as the display transparency of the first obstacle.

3. The method according to claim 1, wherein: In a case where the perception confidence includes two confidences, determining the display transparency of a first obstacle among the one or more obstacles based on the perception confidence of the first obstacle includes: Determining the existence state of the first obstacle according to the two perception confidences of the first obstacle; The display transparency of the first obstacle is determined according to the existence status.

4. The method according to claim 3, wherein: The perception confidence includes a second confidence and a third confidence, the second confidence indicates a probability that the distance between the first obstacle and the vehicle gradually decreases, and the third confidence indicates a probability that the distance between the first obstacle and the vehicle gradually increases, and the determining the existence state of the first obstacle according to the two perception confidences of the first obstacle includes: In response to the second confidence level of the first obstacle being less than a preset first confidence level threshold and the third confidence level of the first obstacle being greater than the preset second confidence level threshold, determining that the existence state of the first obstacle is a second state in which the distance between the first obstacle and the vehicle is gradually increasing; or In response to the third confidence of the first obstacle being less than a preset third confidence threshold and the second confidence of the first obstacle being greater than a preset fourth confidence threshold, determining that the existence state of the first obstacle is a first state in which the distance between the first obstacle and the vehicle is gradually decreasing; The first confidence threshold is smaller than the second confidence threshold, and the third confidence threshold is smaller than the fourth confidence threshold.

5. The method according to claim 4, wherein: The determining, according to the existence state, the display transparency of the first obstacle includes: In response to the existence state being the first state, determining a first time series array in which transparency decreases over time from a transparency time series set, and determining display transparency of the first obstacle based on the first time series array; or In response to the existence state being the second state, determining a second time series array in which transparency increases with time from the transparency time series set, and determining the display transparency of the first obstacle based on the second time series array; The time series array includes the display transparency of the obstacle at each moment in the display animation.

6. The method according to claim 4, wherein: The determining, according to the existence state, the display transparency of the first obstacle includes: According to the existence state, determining a target animation curve matching the existence state from a set of candidate animation curves; Determining the transparency of the first obstacle at each moment in the perception display animation based on the target motion effect curve; Among them, the horizontal axis of the target dynamic effect curve is time, and the vertical axis is transparency.

7. The method according to claim 6, wherein: The step of determining, according to the existence state, a target animation curve matching the existence state from a set of candidate animation curves includes: According to the existence state, determining a first target monotonic motion curve from the candidate motion curve set, and determining a maximum confidence from the two confidences; Acquire a second mapping relationship between the maximum confidence and the first motion effect curve; Based on the second mapping relationship, determining a target dynamic effect curve corresponding to the maximum confidence level of the first obstacle from the first dynamic effect curve; Among them, when the maximum confidence is the second confidence, the lower the second confidence, the smaller the curvature of the first motion effect curve; when the maximum confidence is the third confidence, the lower the third confidence, the greater the curvature of the first motion effect curve.

8. The method according to claim 7, wherein: The step of determining a first motion effect curve with target monotonicity from the candidate motion effect curve set according to the existence state includes: In response to the existence state being the first state, determining the first animation curve with monotonically decreasing transparency from the candidate animation curve set; or In response to the existence state being the second state, determining the first animation curve with monotonically increasing transparency from the candidate animation curve set.

9. The method according to any one of claims 1 to 8, wherein: Before rendering the display animation in combination with the display transparency of the first obstacle, the method further includes: Acquire a first in-vehicle position of a driver and a second in-vehicle position of a human-machine interface device; Determine an edge display area for displaying the animation according to the first in-vehicle position and the second in-vehicle position; A canvas mask is superimposed on the edge display area.

10. The method according to claim 9, wherein: The step of determining the edge display area for displaying the animation according to the first in-vehicle position and the second in-vehicle position includes: Acquire a target direction from the first position to the second position; A display area of ​​a preset proportion in the target direction in the display animation is used as the edge display area.

11. An obstacle rendering device based on perception confidence, wherein: include: An acquisition module, used to acquire environmental perception data of the vehicle; A first determination module, configured to determine a perception confidence of one or more obstacles located around the vehicle according to the environmental perception data; A second determination module, configured to determine a display transparency of a first obstacle among the one or more obstacles based on a perceived confidence level of the first obstacle; A rendering module is used to render and display an animation in combination with the display transparency of the first obstacle.

12. The device according to claim 11, wherein When the perception confidence includes only the first confidence, the second determination module is further configured to: Acquire a first mapping relationship between the first confidence and transparency, where the first confidence is used to indicate a probability that the obstacle exists; Determine, based on the first mapping relationship, a target transparency corresponding to a first confidence level of the first obstacle; The target transparency is determined as the display transparency of the first obstacle.

13. The device according to claim 11, wherein: When the perception confidence includes two confidences, the second determination module is further configured to: Determining the existence state of the first obstacle according to the two perception confidences of the first obstacle; The display transparency of the first obstacle is determined according to the existence status.

14. The device according to claim 13, wherein: The perception confidence includes a second confidence and a third confidence, the second confidence represents a probability that the distance between the obstacle and the vehicle gradually decreases, and the third confidence represents a probability that the distance between the obstacle and the vehicle gradually increases, and the second determination module is further used to: In response to the second confidence of the first obstacle being less than a preset first confidence threshold and the third confidence of the first obstacle being greater than the preset second confidence threshold, determining that the existence state of the first obstacle is a second state in which the distance between the first obstacle and the vehicle is gradually increasing; or In response to the third confidence of the first obstacle being less than a preset third confidence threshold and the second confidence of the first obstacle being greater than a preset fourth confidence threshold, determining that the existence state of the first obstacle is a first state in which the distance between the first obstacle and the vehicle is gradually decreasing; The first confidence threshold is smaller than the second confidence threshold, and the third confidence threshold is smaller than the fourth confidence threshold.

15. The device according to claim 14, wherein: The second determining module is further used for: In response to the existence state being a first state, determining a first time series array in which transparency decreases over time from a transparency time series set, and determining a display transparency of the first obstacle based on the first time series array; or In response to the existence state being the second state, determining a second time series array in which transparency increases with time from the transparency time series set, and determining the display transparency of the first obstacle based on the second time series array; The time series array includes the transparency of the obstacle at each moment in the display animation.

16. The device according to claim 14, wherein: The second determining module is further used for: According to the existence state, determining a target animation curve matching the existence state from a set of candidate animation curves; Determining the transparency of the first obstacle at each moment in the perception display animation based on the target motion effect curve; Among them, the horizontal axis of the target dynamic effect curve is time, and the vertical axis is transparency.

17. The device according to claim 16, wherein: The second determining module is further used for: According to the existence state, determining a first target monotonic motion curve from the candidate motion curve set, and determining a maximum confidence from the two confidences; Acquire a second mapping relationship between the maximum confidence and the first motion effect curve; Based on the second mapping relationship, determining a target dynamic effect curve corresponding to the maximum confidence level of the first obstacle from the first dynamic effect curve; Among them, when the maximum confidence is the second confidence, the lower the second confidence, the smaller the curvature of the first motion effect curve; when the maximum confidence is the third confidence, the lower the third confidence, the greater the curvature of the first motion effect curve.

18. The device according to claim 17, wherein: The second determining module is further used for: In response to the existence state being the first state, determining the first animation curve with monotonically decreasing transparency from the candidate animation curve set; or In response to the existence state being the second state, determining the first animation curve with monotonically increasing transparency from the candidate animation curve set.

19. The device according to any one of claims 11 to 18, wherein: The rendering module is further used for: Acquire a first in-vehicle position of a driver and a second in-vehicle position of a human-machine interface device; Determine an edge display area for displaying the animation according to the first in-vehicle position and the second in-vehicle position; A canvas mask is superimposed on the edge display area.

20. The device according to claim 19, wherein The rendering module is further used for: Acquire a target direction from the first position to the second position; A display area of ​​a preset proportion in the target direction in the display animation is used as the edge display area.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

Images