Information display method, device, storage medium and program product
By generating three-dimensional scene maps and reminder information, the problem of insufficient navigation information of existing assisted driving systems is solved, and more intuitive and accurate navigation information is provided, which improves driving safety and experience.
Patent Information
- Application Number
- CN202310574248.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-05-19
AI Technical Summary
The existing assisted driving system provides a small amount of navigation information and low accuracy, which is not intuitive enough to effectively express the current driving status and road conditions of the vehicle.
By obtaining positioning information and environmental perception information, a three-dimensional scene map is generated, and reminder information is generated based on different driving scenarios, and the three-dimensional guide map is integrated to provide more accurate, rich and intuitive assisted driving information.
It improves driving safety, improves driving experience and fun, provides more intuitive navigation information, and enhances the driver's sense of driving safety.
Smart Images

Figure CN116608879B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of navigation technology, and in particular to an information display method, device, storage medium and program product. Background Art
[0002] Assisted driving is when the driver is driving the vehicle, the car provides auxiliary support through equipment such as radar and camera, so that the driver can drive the vehicle on the road more easily and safely and arrive at the destination safely. Assisted driving includes: lane keeping assistance system, automatic parking assistance system, brake assistance, reversing assistance system and driving assistance system.
[0003] In the related art, the assisted driving system usually uses two-dimensional static information such as static icons and turn-by-turn (TBT) cards to prompt users to plan routes.
[0004] However, in the process of implementing the present application, the inventors found that there are at least the following problems in the prior art: the above method provides users with less information, lower accuracy, and is not intuitive enough. Summary of the invention
[0005] The embodiments of the present application provide an information display method, device, storage medium and program product to improve the comprehensiveness, accuracy and intuitiveness of auxiliary driving information, so as to enhance the driving experience and increase driving pleasure.
[0006] In a first aspect, an embodiment of the present application provides an information display method, comprising:
[0007] In response to a navigation request, acquiring positioning information and environmental perception information, and determining a three-dimensional scene graph according to the positioning information and the environmental perception information;
[0008] Determine the current driving scenario, and generate reminder information according to the driving scenario;
[0009] A three-dimensional guidance map is generated according to the reminder information and the three-dimensional scene map, and the three-dimensional guidance map is displayed.
[0010] In a possible design, determining a three-dimensional scene graph according to the positioning information and the environment perception information includes:
[0011] Inputting the environmental perception information into a deep learning model to obtain an initial model of each element corresponding to the environmental perception information;
[0012] Determine corresponding target vector data from a high-precision vector database according to the positioning information and the initial model; the high-precision vector database includes vector data corresponding to different positioning information collected in advance;
[0013] A triangular mesh of the initial model is generated according to the vector data, and a three-dimensional scene graph corresponding to the environmental perception information is programmatically generated according to the triangular mesh.
[0014] In a possible design, programmatically generating a three-dimensional scene graph corresponding to the environmental perception information according to the triangular mesh includes:
[0015] Programmatically generate an initial scene graph corresponding to the environmental perception information according to the triangular mesh;
[0016] If there is a target element with a high-precision model among the elements corresponding to the environmental perception information, the model of the target element in the initial scene graph is replaced with the corresponding high-precision model to obtain a three-dimensional scene graph corresponding to the environmental perception information.
[0017] In a possible design, generating reminder information according to the driving scenario includes:
[0018] If the current driving scene is a turning scene at an intersection or a lane changing scene, the turning direction is obtained, and a direction change guide line of a preset length that fits the ground is generated according to the turning direction; the direction change guide line includes a plurality of continuous arrows with gradually changing directions.
[0019] In a possible design, generating reminder information according to the driving scenario includes:
[0020] If the current driving scene is a traffic light scene, a three-dimensional model of the traffic light is generated when the distance between the vehicle and the traffic light is less than or equal to a preset distance; if the red light is currently on, the countdown timing information is obtained and displayed, and the transparency of the driving guide line is set to a first transparency; if the countdown timing is detected to be over, the transparency of the driving guide line is set to a second transparency; the first transparency is higher than the second transparency.
[0021] In a possible design, determining the current driving scene includes:
[0022] Performing distance detection on movable elements around the vehicle according to the positioning information and the environmental perception information to obtain distance information corresponding to each movable element;
[0023] If there is a target movable element whose distance information is less than the preset distance among the plurality of movable elements, it is determined that the current driving scene is a driving too close scene;
[0024] The generating reminder information according to the driving scenario includes:
[0025] If the current driving scene is a driving too close scene, a synchronously moving three-dimensional warning color block is generated according to the moving position of the target movable element.
[0026] In a possible design, determining the current driving scene includes:
[0027] Get the current driving speed and the speed limit of the current road section;
[0028] If the current driving speed exceeds the limited speed, the current driving scene is determined to be a speeding scene;
[0029] Accordingly, generating reminder information according to the driving scenario includes:
[0030] If the current driving scene is a speeding scene, a picture card of the current driving speed and the speed limit and a flashing warning color block are generated.
[0031] In a possible design, after displaying the three-dimensional guidance map, the method further includes:
[0032] Determining a search range according to the positioning information;
[0033] Acquiring operational data, and determining a plurality of points of interest within the search range based on the operational data;
[0034] The positions of the plurality of interest points in the three-dimensional guidance map are determined, and the identifiers of the corresponding interest points are respectively displayed at the plurality of interest positions.
[0035] In a second aspect, an embodiment of the present application provides an information display device, including:
[0036] A determination module, configured to obtain positioning information and environmental perception information in response to a navigation request, and determine a three-dimensional scene graph according to the positioning information and the environmental perception information;
[0037] A generation module, used to determine the current driving scene and generate reminder information according to the driving scene;
[0038] The display module is used to generate a three-dimensional guidance map according to the reminder information and the three-dimensional scene map, and display the three-dimensional guidance map.
[0039] In a third aspect, an embodiment of the present application provides an information display device, including: at least one processor and a memory;
[0040] The memory stores computer-executable instructions;
[0041] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method described in the first aspect and various possible designs of the first aspect.
[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect are implemented.
[0043] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect and various possible designs of the first aspect.
[0044] The information display method, device, storage medium and program product provided in this embodiment include responding to a navigation request, obtaining positioning information and environmental perception information, determining a three-dimensional scene graph based on the positioning information and environmental perception information, determining the current driving scene, and generating reminder information based on the driving scene, generating a three-dimensional guidance map based on the reminder information and the three-dimensional scene graph, and displaying the three-dimensional guidance map. The information display method provided in the embodiment of the present application generates a three-dimensional scene graph by sensing the surrounding environment, and combined with reminder information for different scenes, it can provide users with more accurate, rich and intuitive auxiliary driving information, thereby improving driving safety, enhancing driving experience, and increasing driving pleasure. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0046] Figure 1 A schematic diagram of an application scenario of the information display method provided in an embodiment of the present application;
[0047] Figure 2 A schematic diagram of a flow chart of an information display method provided in an embodiment of the present application;
[0048] Figure 3 A schematic diagram of a display interface in a straight driving scenario provided in an embodiment of the present application;
[0049] Figure 4 A schematic diagram of a display interface for a turning scene at an intersection provided in an embodiment of the present application;
[0050] Figure 5 A schematic diagram of a display interface in a lane change scenario provided in an embodiment of the present application;
[0051] Figure 6 Schematic diagram of the display interface in the traffic light scene provided in the embodiment of the present application Figure 1 ;
[0052] Figure 7 Schematic diagram of the display interface in the traffic light scene provided in the embodiment of the present application Figure 2 ;
[0053] Figure 8 Schematic diagram of the display interface in the scenario of a vehicle approaching too close provided in an embodiment of the present application Figure 1 ;
[0054] Fig. 9 Schematic diagram of the display interface in the scenario of a vehicle approaching too close provided in an embodiment of the present application Figure 2 ;
[0055] Fig.10 Schematic diagram of the display interface in the scenario of a vehicle approaching too close provided in an embodiment of the present application Figure 3 ;
[0056] Fig.11 A schematic diagram of a display interface in a speeding scenario provided in an embodiment of the present application;
[0057] Fig.12 A schematic diagram of a display interface for POI recommendation provided in an embodiment of the present application;
[0058] Fig.13 A schematic diagram of the structure of the information display device provided in the embodiment of the present application Figure 1 ;
[0059] Fig.14 A schematic diagram of the hardware structure of the information display device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0061] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0062] It should be noted that the information display method and device provided in the present application can be used in the field of navigation technology, and can also be used in any field other than the field of navigation technology. The application field of the information display method and device provided in the present application is not limited.
[0063] With the development of automobile intelligence, intelligent assisted driving scenarios will reflect more functionality and visual presentation content. In related technologies, during the driving process, the assisted driving system usually only prompts the user to plan the route with static icons, turn-by-turn navigation (TBT) cards, etc. Static icon cards carry too little information and cannot objectively express the current driving status of the vehicle, road conditions, and the relationship between the current vehicle and the real physical space. In this scenario, a more accurate visual presentation of the driving process is particularly important to improve driving safety, driving experience, and increase driving pleasure.
[0064] To solve the above technical problems, the inventors of the present application have found that by sensing various elements in the surrounding environment, a three-dimensional scene graph can be generated, and different reminder information can be generated for different scenes, so that the three-dimensional scene graph and the reminder information can be integrated and displayed to obtain more accurate, comprehensive and intuitive driving assistance information. Based on this, an embodiment of the present application provides an information display method.
[0065] Figure 1 Schematic diagram of application scenarios of the information display method provided in the embodiment of the present application. Figure 1 As shown, the vehicle is provided with a high-precision positioning unit, a camera, a radar and a vehicle computer. The high-precision positioning unit is used to obtain the positioning information of the vehicle, the camera and the radar are used to obtain the environmental perception information, and the vehicle computer is used to generate a three-dimensional guidance map based on the positioning information and the environmental perception information and display it through a display screen. Optionally, the radar may be a millimeter wave radar, an ultrasonic radar or a laser radar; the camera may include a forward camera and a surround view camera.
[0066] In the specific implementation process, the camera obtains the relevant feature information and location information of pedestrians, vehicles, zebra crossings, buildings, obstacles, etc. on the road. The mapping accuracy of the millimeter wave radar can reach the millimeter level, and the perception information of the surrounding environment can be refreshed within 5 milliseconds, and the farthest mapping can exceed 200 meters. By fusing the camera and the millimeter wave radar, the specific surrounding environment information is obtained, including pedestrians, trees, vehicles, zebra crossings, shoulders, road ridges, elevated roads, surrounding buildings and other elements that appear in various driving environments. In response to the navigation request, the vehicle computer obtains positioning information and environmental perception information, and determines a three-dimensional scene graph based on the positioning information and the environmental perception information; determines the current driving scene, and generates reminder information based on the driving scene; generates a three-dimensional guidance map based on the reminder information and the three-dimensional scene map, and displays the three-dimensional guidance map. The user can control the vehicle driving according to the three-dimensional guidance map. The information display method provided in the embodiment of the present application generates a three-dimensional scene map by sensing the surrounding environment, and combined with reminder information for different scenes, it can provide users with more accurate, rich and intuitive auxiliary driving information, thereby improving driving safety, enhancing driving experience, and increasing driving pleasure.
[0067] It should be noted that Figure 1 The scenario diagram shown is merely an example. The information display method and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.
[0068] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0069] Figure 2 The following is a flow chart of the information display method provided in the embodiment of the present application. Figure 2 As shown, the method includes:
[0070] 201. In response to a navigation request, obtain positioning information and environmental perception information, and determine a three-dimensional scene graph according to the positioning information and the environmental perception information.
[0071] The execution subject of this embodiment may be a vehicle or a vehicle computer.
[0072] In this embodiment, the navigation request may be a navigation application opening instruction, or a navigation instruction after inputting a destination, which is not limited in this embodiment. For example, when the vehicle computer opens the navigation application, a three-dimensional scene graph may be determined based on the positioning information obtained by the positioning unit and the environmental perception information obtained by sensors such as cameras and radars, and the three-dimensional scene graph may be displayed in real time through the vehicle display.
[0073] In some embodiments, in order to better fit the user's viewing angle, the three-dimensional scene graph can be a top-down and front-view perspective at a preset angle located above and behind the vehicle along the driving direction, such as Figure 3 As shown, the entire vehicle and a preset distance behind the vehicle are presented in the three-dimensional scene graph, and the vehicle can also be displayed in the center.
[0074] In some embodiments, obtaining positioning information and environmental perception information may include: obtaining positioning information through a positioning unit, and obtaining environmental perception information through a camera and a millimeter wave radar.
[0075] Specifically, positioning information can be obtained through the positioning fusion solution of the Global Navigation Satellite System (GNSS) and the Inertial Measurement Unit (IMU). In the process of obtaining environmental perception information, the positioning of the vehicle itself can be used as a reference, and the camera can be used to obtain relevant feature information and location information of pedestrians, vehicles, zebra crossings, buildings, obstacles, etc. on the road. The mapping accuracy of the millimeter-wave radar can reach the millimeter level, and the perception of the surrounding environment will be refreshed every 5 milliseconds, with the farthest mapping exceeding 200 meters. The specific surrounding environmental perception information can be obtained by fusing the image information obtained by the camera and the radar data obtained by the millimeter-wave radar, including pedestrians, trees, vehicles, zebra crossings, shoulders, road ridges, elevated roads, surrounding buildings and other elements (objects) that appear in the driving environment. Then, based on the environmental perception information, artistic processing can be added through the computer to draw a real-life 3D map that is highly matched with reality and more beautiful.
[0076] In some embodiments, determining a three-dimensional scene graph based on the positioning information and the environmental perception information may include: inputting the environmental perception information into a deep learning model to obtain an initial model of each element corresponding to the environmental perception information; determining corresponding target vector data from a high-precision vector database based on the positioning information and the initial model; the high-precision vector database includes vector data corresponding to different positioning information collected in advance; generating a triangular mesh of the initial model based on the vector data, and programmatically generating a three-dimensional scene graph corresponding to the environmental perception information based on the triangular mesh.
[0077] Specifically, after obtaining environmental perception information, the environmental perception information can be input into a pre-trained neural network deep learning model, and the models in the corresponding model library can be matched to the specific objects corresponding to the monitored environmental perception information for drawing. Then, the game engine can generate triangular meshes of surrounding buildings, lanes, green spaces and other elements based on the desensitized vector data provided by the map provider, and programmatically generate protrusions on the building surface and fine details such as lanes, roads, and elevated roads to present a three-dimensional scene map.
[0078] In some embodiments, in order to improve the realism of the three-dimensional scene graph, fine models of immovable elements such as buildings involved in the map can be collected in advance for replacement to obtain a more realistic three-dimensional scene graph. Specifically, programmatically generating a three-dimensional scene graph corresponding to the environmental perception information based on the triangular mesh can include: programmatically generating an initial scene graph corresponding to the environmental perception information based on the triangular mesh; if there is a target element with a high-precision model among the elements corresponding to the environmental perception information, then replacing the model of the target element in the initial scene graph with the corresponding high-precision model to obtain a three-dimensional scene graph corresponding to the environmental perception information.
[0079] Specifically, after obtaining the programmatically generated preliminary three-dimensional scene graph, the corresponding rough model in the preliminary three-dimensional scene graph can be replaced with the fine models of some landmark buildings to present a more realistic real-life mapping and obtain the final three-dimensional scene graph.
[0080] 202. Determine a current driving scenario, and generate reminder information according to the driving scenario.
[0081] 203. Generate a three-dimensional guidance map according to the reminder information and the three-dimensional scene map, and display the three-dimensional guidance map.
[0082] Specifically, after the user inputs the destination through the navigation application and the navigation application generates a navigation route based on the destination, the vehicle starts and the vehicle passes through different driving scenarios during driving, such as straight driving scenario, turning scenario, traffic light scenario, lane change scenario, speeding scenario, driving too close to an obstacle scenario, reversing scenario, parking scenario, etc. Different reminder information can be generated for different scenarios, and then the reminder information and the three-dimensional scene graph can be fused to obtain a three-dimensional guidance map, and the three-dimensional guidance map is displayed on the display screen.
[0083] In some embodiments, after determining different driving scenarios, different scenarios can also be divided into safety levels. Safety events can be divided into three categories: low risk, medium risk, and high risk, and different assisted driving display and driving guidance strategies are implemented respectively. Specifically, identification can be assigned to low and medium risk levels. In low risk level scenarios, the display interface displays a low risk identification, and in medium risk level scenarios, the display interface displays a medium risk identification, so that users can know the current degree of danger and enhance user alertness. For high-risk scenarios, such as when a vehicle collision occurs or other dangerous events occur, or when the vehicle system fails or freezes, the navigation can be paused.
[0084] In some embodiments, when a straight-ahead scene is involved, the vehicle (self-vehicle) is driving straight along the current square without any intersection or lane change indication, and a reminder message is generated according to the driving scene, which may include: if the current driving scene is a straight-ahead scene, a guide line is generated according to the positioning information of the vehicle, and the guide line may start from the front end of the vehicle and extend forward along the driving direction for a preset length (e.g., 100 meters). Optionally, the guide line may be a gradient strip line segment that gradually fades as the distance from the vehicle increases. The use of a color gradient display style can be adapted to the actual human eye observation habits, making it more intuitive and realistic, and improving the user experience.
[0085] Specifically, Figure 3 As shown, the vehicle always maintains lane guidance during driving, and the display style can be a strip of color that extends from the bottom of the vehicle forward and projects onto the lane, extending to 100m in front of the vehicle and gradually disappears.
[0086] In some embodiments, generating reminder information based on the driving scene may include: if the current driving scene is an intersection turning scene or a lane changing scene, obtaining the turning direction, and generating a direction change guide line of a preset length that fits the ground according to the turning direction; the direction change guide line includes multiple continuous arrows with gradually changing directions.
[0087] Specifically, Figure 3 As shown in the figure, the vehicle always keeps the lane guidance during driving. The display style can be a strip of color block projected forward from the bottom of the vehicle on the lane, extending to 100m in front of the vehicle and gradually disappearing, as shown in the figure. Figure 4 As shown in FIG. 1 , when entering a turning scene at an intersection, a continuous arrow can be superimposed on the gradient of the guide strip above the lane to guide the driver in the driving direction, such as Figure 5 As shown, when entering a lane change scenario, continuous arrows can be superimposed on the guide bar to indicate the lane change direction.
[0088] In some embodiments, generating reminder information based on the driving scene may include: if the current driving scene is a traffic light scene, generating a three-dimensional model of the traffic light when the distance between the vehicle and the traffic light is less than or equal to a preset distance; if the red light is currently on, obtaining and displaying countdown timing information, and setting the transparency of the driving guide line to a first transparency; if the countdown is detected to be over, setting the transparency of the driving guide line to a second transparency; the first transparency is higher than the second transparency.
[0089] Specifically, Figure 6 As shown in the figure, when the vehicle travels to a preset distance from the intersection (for example, 40m), a 3D traffic light prompt will appear in the vertically upper area of the horizontal center, and if there is a countdown information, it will be displayed synchronously below the traffic light; in order to warn users of the parking location, the word STOP can be displayed on the ground of the intersection with the pedestrian crossing line highlighted, and the opacity of the lane auxiliary guide line can be reduced. Figure 7 As shown, after the red light ends, the countdown seconds are completed, the red light turns off and the green light turns on, the STOP sign on the intersection ground disappears, and the lane auxiliary guide line is highlighted to indicate the forward route. The above settings of the position, color, and size of elements such as traffic lights and countdown devices can be adjusted according to actual needs, and this embodiment does not limit this.
[0090] In some embodiments, determining the current driving scene may include: performing distance detection on movable elements around the vehicle according to the positioning information and the environmental perception information to obtain distance information corresponding to each movable element; if there is a target movable element among the multiple movable elements whose distance information is less than a preset distance, then determining that the current driving scene is a scene of driving too close; accordingly, generating reminder information according to the driving scene may include: if the current driving scene is a scene of driving too close, generating a synchronously moving three-dimensional warning color block according to the moving position of the target movable element.
[0091] Specifically, Figure 8 As shown in the figure, based on positioning information and environmental perception information, the distance detection of movable elements such as vehicles and pedestrians around the vehicle can be performed. For people, video clips of human activities can be captured and then reproduced through three-dimensional computer simulation. When the distance to the pedestrian is less than the preset distance, a red gradient semi-transparent cube warning effect can be presented around the pedestrian and move with the pedestrian. Based on this, the dangerous area can be presented to the user more intuitively.
[0092] For scenarios where the vehicle ahead or the vehicle on the side is too close, such as Fig. 9 and Fig.10As shown, during the driving process of the car, the surrounding environment is sensed, data is collected for static and dynamic object recognition, detection and tracking, and calculation and analysis are performed in combination with map data to make a prediction of danger. Specifically, based on the laser radar installed on the car, lasers can be emitted to the outside. After encountering an object, the lasers are refracted back and received by the CMOS sensor, thereby measuring the distance from the body to the obstacle. When the surrounding vehicles are too close to the vehicle (when the distance between the vehicle and the surrounding vehicles is less than the preset distance) and there is a risk of collision, the vehicle body will turn red to warn, and a red alarm area (sector-shaped area) will appear in the too close area of the circular area at the bottom of the vehicle, and a triangular exclamation mark warning sign with a flashing animation can be displayed above the too close vehicle.
[0093] It should be noted that, in this embodiment, the setting of the color and shape of the display style can be adjusted according to actual needs, and this embodiment does not limit this.
[0094] In some embodiments, determining the current driving scene may include: obtaining the current driving speed and the speed limit of the current road section; if the current driving speed exceeds the speed limit, determining that the current driving scene is an overspeeding scene; accordingly, generating reminder information based on the driving scene may include: if the current driving scene is an overspeeding scene, generating a picture card and a flashing warning color block of the current driving speed and the speed limit.
[0095] Specifically, Fig.11 As shown, according to the road information (including the speed limit information of each road section) recorded in the navigation system, when the vehicle speed exceeds the speed limit of the current road section, the speed limit digital logo of the road section can be displayed in the vertical upper area of the horizontal center, and a red gradient flashing warning will appear on both sides of the screen from both sides to the center. The setting of the color and shape of the display style in this embodiment can be adjusted according to actual needs, and this embodiment does not limit this.
[0096] The information display method provided in this embodiment generates a three-dimensional scene map by sensing the surrounding environment, and combines reminder information for different scenes, so as to provide users with more accurate, rich and intuitive auxiliary driving information, thereby improving driving safety, enhancing driving experience and increasing driving pleasure.
[0097] In some embodiments, in order to facilitate users to quickly learn the purpose of surrounding places, POIs can be recommended for users. Based on the above embodiment, after step 203, step 204 can also be included: determining the search range according to the positioning information; obtaining operation data, and determining multiple points of interest within the search range according to the operation data; determining the locations of multiple points of interest in the three-dimensional guidance map, and displaying the corresponding points of interest at the multiple locations.
[0098] Specifically, a POI annotation exploration method based on a 3D map is used to obtain a navigation route request of the vehicle. According to the navigation route and operation data, a search range circle can be drawn around the navigation route with the vehicle positioning as the center and a preset length (for example, 3 km) as the radius. Within the range circle, multiple POI points are obtained in combination with user preferences and operation data, and are presented on the map interface in the form of a card sticker with perspective in combination with 3D buildings. No more than 4 points are displayed at the same time. The POI points correspond to target resources, and the target resources have a unique identifier.
[0099] Among them, Fig.12 As shown, the display position of the POI logo can be set on the surface of the buildings on both sides of the road, and presented through a card with a perspective view of the center of the road, showing the name, type and 3D icon of the POI point.
[0100] The method provided in this embodiment enables users to find places of interest more quickly and learn about the functions and uses of surrounding places, thereby improving the user's driving experience, by displaying the logos of multiple POI points of interest in a three-dimensional guidance map.
[0101] Fig.13 This is a schematic diagram of the structure of the information display device provided in the embodiment of the present application. Fig.13 As shown, the information display device 130 includes: a determination module 1301 , a generation module 1302 and a display module 1303 .
[0102] The determination module 1301 is used to obtain positioning information and environmental perception information in response to a navigation request, and determine a three-dimensional scene graph according to the positioning information and the environmental perception information;
[0103] A generating module 1302 is used to determine the current driving scene and generate reminder information according to the driving scene;
[0104] The display module 1303 is used to generate a three-dimensional guidance map according to the reminder information and the three-dimensional scene map, and display the three-dimensional guidance map.
[0105] The information display device provided in the embodiment of the present application generates a three-dimensional scene map by sensing the surrounding environment, and combines reminder information for different scenarios to provide users with more accurate, rich and intuitive auxiliary driving information, thereby improving driving safety, enhancing driving experience and increasing driving pleasure.
[0106] In some embodiments, the determination module 1301 is specifically configured to:
[0107] Inputting the environmental perception information into a deep learning model to obtain an initial model of each element corresponding to the environmental perception information;
[0108] Determine corresponding target vector data from a high-precision vector database according to the positioning information and the initial model; the high-precision vector database includes vector data corresponding to different positioning information collected in advance;
[0109] A triangular mesh of the initial model is generated according to the vector data, and a three-dimensional scene graph corresponding to the environmental perception information is programmatically generated according to the triangular mesh.
[0110] In some embodiments, the determination module 1301 is specifically configured to:
[0111] Programmatically generate an initial scene graph corresponding to the environmental perception information according to the triangular mesh;
[0112] If there is a target element with a high-precision model among the elements corresponding to the environmental perception information, the model of the target element in the initial scene graph is replaced with the corresponding high-precision model to obtain a three-dimensional scene graph corresponding to the environmental perception information.
[0113] In some embodiments, the generating module 1302 is specifically configured to:
[0114] If the current driving scene is a turning scene at an intersection or a lane changing scene, the turning direction is obtained, and a direction change guide line of a preset length that fits the ground is generated according to the turning direction; the direction change guide line includes a plurality of continuous arrows with gradually changing directions.
[0115] In some embodiments, the generating module 1302 is specifically configured to:
[0116] If the current driving scene is a traffic light scene, a three-dimensional model of the traffic light is generated when the distance between the vehicle and the traffic light is less than or equal to a preset distance; if the red light is currently on, the countdown timing information is obtained and displayed, and the transparency of the driving guide line is set to a first transparency; if the countdown timing is detected to be over, the transparency of the driving guide line is set to a second transparency; the first transparency is higher than the second transparency.
[0117] In some embodiments, the generating module 1302 is specifically used to: perform distance detection on movable elements around the vehicle according to the positioning information and the environment perception information, and obtain distance information corresponding to each movable element;
[0118] If there is a target movable element whose distance information is less than the preset distance among the plurality of movable elements, it is determined that the current driving scene is a driving too close scene;
[0119] The generating reminder information according to the driving scenario includes:
[0120] If the current driving scene is a driving too close scene, a synchronously moving three-dimensional warning color block is generated according to the moving position of the target movable element.
[0121] In some embodiments, the generating module 1302 is specifically used to: obtain the current driving speed and the limited speed of the current road section;
[0122] If the current driving speed exceeds the limited speed, the current driving scene is determined to be a speeding scene;
[0123] Accordingly, generating reminder information according to the driving scenario includes:
[0124] If the current driving scene is a speeding scene, a picture card of the current driving speed and the speed limit and a flashing warning color block are generated.
[0125] In some embodiments, the determination module 1301 is further used to: determine a search range according to the positioning information;
[0126] Acquiring operational data, and determining a plurality of points of interest within the search range based on the operational data;
[0127] The positions of the plurality of interest points in the three-dimensional guidance map are determined, and the identifiers of the corresponding interest points are respectively displayed at the plurality of interest positions.
[0128] The information display device provided in the embodiment of the present application can be used to execute the above-mentioned method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0129] Fig.14 A schematic diagram of the hardware structure of an information display device provided in an embodiment of the present application, which device may be a computer, a tablet device, a car computer, etc.
[0130] Device 140 may include one or more of the following components: a processing component 1401 , a memory 1402 , a power component 1403 , a multimedia component 1404 , an audio component 1405 , an input / output (I / O) interface 1406 , a sensor component 1407 , and a communication component 1408 .
[0131] The processing component 1401 generally controls the overall operation of the device 140, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1401 may include one or more processors 1409 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1401 may include one or more modules to facilitate the interaction between the processing component 1401 and other components. For example, the processing component 1401 may include a multimedia module to facilitate the interaction between the multimedia component 1404 and the processing component 1401.
[0132] The memory 1402 is configured to store various types of data to support operations on the device 140. Examples of such data include instructions for any application or method operating on the device 140, contact data, phone book data, messages, pictures, videos, etc. The memory 1402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0133] The power supply component 1403 provides power to the various components of the device 140. The power supply component 1403 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 140.
[0134] The multimedia component 1404 includes a screen that provides an output interface between the device 140 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1404 includes a front camera and / or a rear camera. When the device 140 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0135] The audio component 1405 is configured to output and / or input audio signals. For example, the audio component 1405 includes a microphone (MIC), and when the device 140 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1402 or sent via the communication component 1408. In some embodiments, the audio component 1405 also includes a speaker for outputting audio signals.
[0136] I / O interface 1406 provides an interface between processing component 1401 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.
[0137] The sensor assembly 1407 includes one or more sensors for providing various aspects of status assessment for the device 140. For example, the sensor assembly 1407 can detect the open / closed state of the device 140, the relative positioning of components, such as the display and keypad of the device 140, the sensor assembly 1407 can also detect the position change of the device 140 or a component of the device 140, the presence or absence of user contact with the device 140, the orientation or acceleration / deceleration of the device 140, and the temperature change of the device 140. The sensor assembly 1407 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1407 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1407 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0138] The communication component 1408 is configured to facilitate wired or wireless communication between the device 140 and other devices. The device 140 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1408 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1408 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0139] In an exemplary embodiment, the device 140 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.
[0140] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1402 including instructions, and the instructions can be executed by a processor 1409 of the device 140 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0141] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0142] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0143] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0144] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the information display method executed by the above information display device is implemented.
[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. An information display method, characterized in that: include: In response to a navigation request, acquiring positioning information and environmental perception information, and determining a three-dimensional scene graph according to the positioning information and the environmental perception information; Determine the current driving scenario, and generate reminder information according to the driving scenario; generating a three-dimensional guidance map according to the reminder information and the three-dimensional scene map, and displaying the three-dimensional guidance map; The determining of a three-dimensional scene graph according to the positioning information and the environment perception information includes: Inputting the environmental perception information into a deep learning model to obtain an initial model of each element corresponding to the environmental perception information; Determine corresponding target vector data from a high-precision vector database according to the positioning information and the initial model; the high-precision vector database includes vector data corresponding to different positioning information collected in advance; Generating a triangular mesh of the initial model according to the target vector data, and programmatically generating an initial scene graph corresponding to the environmental perception information according to the triangular mesh; If there is a target element with a high-precision model among the elements corresponding to the environmental perception information, the model of the target element in the initial scene graph is replaced with the corresponding high-precision model to obtain a three-dimensional scene graph corresponding to the environmental perception information.
2. The method according to claim 1, characterized in that The generating reminder information according to the driving scenario includes: If the current driving scene is a turning scene at an intersection or a lane changing scene, the turning direction is obtained, and a direction change guide line of a preset length that fits the ground is generated according to the turning direction; the direction change guide line includes a plurality of continuous arrows with gradually changing directions.
3. The method according to claim 1, characterized in that The generating reminder information according to the driving scenario includes: If the current driving scene is a traffic light scene, a three-dimensional model of the traffic light is generated when the distance between the vehicle and the traffic light is less than or equal to a preset distance; if the red light is currently on, the countdown timing information is obtained and displayed, and the transparency of the driving guide line is set to a first transparency; if the countdown timing is detected to be over, the transparency of the driving guide line is set to a second transparency; the first transparency is higher than the second transparency.
4. The method according to claim 1, characterized in that: Determining the current driving scene includes: Performing distance detection on movable elements around the vehicle according to the positioning information and the environmental perception information to obtain distance information corresponding to each movable element; If there is a target movable element whose distance information is less than the preset distance among the plurality of movable elements, it is determined that the current driving scene is a driving too close scene; The generating reminder information according to the driving scenario includes: If the current driving scene is a driving too close scene, a synchronously moving three-dimensional warning color block is generated according to the moving position of the target movable element.
5. The method according to claim 1, characterized in that Determining the current driving scene includes: Get the current driving speed and the speed limit of the current road section; If the current driving speed exceeds the limited speed, the current driving scene is determined to be a speeding scene; Accordingly, generating reminder information according to the driving scenario includes: If the current driving scene is a speeding scene, a picture card of the current driving speed and the speed limit and a flashing warning color block are generated.
6. The method according to claim 1, characterized in that After displaying the three-dimensional guidance map, the method further includes: Determining a search range according to the positioning information; Acquiring operational data, and determining a plurality of points of interest within the search range based on the operational data; The positions of the plurality of interest points in the three-dimensional guidance map are determined, and the identifiers of the corresponding interest points are respectively displayed at the plurality of interest positions.
7. An information display device, characterized in that: include: A determination module, configured to obtain positioning information and environmental perception information in response to a navigation request, and determine a three-dimensional scene graph according to the positioning information and the environmental perception information; A generation module, used to determine the current driving scene and generate reminder information according to the driving scene; A display module, configured to generate a three-dimensional guidance map according to the reminder information and the three-dimensional scene map, and display the three-dimensional guidance map; The determination module is specifically used to: input the environmental perception information into a deep learning model to obtain an initial model of each element corresponding to the environmental perception information; Determine corresponding target vector data from a high-precision vector database according to the positioning information and the initial model; the high-precision vector database includes vector data corresponding to different positioning information collected in advance; Generating a triangular mesh of the initial model according to the target vector data, and programmatically generating an initial scene graph corresponding to the environmental perception information according to the triangular mesh; If there is a target element with a high-precision model among the elements corresponding to the environmental perception information, the model of the target element in the initial scene graph is replaced with the corresponding high-precision model to obtain a three-dimensional scene graph corresponding to the environmental perception information.
8. An information display device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the information display method according to any one of claims 1 to 6.
9. A vehicle, characterized in that: include: The information display device as claimed in claim 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the information display method according to any one of claims 1 to 6 is implemented.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the information display method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Map data processing method and device, and three-dimensional map generation method and device
CN106780736A
Four-dimensional real scene traffic simulation vehicle overspeed whole-process tracking detection and alarm system and method
CN108919256A
Intersection recognition method based on GoogLeNet neural network
CN110688958A
Vehicle-mounted equipment information display method and device and vehicle
CN110775063A
Vehicle navigation method and device, electronic equipment and storage medium
CN112985432A
