Display method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a display method and device, electronic equipment and a storage medium, the display method comprises the steps that driving information of a vehicle and / or information of the environment where the vehicle is located are / is acquired, the driving information comprises position information and / or time information, and the information of the environment where the vehicle is located comprises image information of the environment where the vehicle is located; determining atmosphere information related to vehicle driving according to the driving information and / or the information of the environment where the vehicle is located; according to the atmosphere information and an artificial intelligence model, generating corresponding virtual atmosphere data; and displaying the virtual atmosphere data on a head-up display, so that a virtual image displayed by the head-up display is fused with a real scene around the vehicle. According to the method, the head-up display displays the rendering atmosphere scene image based on the driving information of the vehicle and / or the information of the environment where the vehicle is located, so that the head-up display not only can display the basic data of the vehicle, such as the vehicle state and navigation, but also can display the rendering atmosphere scene image corresponding to the real scene.

Description

Technical Field

[0001] The present disclosure relates to the field of display technology, and in particular to a display method, device, electronic device and storage medium. Background Art

[0002] In related technologies, the vehicle head-up display can remind the driver of the vehicle status and navigation information, and one of the biggest features of the vehicle head-up display is the integration of virtual images and real scenes. However, the existing vehicle head-up display can usually only provide navigation AR element rendering in the center of the road, and cannot interact with the current surrounding driving scene (such as plants on both sides of the road, street lamps and other elements), nor can it realize the dynamic change of the surrounding environment scene. Summary of the invention

[0003] The purpose of the embodiments of the present disclosure is to provide a display method, device, electronic device and storage medium, which can display a rendered atmosphere scene image corresponding to a real scene.

[0004] The embodiments of the present disclosure adopt the following technical solutions: a display method is provided, comprising: obtaining driving information of a vehicle and / or information about the environment in which the vehicle is located, the driving information including location information and / or time information, and the information about the environment in which the vehicle is located including image information of the environment in which the vehicle is located; determining atmosphere information related to the driving of the vehicle based on the driving information and / or information about the environment in which the vehicle is located; generating corresponding virtual atmosphere data based on the atmosphere information and an artificial intelligence model; and displaying the virtual atmosphere data on a head-up display so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

[0005] In an exemplary embodiment, atmosphere information related to the driving of the vehicle is determined based on the driving information and / or information about the environment in which the vehicle is located, including: using a trained neural network model to process at least one of the driving information and the information about the environment in which the vehicle is located; and splicing at least part of the processing results as the atmosphere information related to the driving of the vehicle.

[0006] In an exemplary embodiment, the use of a trained neural network model to process at least one of the driving information and the environmental information includes at least one of the following: for the time information in the driving information, using a trained first neural network model unit to extract date-significant keywords corresponding to the time information; for the location information in the driving information, obtaining a historical environment image corresponding to the location information, and using a trained second neural network model unit to extract image feature data of the historical environment image; for the voice interaction information collected in the driving information, using a trained third neural network model unit to extract semantic keywords corresponding to the voice interaction information; for the image information of the vehicle's environment in the information about the vehicle's environment, using a trained fourth neural network model unit to extract scene feature keywords corresponding to the image information of the vehicle's environment.

[0007] In an exemplary embodiment, corresponding virtual atmosphere data is generated based on the atmosphere information and the artificial intelligence model, including: when the artificial intelligence model is an AIGC model, the atmosphere information is input into the AIGC model so that the AIGC model outputs data to be rendered of a real scene model corresponding to the atmosphere information; and the data to be rendered is rendered to obtain virtual atmosphere data corresponding to the real scene model.

[0008] In an exemplary embodiment, displaying the virtual atmosphere data on a head-up display includes: determining a posture of a real scene around a vehicle relative to eyes of a user in the vehicle; and displaying the virtual atmosphere data on the head-up display according to the posture of the real scene around the vehicle relative to eyes of the user in the vehicle.

[0009] In an exemplary embodiment, determining the posture of a real scene around a vehicle relative to the eyes of an in-vehicle user includes: acquiring the vehicle's position information and body posture data, as well as the sitting position of the in-vehicle user and position information of external environmental objects; and determining the position and posture of environmental objects in the real scene around the vehicle relative to the eyes of the in-vehicle user based on the vehicle's position information, body posture data, position information of external environmental objects and the sitting position of the in-vehicle user.

[0010] In an exemplary embodiment, the virtual atmosphere data is displayed on a head-up display according to the posture of the real scene around the vehicle relative to the eyes of the user in the vehicle, including: determining the field of view angle position of the environmental objects in the real scene relative to the eyes of the user in the vehicle according to the position of the environmental objects in the real scene around the vehicle relative to the eyes of the user in the vehicle; determining the perspective relationship between the virtual atmosphere data and the head-up display according to the field of view angle range and the posture of the environmental objects in the real scene around the vehicle relative to the eyes of the user in the vehicle; adjusting the parameters of the display model according to the field of view angle position and the perspective relationship, wherein the parameters of the display model include shape and / or size; and displaying the virtual atmosphere data on the adjusted display model.

[0011] In an exemplary embodiment, the method further includes: comparing the size of the display model with the size of the target real object in the real scene; and determining the display content and / or display position of the virtual atmosphere data according to the comparison result.

[0012] In an exemplary embodiment, the method further includes: acquiring control information sent by a user in the vehicle; adjusting the display of the virtual atmosphere data according to the control information; and / or sending interaction information to the user in the vehicle.

[0013] The disclosed embodiment also provides a display device, including: an acquisition module, used to acquire driving information of a vehicle and / or information about the environment in which the vehicle is located, the driving information including location information and / or time information, and the information about the environment in which the vehicle is located including image information of the environment in which the vehicle is located; a determination module, used to determine atmosphere information related to the driving of the vehicle based on the driving information and / or information about the environment in which the vehicle is located; a generation module, used to generate corresponding virtual atmosphere data based on the atmosphere information and an artificial intelligence model; and a display module, used to display the virtual atmosphere data on a head-up display, so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

[0014] The disclosed embodiment also provides a vehicle, comprising the above-mentioned display device.

[0015] An embodiment of the present disclosure further provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program in the memory.

[0016] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0017] According to the display method of the embodiment of the present disclosure, the driving information of the vehicle and / or the information of the environment in which the vehicle is located are obtained, the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located, and then the atmosphere information related to the driving of the vehicle is determined based on the driving information and / or the information of the environment in which the vehicle is located, and then the corresponding virtual atmosphere data is generated based on the atmosphere information and the artificial intelligence model, and the virtual atmosphere data is displayed on the head-up display so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle. The method realizes the display of the rendered atmosphere scene image on the head-up display based on the driving information of the vehicle and / or the information of the environment in which the vehicle is located, so that the head-up display can not only display basic vehicle data, such as vehicle status, navigation, etc., but also can display the rendered atmosphere scene image corresponding to the real scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 A hardware structure block diagram of a terminal device for a display method according to an embodiment of the present disclosure;

[0020] Figure 2 is a flow chart of a display method according to an embodiment of the present disclosure;

[0021] Figure 3 is a flow chart of a display method according to a specific embodiment of the present disclosure;

[0022] Figure 4 It is a structural block diagram of a display device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0025] According to one aspect of an embodiment of the present disclosure, a display method is provided. The display method can be executed in a terminal device or a similar computing device, taking running on a terminal device as an example. Figure 1 1 is a hardware structure block diagram of a terminal device of a display method according to an embodiment of the present invention. Figure 1 As shown, the terminal device may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the terminal device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal device. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the terminal device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the switching device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] In this embodiment, a display method is provided, which can be applied to the above-mentioned terminal device and can also be configured in a server. Figure 2 FIG. 1 is a flow chart showing a display method according to an embodiment of the present disclosure. Figure 2 As shown, the embodiment of the present disclosure provides a display method, including:

[0029] S210, obtaining driving information of the vehicle and / or information of the environment in which the vehicle is located, where the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located.

[0030] In the embodiments of the present disclosure, the location information in the driving information of the vehicle generally refers to the coordinate position of the vehicle in the geographic space. For example, the location information can be obtained through a satellite positioning system (such as a GPS system) on the vehicle.

[0031] In the embodiments of the present disclosure, the time information in the driving information of the vehicle includes but is not limited to time dimension information such as calendars and holidays.

[0032] In the embodiments of the present disclosure, the image information of the environment in which the vehicle is located can be understood as image data about the environment around the vehicle. The image data includes but is not limited to image information of objects such as roads, traffic signs, vehicles, pedestrians, and buildings. For example, the image information of the environment in which the vehicle is located can be collected in real time by a camera.

[0033] In one embodiment of the present disclosure, the vehicle's driving information may also include voice interaction information, wherein the vehicle's voice interaction information may be acquired through voice recognition technology, or the vehicle's driving information may also include other information without limitation.

[0034] S220, determining atmosphere information related to vehicle driving according to driving information and / or information about the environment in which the vehicle is located.

[0035] In the embodiments of the present disclosure, when the driving information of the vehicle and / or the information of the environment in which the vehicle is located is obtained, the driving information of the vehicle and / or the information of the environment in which the vehicle is located can be processed by the trained neural network model to determine the atmosphere information related to the driving of the vehicle. The specific implementation method can refer to the subsequent embodiments.

[0036] S230, generating corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model.

[0037] In an embodiment of the present disclosure, when determining the atmosphere information related to vehicle driving, corresponding virtual atmosphere data can be generated based on an artificial intelligence model (e.g., an AIGC model) and the atmosphere information. For specific implementation methods, reference can be made to subsequent embodiments. The virtual atmosphere data may include a video or image that matches the location of the external environment and / or the atmosphere of the current date.

[0038] S240, displaying the virtual atmosphere data on a head-up display so that the virtual image displayed on the head-up display is integrated with the real scene around the vehicle.

[0039] According to the display method of the embodiment of the present disclosure, the driving information of the vehicle and / or the information of the environment in which the vehicle is located are obtained, the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located, and then the atmosphere information related to the driving of the vehicle is determined based on the driving information and / or the information of the environment in which the vehicle is located, and then the corresponding virtual atmosphere data is generated based on the atmosphere information and the artificial intelligence model, and the virtual atmosphere data is displayed on the head-up display so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle. The method realizes the display of the rendered atmosphere scene image on the head-up display based on the driving information of the vehicle and / or the information of the environment in which the vehicle is located, so that the head-up display can not only display basic vehicle data, such as vehicle status, navigation, etc., but also can display the rendered atmosphere scene image corresponding to the real scene.

[0040] In order to make it easier for those skilled in the art to understand the present disclosure, Figure 3 As shown, Figure 3 is a flow chart of a display method according to a specific embodiment of the present disclosure. Figure 3 As shown, the display method includes:

[0041] S310, obtaining driving information of the vehicle and / or information of the environment in which the vehicle is located, where the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located.

[0042] In the embodiment of the present disclosure, the implementation of step S310 may refer to the implementation of the above-mentioned step S210, and the present disclosure will not elaborate on it here.

[0043] S320: Use the trained neural network model to process at least one of the driving information and the information of the vehicle's environment.

[0044] In an embodiment of the present disclosure, when the driving information of the vehicle and the information of the environment in which the vehicle is located are obtained, the trained first neural network model unit is used to extract date-significant keywords corresponding to the time information for the time information in the driving information.

[0045] For example, when the time information obtained in the driving information is October 1st, based on the trained first neural network model unit, the date meaning keywords corresponding to October 1st are extracted, such as the date meaning keywords including National Day, the founding of the People's Republic of China, the National Day holiday, the military parade, national prosperity, patriotism, and commemoration day. Alternatively, the date meaning keywords corresponding to February 14th can be extracted, such as Valentine's Day, etc.

[0046] With respect to the location information in the driving information, a historical environment image corresponding to the location information is obtained, and image feature data of the historical environment image is extracted using the trained second neural network model unit.

[0047] For example, when the location information in the driving information obtained is building A, the historical environment image corresponding to building A is obtained, and the shape features, color features, and texture features of the historical environment image are extracted based on the trained second neural network model unit. For another example, when the location information in the driving information obtained is an old street or ancient town, the historical environment image corresponding to the old street or ancient town can be obtained, and the sycamore trees, alleys, and old streets of the historical environment image are extracted based on the trained second neural network model unit.

[0048] Among them, obtaining the historical environment image corresponding to the location information can be achieved in the following ways: obtaining the driving images and / or videos of the current street view at a certain moment uploaded by other users stored in the first storage pool, and obtaining the historical street view data of the location and its surrounding areas collected through digitized historical images and maps and other documentary materials in the second storage pool.

[0049] For the voice interaction information collected in the driving information, the trained third neural network model unit is used to extract semantic keywords corresponding to the voice interaction information.

[0050] For example, when the voice interaction information obtained in the driving information is "I am on my way to the zoo", the semantic keyword that can be extracted is zoo.

[0051] For the image information of the vehicle's environment in the information of the vehicle's environment, the trained fourth neural network model unit is used to extract scene feature keywords corresponding to the image information of the vehicle's environment.

[0052] For example, when the acquired image information of the environment where the vehicle is located includes a building, a city hall, based on the trained fourth neural network model unit, the scene feature keyword extracted is the city hall.

[0053] S330, splicing at least part of the processing results as atmosphere information related to vehicle driving.

[0054] For example, after processing at least one of the driving information and the information of the environment in which the vehicle is located, date meaning keywords, image feature data, semantic keywords, etc. may be spliced ​​to form atmosphere information.

[0055] S340, generating corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model.

[0056] In an embodiment of the present disclosure, when the artificial intelligence model is an AIGC model, the atmosphere information is input into the AIGC model so that the AIGC model outputs the data to be rendered of the real scene model corresponding to the atmosphere information, and the data to be rendered is rendered to obtain virtual atmosphere data corresponding to the real scene model.

[0057] When the data to be rendered is obtained, the data to be rendered can be input into a rendering engine, so that the rendering engine outputs virtual atmosphere data corresponding to the real scene model, wherein the virtual atmosphere data includes a virtual atmosphere video.

[0058] S350: Determine the posture of the real scene around the vehicle relative to the eyes of the user in the vehicle.

[0059] In an embodiment of the present disclosure, the vehicle's position information and body posture data, as well as the sitting position of the user in the vehicle and the position information of external environmental objects are obtained; based on the vehicle's position information, body posture data, the position information of external environmental objects and the sitting position of the user in the vehicle, the position and posture of environmental objects in the real scene around the vehicle relative to the eyes of the user in the vehicle are determined.

[0060] For example, the vehicle's location information can be obtained through the vehicle's GPS system, the vehicle body posture data can be obtained through on-board sensors, the user's sitting position in the vehicle can be obtained through the cabin camera, and the location information of external environment objects can be obtained through millimeter waves and lidar.

[0061] S360 displays virtual atmosphere data on the head-up display based on the posture of the real scene around the vehicle relative to the eyes of the user in the car.

[0062] In an embodiment of the present application, the field of view angle position of the environmental objects in the real scene relative to the eyes of the user in the vehicle is determined according to the position of the environmental objects in the real scene of the vehicle's surrounding environment relative to the eyes of the user in the vehicle; the perspective relationship between the virtual atmosphere data and the head-up display is determined according to the field of view angle range and the posture of the environmental objects in the real scene of the vehicle's surrounding environment relative to the eyes of the user in the vehicle; the parameters of the display model are adjusted according to the field of view angle position and the perspective relationship, wherein the parameters of the display model include shape and / or size; and the virtual atmosphere data is displayed on the adjusted display model.

[0063] That is, the head-up display is usually located at the driver's eye level in front of the vehicle. By calculating the field of view angle range and the posture of the environmental objects in the real scene of the vehicle's surrounding environment relative to the eyes of the user in the vehicle, the perspective relationship of the virtual atmosphere data relative to the head-up display can be determined, and then the perspective relationship between the virtual atmosphere data and the head-up display is applied to the environmental objects in the real scene of the vehicle's surrounding environment. That is, by projecting the virtual atmosphere data according to the perspective relationship, the virtual atmosphere data can be integrated with the actual environmental objects, thereby achieving a realistic augmented reality effect.

[0064] In one embodiment of the present disclosure, control information sent by the user in the vehicle may be obtained, the display of the virtual atmosphere data may be adjusted according to the control information, and / or interaction information may be sent to the user in the vehicle.

[0065] For example, the user can determine the control information by sending a voice, and then the control information sent by the user can be obtained. For example, when a voice message of "enlarge the display information" is detected, the display of the virtual atmosphere data can be enlarged.

[0066] According to the first specific embodiment of the present disclosure, taking driving on the road to building A as an example, the location information in the driving information can be obtained first, and then the historical environment image corresponding to building A can be obtained, wherein obtaining the historical environment image corresponding to building A includes obtaining the driving images and / or videos of the current street view at a certain moment uploaded by other users stored in the first storage pool, and obtaining the historical street view data of building A and its surrounding areas collected through digitized historical images and maps and other documentary materials in the second storage pool; then the historical environment image is input into the trained neural network model so that the second neural network model unit in the neural network model extracts the image feature data of the historical environment image, and then the image feature data of multiple historical environment images are combined, and the combined result is used as the atmosphere information related to vehicle driving; then the atmosphere information is input into the AIGC model so that the AIGC model outputs the data to be rendered of the real scene model corresponding to the atmosphere information, and the data to be rendered is imported into a high-performance rendering engine to generate virtual atmosphere data corresponding to the real scene model.

[0067] Through data from external cameras, cockpit cameras, millimeter waves, lidar and other sensors, the vehicle's location information and body posture data, as well as the user's seat position in the car and the location information of external environmental objects are sensed, and then the position and posture of environmental objects in the real scene around the vehicle relative to the user's eyes in the car are determined. According to the posture of the real scene around the vehicle relative to the user's eyes in the car, the virtual atmosphere data is displayed on the head-up display, such as superimposing the virtual historical street scene on the real environment in the driver's field of view. For example, on the way to building A, the head-up display can display virtual ancient buildings, city walls and city gates in the driver's field of view. In this way, the driver can feel the fusion of history and reality, enhancing the fun and cultural connotation of the driving experience.

[0068] In order to enhance the interactive experience, the functions of the head-up display can also be controlled by touching the screen or using voice commands, such as selecting the display mode of the historical street view, adjusting the transparency and zoom of the display model, etc. In this way, the driver can better experience the historical street view and have a new experience of interacting with history.

[0069] According to the second specific embodiment of the present disclosure, taking driving on a road full of ginkgo trees on both sides of the road in autumn as an example, the location information in the driving information can be obtained first, for example, the location information includes collection side data and user side data, wherein the collection side data obtains the characteristics of both sides of the road through external cameras, millimeter waves, laser radars and other sensors and digital map data, satellite remote sensing images or field measurements, such as: the distribution, height, crown density and other information of ginkgo trees, and the user side data collects user-side atmosphere demand prompt words through natural language interaction, such as: "romantic atmosphere", "falling leaves", "fruitful", etc., and then the location information in the driving information is processed through a neural network model to determine the atmosphere information related to vehicle driving; then the atmosphere information is input into the AIGC model so that the AIGC model outputs the data to be rendered of the real scene model corresponding to the atmosphere information, and the data to be rendered is imported into a high-performance rendering engine to generate virtual atmosphere data corresponding to the real scene model. For example, in order to increase the virtual atmosphere, a seasonal change effect can be added to the head-up display to simulate the effects of falling leaves and color changes of ginkgo trees on both sides of the road in autumn. At the same time, the rendering engine combines the vehicle speed information to adjust the rendering image to a suitable look and feel; through the high-performance rendering engine, a realistic virtual natural environment atmosphere video based on the model generated by the previous process is generated.

[0070] Through the data of external cameras, cockpit cameras, millimeter waves, laser radars and other sensors, the vehicle's position information and body posture data, as well as the sitting position of the user in the car and the position information of the external environment objects are sensed, and then the position and posture of the environmental objects in the real scene around the vehicle relative to the human eyes of the user in the car are determined. According to the posture of the real scene around the vehicle relative to the human eyes of the user in the car, the virtual atmosphere data is displayed on the head-up display. For example, when driving on a road full of ginkgo trees on both sides of the road in autumn, the head-up display presents golden leaves on both sides and the effect of colorful fallen leaves. According to the third specific embodiment of the present disclosure, taking driving on the road at night as an example, the position information in the driving information can be obtained first, for example, the position information includes collected astronomical data, wherein the astronomical data includes the constellation information of the northern and southern hemispheres, astronomical star maps, star positions, brightness and other data, which can be obtained through astronomy software, Internet astronomy websites, etc. Afterwards, the position information in the driving information is processed by a neural network model to determine the atmosphere information related to the vehicle driving; then the atmosphere information is input into the AIGC model so that the AIGC model outputs the data to be rendered of the real scene model corresponding to the atmosphere information, and the data to be rendered is imported into a high-performance rendering engine to generate virtual atmosphere data corresponding to the real scene model. For example, in order to increase the sense of virtual atmosphere, you can add light and shadow effects to the head-up display to simulate the effects of meteors passing by. At the same time, the rendering engine combines the vehicle speed information to adjust the rendering image to a suitable look and feel. A high-performance rendering engine generates realistic virtual astronomical videos based on the model generated by the previous process.

[0071] Through the data of external cameras, cockpit cameras, millimeter waves, laser radars and other sensors, the vehicle's position information and body posture data, as well as the sitting position of the user in the car and the position information of external environmental objects are sensed, and then the position and posture of the environmental objects in the real scene around the vehicle relative to the human eyes of the user in the car are determined. According to the posture of the real scene around the vehicle relative to the human eyes of the user in the car, the virtual atmosphere data is displayed on the head-up display. For example, through the data of external cameras, millimeter waves, laser radars and other sensors, the position coordinates of the external driving environment are sensed, and combined with the vehicle's GPS position, direction and other information, the position of the vehicle and the astronomical information that should be displayed under the current perspective are calculated. The spatial tracking technology in AR technology, such as SLAM, can be used to align the virtual astronomical information with the real scene to ensure its consistency under different perspectives. Then, AR technology is used to integrate the virtual astronomical information with the real scene to form an augmented reality star map, including information such as constellation patterns and star brightness, as well as special effects such as celestial motion simulation.

[0072] In order to improve the user experience, we can consider increasing user interaction, such as controlling the display and hiding of astronomical information through gestures or voice, and asking questions to the driver through question-and-answer methods to enhance the interactive experience and impression.

[0073] According to the fourth specific embodiment of the present disclosure, taking driving on the road on Christmas Eve as an example, the time information in the driving information can be obtained first, and the date meaning keywords corresponding to the time information can be extracted through the neural network model, such as reading the festival keywords of the current time according to the calendar and the user-defined personal anniversaries (marriage, birthday, etc.), and collecting the atmosphere scene photos or video information of the relevant keywords, taking Christmas Eve as an example, including the position, shape, color and other information of decorations such as Christmas lights, gifts, ribbons, etc., and combining multiple date meaning keywords to generate atmosphere information related to vehicle driving; then the atmosphere information is input into the AIGC model so that the AIGC model outputs the data to be rendered of the real scene model corresponding to the atmosphere information, and the data to be rendered is imported into the high-performance rendering engine to generate virtual atmosphere data corresponding to the real scene model. For example, in order to increase the sense of virtual atmosphere, effects such as lighting and color matching can be added to the head-up display to simulate the effects of Christmas tree lights and decorations. At the same time, the rendering engine combines the vehicle speed information to adjust the rendering picture to a suitable look and feel. A realistic virtual 3D Christmas atmosphere decoration video based on the model generated by the previous process is generated by a high-performance rendering engine.

[0074] Through data from external cameras, cockpit cameras, millimeter waves, lidar and other sensors, the vehicle's position information and body posture data, as well as the sitting position of the user in the car and the position information of external environmental objects are perceived, and then the position and posture of environmental objects in the real scene around the vehicle relative to the eyes of the user in the car are determined. According to the posture of the real scene around the vehicle relative to the eyes of the user in the car, the virtual atmosphere data is displayed on the head-up display.

[0075] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.

[0076] Figure 4 is a structural block diagram of a display device according to an embodiment of the present disclosure; Figure 4 As shown, including:

[0077] An acquisition module 410 is used to acquire driving information of the vehicle and / or information of the environment in which the vehicle is located, wherein the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located;

[0078] A determination module 420, configured to determine atmosphere information related to the vehicle driving according to the driving information and / or information about the environment in which the vehicle is located;

[0079] A generation module 430, for generating corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model;

[0080] The display module 440 is used to display the virtual atmosphere data on a head-up display, so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

[0081] According to the display device of the embodiment of the present disclosure, the driving information of the vehicle and / or the information of the environment in which the vehicle is located are obtained, the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located, and then the atmosphere information related to the driving of the vehicle is determined based on the driving information and / or the information of the environment in which the vehicle is located, and then the corresponding virtual atmosphere data is generated based on the atmosphere information and the artificial intelligence model, and the virtual atmosphere data is displayed on the head-up display, so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle. In this way, the head-up display can display the rendered atmosphere scene image based on the driving information of the vehicle and / or the information of the environment in which the vehicle is located, so that the head-up display can not only display the basic data of the vehicle, such as vehicle status, navigation, etc., but also can display the rendered atmosphere scene image corresponding to the real scene.

[0082] In an exemplary embodiment, the determination module 420 is specifically configured to process at least one of the driving information and the information of the vehicle's environment using a trained neural network model; and splice at least part of the processing results as the atmosphere information related to the vehicle's driving.

[0083] In an exemplary embodiment, the determination module 420 is specifically used to extract, for the time information in the driving information, date-significant keywords corresponding to the time information using the trained first neural network model unit; to obtain, for the location information in the driving information, a historical environment image corresponding to the location information, and to extract image feature data of the historical environment image using the trained second neural network model unit; to extract, for the voice interaction information collected in the driving information, semantic keywords corresponding to the voice interaction information using the trained third neural network model unit; and to extract, for the image information of the vehicle environment in the information of the vehicle environment, scene feature keywords corresponding to the image information of the vehicle environment using the trained fourth neural network model unit.

[0084] In an exemplary embodiment, the generation module 430 is specifically used to input the atmosphere information into the AIGC model when the artificial intelligence model is an AIGC model, so that the AIGC model outputs the data to be rendered of the real scene model corresponding to the atmosphere information; and render the data to be rendered to obtain virtual atmosphere data corresponding to the real scene model.

[0085] In an exemplary embodiment, the display module 440 is specifically used to determine the posture of the real scene around the vehicle relative to the eyes of the user in the vehicle; based on the posture of the real scene around the vehicle relative to the eyes of the user in the vehicle, the virtual atmosphere data is displayed on the head-up display.

[0086] In an exemplary embodiment, the display module 440 is specifically used to obtain the vehicle's position information and body posture data, as well as the sitting position of the user in the vehicle and the position information of external environmental objects; based on the vehicle's position information, body posture data, the position information of external environmental objects and the sitting position of the user in the vehicle, determine the position and posture of environmental objects in the real scene around the vehicle relative to the eyes of the user in the vehicle.

[0087] In an exemplary embodiment, the display module 440 is specifically used to determine the field of view angle position of the environmental objects in the real scene of the vehicle's surrounding environment relative to the eyes of the user in the vehicle; determine the perspective relationship between the virtual atmosphere data and the head-up display according to the field of view angle range and the posture of the environmental objects in the real scene of the vehicle's surrounding environment relative to the eyes of the user in the vehicle; adjust the parameters of the display model according to the field of view angle position and the perspective relationship, wherein the parameters of the display model include shape and / or size; and display the virtual atmosphere data on the adjusted display model.

[0088] In an exemplary embodiment, the display module 440 is specifically configured to compare the size of the display model with the size of the target real object in the real scene; and determine the display content and / or display position of the virtual atmosphere data according to the comparison result.

[0089] In an exemplary embodiment, the device further includes an adjustment module, which is used to obtain control information sent by a user in the car; adjust the display of the virtual atmosphere data according to the control information; and / or send interactive information to the user in the car.

[0090] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0091] S1, obtaining driving information of a vehicle and / or information of an environment in which the vehicle is located, wherein the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located;

[0092] S2, determining atmosphere information related to the vehicle driving according to the driving information and / or information about the environment in which the vehicle is located;

[0093] S3, generating corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model;

[0094] S4, displaying the virtual atmosphere data on a head-up display, so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

[0095] An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0096] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0097] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0098] S1, obtaining driving information of a vehicle and / or information of an environment in which the vehicle is located, wherein the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located;

[0099] S2, determining atmosphere information related to the vehicle driving according to the driving information and / or information about the environment in which the vehicle is located;

[0100] S3, generating corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model;

[0101] S4, displaying the virtual atmosphere data on a head-up display, so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

[0102] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0103] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0104] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present disclosure can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any specific combination of hardware and software.

[0105] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the present disclosure.

Claims

1. A display method, characterized in that: include: Acquire driving information of the vehicle and / or information of the environment in which the vehicle is located, wherein the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located; Determining atmosphere information related to the vehicle driving according to the driving information and / or information about the environment in which the vehicle is located; Generate corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model; The virtual atmosphere data is displayed on a head-up display, so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

2. The method according to claim 1, characterized in that: Determining atmosphere information related to the vehicle driving according to the driving information and / or information about the environment in which the vehicle is located includes: Processing at least one of the driving information and the information of the environment where the vehicle is located using the trained neural network model; At least part of the processing results are spliced ​​as the ambience information related to the driving of the vehicle.

3. The method according to claim 2, characterized in that The using the trained neural network model to process at least one of the driving information and the environment information includes at least one of the following: For the time information in the driving information, using the trained first neural network model unit, extracting date meaning keywords corresponding to the time information; For the location information in the driving information, a historical environment image corresponding to the location information is obtained, and image feature data of the historical environment image is extracted using the trained second neural network model unit; For the voice interaction information collected in the driving information, using the trained third neural network model unit, extracting semantic keywords corresponding to the voice interaction information; For the image information of the environment where the vehicle is located in the information of the environment where the vehicle is located, the trained fourth neural network model unit is used to extract scene feature keywords corresponding to the image information of the environment where the vehicle is located.

4. The method according to claim 1, characterized in that: Generate corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model, including: In the case where the artificial intelligence model is an AIGC model, the atmosphere information is input into the AIGC model so that the AIGC model outputs data to be rendered of the real scene model corresponding to the atmosphere information; The data to be rendered is rendered to obtain virtual atmosphere data corresponding to the real scene model.

5. The method according to claim 1, characterized in that Displaying the virtual atmosphere data on a head-up display comprises: Determine the pose of the real scene around the vehicle relative to the eyes of the user in the vehicle; The virtual atmosphere data is displayed on a head-up display according to the posture of the real scene around the vehicle relative to the eyes of the user in the vehicle.

6. The method according to claim 5, characterized in that Determine the pose of the real world around the vehicle relative to the eyes of the user inside the vehicle, including: Obtaining the vehicle's location information and body posture data, as well as the seat position of the user in the vehicle and the location information of external environment objects; Based on the position information of the vehicle, the body posture data, the position information of external environmental objects and the seating position of the user in the vehicle, the position and posture of environmental objects in the real scene around the vehicle relative to the eyes of the user in the vehicle are determined.

7. The method according to claim 6, characterized in that Displaying the virtual atmosphere data on a head-up display according to the posture of the real scene around the vehicle relative to the eyes of the user in the vehicle, including: Determining the viewing angle position of the environmental object in the real scene relative to the human eyes of the user in the vehicle according to the position of the environmental object in the real scene of the vehicle surrounding environment relative to the human eyes of the user in the vehicle; Determine the perspective relationship between the virtual atmosphere data and the head-up display according to the field of view angle range and the posture of the environmental objects in the real scene of the vehicle surrounding environment relative to the eyes of the user in the vehicle; adjust the parameters of the display model according to the field of view angle position and the perspective relationship, wherein the parameters of the display model include shape and / or size; The virtual atmosphere data is displayed on the adjusted display model.

8. The method according to claim 7, characterized in that The method further comprises: Comparing the size of the displayed model with the size of the target real object in the real scene; The display content and / or display position of the virtual atmosphere data is determined according to the comparison result.

9. The method according to claim 1, characterized in that: The method further comprises: Get the control information sent by the user in the car; Adjusting the display of the virtual atmosphere data according to the control information; and / or The interaction information is sent to the in-vehicle user.

10. A display device, characterized in that: include: An acquisition module, used to acquire driving information of the vehicle and / or information of the environment in which the vehicle is located, wherein the driving information includes location information and / or time information, and the information of the environment in which the vehicle is located includes image information of the environment in which the vehicle is located; A determination module, configured to determine atmosphere information related to the driving of the vehicle based on the driving information and / or information about the environment in which the vehicle is located; A generation module, used to generate corresponding virtual atmosphere data according to the atmosphere information and the artificial intelligence model; The display module is used to display the virtual atmosphere data on a head-up display so that the virtual image displayed by the head-up display is integrated with the real scene around the vehicle.

11. A vehicle, characterized in that: Comprising the display device as claimed in claim 10.

12. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 9 when executing the computer program on the memory.

13. A computer-readable storage medium, characterized in that: The computer readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images