Car window projection system and method

Abstract

The invention discloses a vehicle window projection system and method, and relates to the technical field of vehicles. According to the specific implementation mode of the method, the system comprises an intelligent cabin host, detection equipment and a projector, and the detection equipment is used for detecting the position of a passenger in a vehicle; the intelligent cabin host is used for determining a target vehicle window corresponding to the passenger from a plurality of vehicle windows of the vehicle according to the position of the passenger in the vehicle; determining environment information of an area where the vehicle is located, and generating a scene image corresponding to the target vehicle window according to the environment information; and the projector is used for projecting the scene image on the target vehicle window. According to the implementation mode, the environment information outside the vehicle window corresponding to the passenger is displayed on the vehicle window in a close range in an image mode, so that practical or interesting scene information is provided for the passenger through the image, the dull riding of the passenger is relieved, and the riding experience is enriched.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular, to a window projection system and method. Background Art

[0002] Currently, the functions of in-vehicle projection devices on the market are basically limited to providing multimedia entertainment services for rear-seat passengers, or providing safe driving services such as projecting safety-related symbols and signs on the road or wall in front of the vehicle. However, current in-vehicle projection devices do not provide relatively practical projection experience content for rear-seat occupants, resulting in rear-seat passengers usually looking out of the window bored. Since the scenery outside the window is either unfamiliar or far away, no useful information can be obtained, thus making the riding experience of rear-seat passengers poor. Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a window projection system and method, including an intelligent cockpit host, a detection device, and a projector. Among them, the detection device is used to detect the position of the occupant in the vehicle; the intelligent cockpit host is used to determine the target window corresponding to the occupant from multiple windows of the vehicle according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information; the projector is used to project the scene image onto the target window. Thus, the environmental information outside the window corresponding to the occupant is displayed close to the window in the form of an image, so as to provide practical or interesting scene information for the occupant through the image, relieve the boredom of the occupant during the ride, and enrich the riding experience.

[0004] To achieve the above object, according to one aspect of the embodiments of the present invention, a window projection system is provided.

[0005] A window projection system according to an embodiment of the present invention includes: an intelligent cockpit host, a detection device, and a projector, where

[0006] the detection device is used to detect the position of the occupant in the vehicle;

[0007] the intelligent cockpit host is used to determine the target window corresponding to the occupant from multiple windows of the vehicle according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information;

[0008] the projector is used to project the scene image onto the target window.

[0009] Optionally, the detection device is further used to detect the head position and eye focus position of the occupant;

[0010] The intelligent cockpit host is further configured to generate real-time calibration parameters based on the head position and eye focus position of the occupant, and calibrate the projection angle of the projector according to the real-time calibration parameters.

[0011] Optionally, the intelligent cockpit host is further configured to determine the points of interest of the occupant in the scene image, and provide the detailed information included in the points of interest to the projector.

[0012] The projector is further configured to project the detailed information of the points of interest on the target window.

[0013] Optionally, the intelligent cockpit host is further configured to use the scene image as the underlying content of augmented reality, and superimpose the detailed information of the points of interest as supplementary information on the upper layer of the scene image.

[0014] The projector is further configured to project the scene image as the underlying layer and the detailed information as the upper layer onto the target window according to a preset priority policy.

[0015] Optionally, the scene image includes multiple real-world scenes.

[0016] The intelligent cockpit host is further configured to determine the eye focus duration of the occupant for multiple real-world scenes according to the head position and eye focus position of the occupant collected by the detection device, and determine one or more points of interest from the multiple scenes according to the eye focus duration of the occupant for the multiple real-world scenes.

[0017] Optionally, the detection device is further configured to collect the gestures of the occupant, finger pointing, or the position where the occupant triggers an operation on the target window.

[0018] The intelligent cockpit host is further configured to determine one or more points of interest from the multiple real-world scenes displayed on the target window according to the gestures of the occupant, finger pointing, or the position where the occupant triggers an operation on the target window.

[0019] Optionally, the system further includes a sound collection device, where

[0020] The sound collection device is configured to receive a user request input by the occupant through voice.

[0021] The intelligent cockpit host is further configured to generate a reply message for the user request according to the voice input by the occupant.

[0022] The projector is further configured to display the reply message in the scene image displayed on the target window.

[0023] Optionally, the system further includes a voice playback module, wherein,

[0024] the voice playback module is configured to play the reply information.

[0025] Optionally, the intelligent cockpit host is further configured to determine the scene type corresponding to the point of interest, and determine the corresponding detailed information according to the place type; the place type includes one or more of a shopping mall, a parking lot, a natural landscape, a road, and a cultural and entertainment venue; the detailed information includes one or more of a landmark name, a location name, advertising and promotion information, a parking lot entrance, a navigation route, venue details, a road name and driving tips, a ticket price, a performance time, and a performance schedule.

[0026] Optionally, the window includes a window glass and a translucent film embedded in the window glass;

[0027] the translucent film is configured to display the scene image projected by the projector.

[0028] Optionally, the intelligent cockpit host is further configured to extract key features of multiple scenes included in the external environment information of the area where the vehicle is located, and generate a 3D scene map according to the key features; the key features include one or more of the following features: contour, color, texture, material, and visual hierarchy;

[0029] the projector is further configured to project the 3D scene map onto the target window.

[0030] Optionally, the intelligent cockpit host is further configured to obtain the map data corresponding to the positioning information from the navigation map data according to the current positioning information of the vehicle; generate scene images respectively corresponding to multiple windows according to the map data and the position relationship between each seat in the vehicle and the corresponding window;

[0031] the projector is further configured to project the scene images respectively corresponding to multiple windows onto the corresponding windows.

[0032] Optionally, the intelligent cockpit host is further configured to calculate the map data slices of the real scenes corresponding to each window according to the map data and the position relationship between each seat in the vehicle and the corresponding window; and generate the scene images corresponding to each window according to the map data slices.

[0033] Optionally, the intelligent cockpit host is further configured to dynamically update the scene images respectively corresponding to multiple windows according to the real-time updated positioning information of the vehicle;

[0034] the projector is further configured to project the updated scene images respectively corresponding to multiple windows onto the corresponding windows.

[0035] Optionally, the intelligent cockpit host is further configured to determine the content to be displayed input by the user and provide the content to be displayed to the projector;

[0036] The projector is further configured to project the content to be displayed onto the sunroof of the vehicle.

[0037] Optionally, the intelligent cockpit host is further configured to obtain the current nebula map corresponding to the positioning information and the current time, as well as the constellation names included in the nebula map, according to the current positioning information and the current time of the vehicle;

[0038] The projector is further configured to project the nebula map and the constellation names onto the sunroof of the vehicle correspondingly.

[0039] To achieve the above object, according to another aspect of the embodiments of the present invention, a window projection method is provided.

[0040] A window projection method according to an embodiment of the present invention includes:

[0041] Detecting the position of the occupant in the vehicle by a detection device;

[0042] Determining, by an intelligent cockpit host, a target window corresponding to the occupant from multiple windows of the vehicle according to the position of the occupant in the vehicle; determining environmental information of the area where the vehicle is located, and generating a scene image corresponding to the target window according to the environmental information;

[0043] Projecting the scene image onto the target window by a projector.

[0044] To achieve the above object, according to still another aspect of the embodiments of the present invention, an electronic device for window projection is provided.

[0045] An electronic device for window projection according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement a window projection method according to an embodiment of the present invention.

[0046] To achieve the above object, according to yet another aspect of the embodiments of the present invention, a computer-readable storage medium is provided.

[0047] A computer-readable storage medium according to an embodiment of the present invention, on which a computer program is stored, and when the program is executed by a processor, implementing a window projection method according to an embodiment of the present invention.

[0048] One embodiment of the above invention has the following advantages or beneficial effects: The window projection system includes an intelligent cockpit host, a detection device, and a projector. Among them, the detection device is used to detect the position of the occupant in the vehicle; the intelligent cockpit host is used to determine the target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information; the projector is used to project the scene image onto the target window. Thus, the environmental information outside the vehicle corresponding to the occupant is displayed closely on the window in the form of an image, so as to provide practical or interesting scene information for the occupant through the image, relieve the boredom of the occupant during the ride, and enrich the riding experience.

[0049] The further effects of the above non-conventional alternative methods will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings are used to better understand the present invention and do not constitute an improper limitation to the present invention. Among them:

[0051] Figure 1 is a schematic diagram of the main modules of a window projection system according to an embodiment of the present invention;

[0052] Figure 2 is a schematic diagram of a window projection effect according to an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of another window projection effect according to an embodiment of the present invention;

[0054] Figure 4 is a schematic diagram of a hardware deployment solution of a window projection system according to an embodiment of the present invention;

[0055] Figure 5 is a schematic diagram of the main steps of a window projection method according to an embodiment of the present invention;

[0056] Figure 6 is an exemplary system architecture diagram to which the embodiments of the present invention can be applied;

[0057] Figure 7 is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0059] It should be noted that, without conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0060] Figure 1 It is a schematic diagram of the main modules of a window projection system according to an embodiment of the present invention.

[0061] As Figure 1 shown, the window projection system 100 according to an embodiment of the present invention includes an intelligent cockpit host 102, a detection device 101, and a projector 103. Among them, the detection device 101 is used to detect the position of the occupant in the vehicle; the intelligent cockpit host 102 is used to determine the target window corresponding to the occupant from multiple windows of the vehicle according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information; the projector 103 is used to project the scene image onto the target window. In an alternative embodiment of the present invention, the window includes a window glass and a semi-transparent film embedded in the window glass; the semi-transparent film is used to display the scene image projected by the projector. The scene image can be a 2D scene map or a 3D scene map. The multiple windows can include all the windows of the vehicle, such as windows and sunroofs. According to the external environmental information that the occupant can see through each window, a scene image is generated and projected onto the corresponding window respectively, so that the occupant sitting near each window can obtain practical information through the scene image displayed on the window, thus enriching the occupant's driving experience. In addition, the detection device can include a camera, and the position of the occupant in the vehicle seat is detected by the camera, and the scene image is projected onto the window corresponding to the seat. Among them, after the scene image is projected onto the window, a schematic diagram of the effect of the window displaying the scene image is as Figure 2 shown. It can be understood that the content shown in the figure is only for illustration and is not for limiting use.

[0062] Among them, the number of projectors can be one or multiple. When the number of projectors is one, only one window can be projected with a scene image at the same time. When the number of projectors is multiple, different projectors can project onto different windows respectively at the same time. Similar to the projectors, the number of cameras can be one or multiple. The number of projectors and windows can be equal and in a one-to-one correspondence. The number of cameras and windows can be equal and in a one-to-one correspondence.

[0063] The scene images projected on the windows are generated based on the map data of the vehicle's location, and the scene images are updated in real time as the vehicle's position moves. In an alternative embodiment of the present invention, the intelligent cockpit host is further configured to obtain the map data corresponding to the positioning information from the navigation map data according to the current positioning information of the vehicle; generate scene images corresponding to multiple windows respectively according to the map data and the position relationship between each seat in the vehicle and the corresponding window; the projector is further configured to project the scene images corresponding to multiple windows respectively onto the corresponding windows. Specifically, by positioning the current position of the vehicle, the map data corresponding to this position is obtained; according to the map data, the map data slices of the real scene corresponding to each window are calculated; and according to the map data slices, the scene images corresponding to each window are generated. Therefore, in an alternative embodiment of the present invention, the intelligent cockpit host is further configured to calculate the map data slices of the real scene corresponding to each window according to the map data and the position relationship between each seat in the vehicle and the corresponding window; and generate the scene images corresponding to each window according to the map data slices. This implementation method realizes slicing the map data of the current position to obtain multiple map data slices, and generating multiple scene images according to the multiple map data slices. Among them, the perspectives of the multiple scene images are equivalent to the perspectives of the occupants looking out through multiple windows.

[0064] During the driving of the vehicle, the scene images displayed on the windows are continuously updated as the external environment of the vehicle changes. The intelligent cockpit host is further configured to dynamically update the scene images corresponding to multiple windows respectively according to the real-time updated positioning information of the vehicle; the projector is further configured to project the updated scene images corresponding to multiple windows respectively onto the corresponding windows. Among them, due to the driving of the vehicle, the location of the vehicle changes, the map data of the vehicle's location is obtained in real time, and according to the obtained map data, new scene images are generated and projected onto the windows.

[0065] To reduce the resource overhead for the intelligent cockpit host to calculate and render the scene image, and also to cooperate with the imaging effect of the semi-transparent film inside the window glass, the generated scene image can include only the key features in the real scene, such as building outlines, colors, materials, visual hierarchies, etc., such as the outlines, colors, materials, visual hierarchies of the lawn, etc. In an alternative embodiment of the present invention, the intelligent cockpit host is further configured to extract the key features of multiple scenes included in the external environment information of the area where the vehicle is located, and generate a 3D scene map according to the key features; the key features include one or more of the following features: outline, color, texture, material, and visual hierarchy; the projector is further configured to project the 3D scene map onto the target window. This embodiment achieves the purpose of reducing resource overhead by reducing the resolution.

[0066] In addition to generating the scene image corresponding to the window according to the position relationship between the seat where the occupant is sitting in the vehicle and the window, the viewing angle of the scene image displayed on the window can also be adjusted according to the sitting posture of the occupant on the seat, so as to provide a more three-dimensional and realistic visual effect for the occupant. The detection device is further configured to detect the head position and the eye focus position of the occupant; the intelligent cockpit host is further configured to generate real-time calibration parameters according to the head position and the eye focus position of the occupant; and calibrate the projection angle of the projector according to the real-time calibration parameters. Specifically, when the occupant is sitting on the seat, the body posture may change continuously, such as leaning on the seat back, sitting upright on the seat with a straight back, or reclining on the armrest box, etc. To adapt to the changes in the passenger's sitting posture, the camera is used to detect the head position and the eye diagonal position of the occupant, determine the line of sight direction of the occupant, generate real-time calibration parameters according to the line of sight direction, and use the real-time calibration parameters to correct the projection angle of the projector in real time, so that the scene image projected onto the window corresponds to the sitting posture of the occupant, and the visual effect of the occupant viewing the scene image on the window is more realistic. In addition, the intelligent cockpit host can also adjust the generated scene image according to the real-time calibration parameters, and project the adjusted scene image onto the window through the projector.

[0067] In addition, the scene image displayed on a window may include multiple real-world scenes, such as shopping mall buildings, parks, and movie theaters. By detecting the occupant's state, it can be determined which real-world scene the occupant is interested in, and thus in-depth information about the point of interest can be further provided to the occupant. In an alternative embodiment of the present invention, the intelligent cockpit host is further configured to determine the point of interest of the occupant in the scene image and provide the detailed information included in the point of interest to the projector; the projector is further configured to project the detailed information of the point of interest on the target window. For example, by detecting the occupant's state collected by the detection device, it is determined that the occupant is interested in XX Shopping Mall in the scene image. Then the intelligent cockpit host obtains the in-depth information of XX Shopping Mall, such as recent discount promotion information, etc., and projects the in-depth information on the window through the projector. The display effect is as Figure 3 shown, where "big sale on X month X day" is the detailed information. The occupant can learn about the discount promotion information of the shopping mall through the in-depth information displayed on the window. This embodiment discovers the occupant's point of interest by intelligently tracking the occupant's state and provides in-depth information about the point of interest, giving the occupant a relatively high degree of intelligent and technological riding experience.

[0068] In addition, in an alternative embodiment of the present invention, the scene image includes multiple real-world scenes, and the intelligent cockpit host is further configured to determine the eye fixation duration of the occupant for multiple real-world scenes according to the head position and eye fixation position of the occupant collected by the detection device; and determine one or more points of interest from multiple scenes according to the eye fixation duration of the occupant for multiple real-world scenes. This embodiment determines whether the occupant is interested in a position by detecting the position where the occupant's line of sight focuses on the scene image and the duration of the line of sight focusing on that position. For example, it can be set that when the occupant's fixation duration at a certain position reaches 10 seconds or more (including 10 seconds), it is determined that the occupant is interested in that position, that is, that position is determined as the occupant's point of interest. In addition, the occupant may be interested in one or more real-world scenes in the scene image.

[0069] Further, in addition to determining the occupant's point of interest by tracking the occupant's line of sight, the occupant's point of interest can also be determined by tracking the occupant's actions and gestures, etc. In an alternative embodiment of the present invention, the detection device is further configured to collect the occupant's gestures, finger pointing, or the position where the occupant triggers the target window; the intelligent cockpit host is further configured to determine one or more points of interest from a variety of real-world scenarios displayed on the target window according to the occupant's gestures, finger pointing, or the position where the occupant triggers the target window. Specifically, the corresponding relationship between the gesture and the position point on the window can be preset, so that the position point on the window indicated by the occupant can be determined by recognizing the occupant's gesture, and this position point is determined as the occupant's point of interest. Since the scene image is displayed on the window, the real-world scenario corresponding to the position point on the window can be used to determine the real-world scenario that the occupant is interested in. For example, if the occupant makes a heart gesture with their hand, according to the preset corresponding relationship, it can be determined that the heart gesture corresponds to the center position of the window, and the middle position of the scene image displayed on the window is a movie theater, then the occupant's point of interest can be determined as the movie theater. In addition, the finger pointing can also be recognized, and the position where the intersection of the straight line where the finger is located and the window is located is determined as the point of interest; the position where the occupant's finger touches the window can also be determined as the point of interest. For example, if the occupant makes a click action with their finger at a certain position, then this position is determined as the point of interest, and the real-world scenario corresponding to this point of interest on the scene image, such as a shopping mall building, is the real-world scenario that the occupant is interested in.

[0070] After identifying the target point of interest of the occupant in the scene image, the depth information of the point of interest is displayed in the scene image. Among them, the detailed information corresponding to different scene types is different. In an optional embodiment of the present invention, the intelligent cockpit host is further configured to determine the scene type corresponding to the point of interest, and determine the corresponding detailed information according to the place type; the place type includes one or more of a shopping mall, a parking lot, a natural landscape, a road, and a cultural and entertainment venue; the detailed information includes one or more of a landmark name, a location name, advertising and promotion information, a parking lot entrance, a navigation route, place details, a road name and driving tips, a ticket price, a performance time, and a performance schedule. For example, if the point of interest of the occupant is identified as Scenic Area A, the historical information, current number of tourists, and visiting time of Scenic Area A are used as detailed information and displayed in the scene image. For another example, when the detection device identifies that the occupant is interested in Parking Lot B, the intelligent cockpit host can obtain the entrance location of Parking Lot B and project the entrance location and the navigation route onto the window through a projector, so that the entrance location is displayed in the scene image. In order to make the detailed information more prominent, the detailed information can be highlighted in a more prominent color in the scene image. For another example, if the point of interest of the occupant is a shopping mall building, one or more of the merchants, brands, discount promotion information, and business hours in the shopping mall can be used as detailed information and displayed in the scene image.

[0071] In order to make the display of the scene image and the detailed information more hierarchical and technological, in an optional embodiment of the present invention, the intelligent cockpit host is further configured to use the scene image as the underlying content of augmented reality and use the detailed information of the point of interest as supplementary information and superimpose it on the upper layer of the scene image; the projector is further configured to project the scene image as the underlying layer and the detailed information as the upper layer onto the target window according to a preset priority strategy. The display order of the scene image and the detailed information includes three types: the scene image and the detailed information are displayed simultaneously, the scene image is displayed first and then the detailed information, and the detailed information is displayed first and then the scene image. The scene image includes some basic information of the real scene, such as building landmarks, scenic area signs, parking lot indication signs, etc. The detailed information is supplementary information to the above basic information.

[0072] In an optional embodiment of the present invention, the intelligent cockpit host is further configured to determine the content to be displayed input by the user and provide the content to be displayed to the projector; the projector is further configured to project the content to be displayed onto the sunroof of the vehicle. The content to be displayed can be a movie video accessed by the user. The movie video is projected onto the vehicle sunroof through the projector, and the movie content is played through the sunroof, so that the user can recline the vehicle seat and watch a movie in the car.

[0073] In an alternative embodiment of the present invention, the intelligent cockpit host is further configured to obtain the current nebula map corresponding to the current positioning information and the current time of the vehicle, as well as the constellation names included in the nebula map; the projector is further configured to project the nebula map and the constellation names onto the sunroof of the vehicle correspondingly. This embodiment realizes the purpose of viewing the starry sky inside the vehicle by projecting the nebula map and constellation names onto the sunroof. This embodiment is more applicable to the following scenarios: when the user is camping in the wild and has a plan to watch the stars, but due to natural conditions such as rain or thick clouds on that day, the wish to watch the starry sky cannot be realized. However, through the technical means of projecting the nebula map of that time and place onto the sunroof in this embodiment, the user's wish to watch the starry sky is satisfied, giving the user a better driving experience.

[0074] To further enhance the interaction experience between the user and the vehicle, in an alternative embodiment of the present invention, the system further includes a sound collection device. Among them, the sound collection device is configured to receive the user request input by the occupant through voice; the intelligent cockpit host is further configured to generate a reply message for the user request according to the voice input by the occupant; the projector is further configured to display the reply message in the scene image displayed on the target window. Specific application examples: The user points to the parking lot in the scene image and says, "Where is the parking lot entrance?" The sound collection device, such as a microphone, receives the sound information indicating "Where is the parking lot entrance?" The intelligent cockpit host generates a reply message, such as an entrance indication sign, according to this sound information, and projects the entrance indication sign onto the window through the projector, and highlights the entrance indication sign in the scene image. In this way, the user can quickly determine the location of the parking lot entrance according to the entrance indication sign. In addition, the reply message can also be text information, such as the parking lot entrance is on the left side of XX Building, go straight along the current road for 500 meters and turn left to reach the entrance location. Another example is that a child occupant in the vehicle can point to the park in the scene image displayed on the window and ask, "What place is this?" After the sound collection device collects this sound, the intelligent cockpit host generates a reply message according to this sound information, such as "This is People's Park, and this park includes many scenic spots, which are respectively...", and plays the reply message in the form of sound through a voice playback module, such as a speaker. Therefore, in an alternative embodiment of the present invention, the system further includes a voice playback module, where the voice playback module is configured to play the reply message. The embodiment of the present invention further improves the effect of human-machine interaction and increases the fun of human-machine interaction through the sound collection device and the voice playback module, enabling the user to obtain practical information in a convenient way and improving the user's driving experience.

[0075] The following is an example of a car window projection system using a specific embodiment. In this embodiment, a laser projector in the car is used as a playback device, a smart cockpit host in the car is used as a computing center, and other car windows in the car except the front windshield (including the windows of multiple doors in the front, rear, left and right, and the panoramic sunroof on the roof) are used as projection carriers to provide users with an interactive method for AR enhanced experience combined with real scenes in addition to audio and video playback. To implement the hardware deployment solution of this embodiment, Figure 4 As shown in the figure, the hardware components of this deployment solution are as follows: 1. In-car smart cockpit host; 2. In-car TBOX Internet of Vehicles system and GPS positioning system; 3. In-car laser projector (one projector is matched for each window); 4. Semi-transparent film embedded in the window for projector imaging; 5. Camera for capturing the sight and movement of each occupant in the car; 6. Microphone for collecting the voice of each occupant in the car; 7. Vehicle cloud server.

[0076] In addition, the software deployment solution for implementing this embodiment includes the following contents:

[0077] (1) The intelligent cockpit host in the vehicle, based on the real-time updated high-precision positioning results and the 3D coordinate information in the built-in navigation map, not only generates a 3D map on the central control screen, but also calculates and generates a 3D model image that can be displayed through multiple side windows. When calculating, it is necessary to adjust the timing and angle according to different seats, and adjust the display effect of the 3D model image to a 3D rendering view that is visible to passengers in each seat at all times. The 3D model image is dynamically updated as the vehicle moves. The display effect of the 3D model image is related to the refresh frequency. To ensure the user experience, the refresh frequency of the 3D model image is between 10Hz and 25Hz, and ultimately achieves synchronization with the position and movement speed of the real scene outside the window that each passenger sees in real time.

[0078] (2) The 3D model images of each window generated by the intelligent cockpit host are sent to the laser projector corresponding to each window through the video output interface or Ethernet interface. The installation position of the laser projector needs to be pre-calibrated to project the anti-distortion 3D model image on the window so that the position deviation between the actual scene seen by the user from the window and the 3D model image displayed on the window is within the allowable range. The window glass is embedded with a translucent film, based on which the projected content can be imaged on the glass.

[0079] (3) A capture camera is arranged at the front position of each occupant position to capture the line of sight and actions, identify and track in real time the head position, eyeball position, and line of sight direction of each seat occupant. At the same time, it is also necessary to identify the gesture actions of the occupant and capture the pointing direction of the user's finger. Based on the capture of the occupant's head position and the eyeball focus position, corresponding real-time position calibration parameters are generated, so that the intelligent cockpit host generates a 3D model diagram according to the real-time position correction parameters. The intelligent cockpit host dynamically calculates the real-world scene of the corresponding window position according to the change of the real-time position parameters of each occupant, such as the real-time 3D model diagram of a shopping mall building, so as to realize that the display angle of the 3D model diagram projected on the window is adjusted in real time as the occupant's body rotates, leans forward or backward, ensuring that the position of a certain target in the 3D model diagram seen by the user is consistent with the position of the external actual target. At the same time, the intelligent cockpit host determines the target object where the user's line of sight stays for a long time according to the user's eyeball focus position and the landmark identification in the 3D model diagram, supplements detailed information for the target object, and projects the detailed information onto the corresponding window through a projector.

[0080] (4) The 3D model diagram serves as the basic projection map and the underlying content for the AR real-scene enhancement of this solution. Various detailed information can be superimposed on this layer of map and projected onto the window according to different priority strategies. The 3D model diagram contains some basic information in the real-world scene, and this basic information can be set by the user customarily. The intelligent cockpit host is connected to the cloud server through the vehicle network to obtain more real-time information or advertising promotion information, and uses the obtained information as detailed information. The obtained information includes but is not limited to the following: Figure 1 a. Building, landmark name and location annotation;

[0081] b. Park, square, lake, river, mountain, village name and location annotation;

[0082] c. Display and annotation of road name, driving direction, and railway name;

[0083] d. Display and location annotation of interest point category icons, brand Logos and names, as well as the highlighted cursor display and location annotation of the interest point (such as a brand icon with a position pointing arrow);

[0084] e. Position cursor and name of the interest point on any floor inside the building;

[0085] f. Preferential summary text information of the real-time promotion activities of shopping mall and restaurant type interest points;

[0086] g. Highlight display of the specified building within the building group;

[0087] ​

[0088] h. Display of the route to the designated destination;

[0089] i. Performance and ticket price information of cinemas and theaters;

[0090] j. Display of the entrance and name of the parking lot and marking of the location.

[0091] The following is an example of the actual application scenario of the embodiments of the present invention:

[0092] Based on the above software and hardware solutions and the supplementary detailed information, the AR in-depth experience of the following application scenarios can be realized but is not limited to.

[0093] (1) When a user takes a vehicle in an unfamiliar city, according to the user's line of sight position, the names of various landmarks and buildings, as well as the names of the side roads or rivers, are displayed in real time on the car window, and the building in the direction pointed by the user's finger can be highlighted to increase the in-depth introduction of the user's points of interest and play the role of a free senior tour guide.

[0094] (2) When a father drives with his child through the city, the child is sitting in the back seat looking at the high-rise buildings beside and constantly asking his father where is here? What's the name of that park? The voice collection device and camera in the car make a comprehensive judgment based on the child's voice, line of sight, and gestures or actions, identify the location of the building or scenic spot, as well as the user's request, and send the identified information to the in-car navigation map to determine the name and introduction information of the identified scene. Through the voice playback module, such as the in-car voice assistant, the name and introduction information are output in a specific voice to achieve the scene introduction in an interactive way with the child.

[0095] (3) When a user takes a long-distance vehicle in summer, the mountains and rivers on the road are faintly visible under the cover of other vehicles or road trees. The window projection solution of this embodiment can project the information of the mountains, rivers, villages, and scenic spots covered by vehicles or trees onto the user's car window, allowing the user to view all the scenery at a glance.

[0096] (4) When a driver is looking for a building without a name label on the map in a building community or building complex at night, the co-driver or the passenger in the back seat sees a building they know through the car window and points it out to the driver on the car window. The driver views the navigation route according to the position indicated by the passenger. At the same time, the navigation route and the corresponding turning arrows are also displayed on the main driver's and co-driver's car window glasses. The driver can clearly see the route arrows in 3 directions through the head-up display device in the front and the left and right car windows. The passenger can also see the navigation route on their corresponding car window.

[0097] (5) A family is driving and passing by a large shopping mall. The wife, who is sitting in the back seat, sees the mall promotion notice on the car window and sees that her favorite Starbucks coffee is having a buy-one-get-one-free promotion today. The movie theater on the 8th floor of the mall is also showing the new movie she has been waiting for a long time. So she tells her husband, who is driving. The husband drives directly into the mall parking lot according to the sign for the entrance to the mall parking lot displayed on the car window.

[0098] (6) The user drives to a building but cannot find the entrance to the parking lot of the building. He points to the building seen through the left window and asks, "Where do I enter the parking lot of this building?" The sound collection module and the camera recognize the building pointed to by the user and the user's question, mark the parking lot entrance on the navigation map, and display the sign of the parking lot entrance on the central control navigation map and / or on the 3D model projected on the window, so that the user can quickly find the parking lot entrance.

[0099] (7) The user's vehicle reaches the maintenance mileage and drives past a corresponding brand 4S store hidden in a building complex. The vehicle receives the maintenance reminder information and enters the 4S store. Then the smart cockpit host plays the maintenance reminder voice through the sound playback module, such as a speaker, to remind the user to perform vehicle maintenance. At the same time, the location of the 4S store is displayed on the central control navigation map and / or the 3D model projected on the car window. The driver turns his head to look at the 3D model projected on the car window, inputs a voice through the sound acquisition device, indicating that he accepts this maintenance reminder, and then the smart cockpit host calculates and generates a navigation route to the 4S store.

[0100] (8) Family, friends or lovers go camping in the wild and plan to watch the stars together at night, but they cannot realize their wish to watch the stars due to bad weather that night. Through the car window projection system implemented in this embodiment, the seat in the car can be folded down and lying on the seat, and the sunroof projection function can be turned on to display the obtained clear nebula map on the sunroof glass, including the name of each constellation. Even if you look out the side window, the distant and low-angle constellations that are inconspicuous can be clearly displayed.

[0101] The above are only part of the user scenarios that can be realized by the vehicle window projection system based on the embodiment of the present invention. It can be understood that the above application scenarios are only examples and are not intended to limit the application scenarios of the vehicle window projection system.

[0102] The window projection system according to an embodiment of the present invention includes an intelligent cockpit host, a detection device, and a projector. Among them, the detection device is used to detect the position of an occupant in the vehicle; the intelligent cockpit host is used to determine the target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information; the projector is used to project the scene image onto the target window. Thus, the environmental information outside the vehicle corresponding to the occupant is displayed close to the window in the form of an image, so as to provide practical or interesting scene information for the occupant through the image, relieve the boredom of the occupant during the ride, and enrich the riding experience.

[0103] Figure 5 It is a schematic diagram of the main steps of the window projection method according to an embodiment of the present invention.

[0104] As Figure 5 shown, the window projection method according to an embodiment of the present invention includes steps S501 - S503:

[0105] Step S501, detect the position of an occupant in the vehicle through a detection device;

[0106] Step S502, through the intelligent cockpit host, determine the target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information;

[0107] Step S503, project the scene image onto the target window through a projector.

[0108] The window projection method according to an embodiment of the present invention detects the position of an occupant in the vehicle through a detection device; through the intelligent cockpit host, determines the target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determines the environmental information of the area where the vehicle is located, and generates a scene image corresponding to the target window according to the environmental information; through a projector, projects the scene image onto the target window. Thus, the environmental information outside the vehicle corresponding to the occupant is displayed close to the window in the form of an image, so as to provide practical or interesting scene information for the occupant through the image, relieve the boredom of the occupant during the ride, and enrich the riding experience.

[0109] Figure 6 Fig. shows an exemplary system architecture 600 to which the window projection system or the window projection method according to an embodiment of the present invention can be applied.

[0110] As Figure 6As shown, the system architecture 600 may include detection devices 601, 602, 603, networks 608, 609, an in-vehicle infotainment (IVI) host 604, and projectors 605, 606, 607. The networks 608, 609 are used to provide a medium for communication links between the detection devices 601, 602, 603, the IVI host 604, and the projectors 605, 606, 607. The networks 608, 609 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0111] The detection devices 601, 602, 603 and the projectors 605, 606, 607 interact with the IVI host 604 via the networks 608, 609 to receive or send messages, etc. The detection devices 601, 602, 603 are devices with cameras for collecting the positions of vehicle occupants. The IVI host 604 obtains images indicating the occupant positions from the detection devices 601, 602, 603, and generates a scene image corresponding to which window based on the images indicating the occupant positions, and sends the generated scene image to the projectors 605, 606, 607.

[0112] The IVI host 604 may be a server that provides various services. For example, the IVI host 604 is a back-end management server that provides support for obtaining the occupant position images from the detection devices 601, 602, 603 and analyzing and processing the occupant position images. The back-end management server may analyze and process the obtained occupant position images, and send the processing results, such as scene images, to the projectors 605, 606, 607.

[0113] It should be noted that the window projection method provided by the embodiments of the present invention is generally executed by the IVI host 604.

[0114] It should be understood that Figure 6 the numbers of the detection devices, networks, IVI host, and projectors in

[0115] are merely illustrative. According to the implementation requirements, there may be any number of detection devices, networks, IVI hosts, and projectors.

[0115] Next, refer to Figure 7 , which shows a schematic structural diagram of a computer system 700 of an electronic device suitable for implementing the embodiments of the present invention. Figure 7 The IVI host shown is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0116] As Figure 7As shown, computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the computer system 700 are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0117] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom can be installed into the storage section 708 as needed.

[0118] Specifically, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowchart can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above functions defined in the system of the present invention are executed.

[0119] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0121] The modules involved in the embodiments of the present invention can be implemented in software or in hardware. The described modules can also be provided in a processor. For example, it can be described as: a processor includes a detection module, a processing module, and a projection module, where the detection module is configured to detect the position of an occupant in the vehicle; the processing module is configured to determine a target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information; the projection module is configured to project the scene image onto the target window. Among them, the names of these modules do not constitute a limitation on the modules themselves in some cases. For example, the projection module can also be described as "the module that projects the scene image onto the target window".

[0122] As another aspect, the present invention also provides a computer-readable medium, which can be included in the device described in the above embodiments; or can exist alone without being assembled into the device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the device includes: detecting the position of an occupant in the vehicle through a detection device; determining a target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle through an intelligent cockpit host; determining the environmental information of the area where the vehicle is located, and generating a scene image corresponding to the target window according to the environmental information; projecting the scene image onto the target window through a projector.

[0123] According to the technical solution of the embodiments of the present invention, the position of an occupant in the vehicle is detected through a detection device; a target window corresponding to the occupant is determined from multiple vehicle windows according to the position of the occupant in the vehicle through an intelligent cockpit host; the environmental information of the area where the vehicle is located is determined, and a scene image corresponding to the target window is generated according to the environmental information; the scene image is projected onto the target window through a projector. Thus, the environmental information outside the vehicle window corresponding to the occupant is displayed near the window in the form of an image, so as to provide practical or interesting scene information for the occupant through the image, relieve the boredom of the occupant during the ride, and enrich the riding experience.

[0124] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A window projection system, characterized in that, Including: An intelligent cockpit host, a detection device, and a projector, where the detection device is used to detect the position of the occupant in the vehicle; the intelligent cockpit host is used to determine the target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information; the projector is used to project the scene image onto the target window.

2. The system according to claim 1, characterized in that the detection device is further used to detect the head position and eye focus position of the occupant; the intelligent cockpit host is further used to generate real-time calibration parameters according to the head position and eye focus position of the occupant; calibrate the projection angle of the projector according to the real-time calibration parameters.

3. The system according to claim 2, characterized in that the intelligent cockpit host is further used to determine the interest points of the occupant in the scene image, and provide the detailed information included in the interest points to the projector; the projector is further used to project the detailed information of the interest points on the target window.

4. The system according to claim 3, characterized in that the intelligent cockpit host is further used to use the scene image as the underlying content of augmented reality, and use the detailed information of the interest points as supplementary information, and superimpose it on the upper layer of the scene image; the projector is further used to project the scene image as the underlying layer and the detailed information as the upper layer onto the target window according to a preset priority strategy.

5. The system according to claim 3, characterized in that, The scene image includes various real-world scenes, the intelligent cockpit host is further used to determine the eye focus duration of the occupant for various real-world scenes according to the head position and eye focus position of the occupant collected by the detection device; determine one or more interest points from various scenes according to the eye focus duration of the occupant for various real-world scenes.

6. The system according to claim 3, characterized in that the detection device is further used to collect the gestures of the occupant, finger pointing, or the position where the occupant triggers the target window; the intelligent cockpit host is further used to determine one or more interest points from various real-world scenes displayed on the target window according to the gestures of the occupant, finger pointing, or the position where the occupant triggers the target window.

7. The system according to claim 3, 5 or 6, characterized in that, It further includes a sound collection device, where the sound collection device is used to receive the user request input by the occupant through voice; the intelligent cockpit host is further used to generate a reply message for the user request according to the voice input by the occupant; the projector is further used to display the reply message in the scene image displayed on the target window.

8. The system according to claim 7, wherein It further includes a voice playback module, where the voice playback module is used to play the reply message.

9. The system according to claim 3, characterized in that The intelligent cockpit host is further configured to determine the scene type corresponding to the point of interest, and determine corresponding detailed information according to the venue type; the venue type includes one or more of a shopping mall, a parking lot, a natural landscape, a road, and a cultural and entertainment venue; the detailed information includes one or more of a landmark name, a location name, advertising and promotion information, a parking lot entrance, a navigation route, venue details, a road name and driving tips, a ticket price, a performance time, and a performance schedule.

10. The system according to claim 1, wherein the window includes a window glass and a semi-transparent film embedded in the window glass; the semi-transparent film is configured to display the scene image projected by the projector.

11. The system according to claim 1, wherein the intelligent cockpit host is further configured to extract key features of multiple scenes included in the external environment information of the area where the vehicle is located, and generate a 3D scene map according to the key features; the key features include one or more of the following features: contour, color, texture, material, and visual hierarchy; the projector is further configured to project the 3D scene map onto the target window.

12. The system according to claim 1, wherein the intelligent cockpit host is further configured to obtain map data corresponding to the positioning information from the navigation map data according to the current positioning information of the vehicle; generate scene images corresponding to multiple windows respectively according to the map data and the position relationship between each seat in the vehicle and the corresponding window; the projector is further configured to project the scene images corresponding to multiple windows respectively onto the corresponding windows.

13. The system according to claim 12, wherein the intelligent cockpit host is further configured to calculate a map data slice of the real scene corresponding to each window according to the map data and the position relationship between each seat in the vehicle and the corresponding window; and generate a scene image corresponding to each window according to the map data slice.

14. The system according to claim 12, wherein the intelligent cockpit host is further configured to dynamically update the scene images corresponding to multiple windows respectively according to the real-time updated positioning information of the vehicle; the projector is further configured to project the updated scene images corresponding to multiple windows respectively onto the corresponding windows.

15. The system according to claim 1, wherein the intelligent cockpit host is further configured to determine the content to be displayed input by the user, and provide the content to be displayed to the projector; the projector is further configured to project the content to be displayed onto the sunroof of the vehicle.

16. The system according to claim 1, wherein the intelligent cockpit host is further configured to obtain the current nebula map and the constellation names included in the nebula map corresponding to the current positioning information and the current time according to the current positioning information of the vehicle and the current time; the projector is further configured to project the nebula map and the constellation names correspondingly onto the sunroof of the vehicle.

17. A window projection method, characterized in that, including: detecting the position of the occupant in the vehicle through a detection device; Through the intelligent cockpit host, determine the target window corresponding to the occupant from multiple vehicle windows according to the position of the occupant in the vehicle; determine the environmental information of the area where the vehicle is located, and generate a scene image corresponding to the target window according to the environmental information. Project the scene image onto the target window through a projector.

18. An electronic device for window projection, characterized in that, Comprising: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in claim 17.

19. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the method as described in claim 17 is implemented.

Images