Attention position sharing method and vehicle

Abstract

The invention provides an attention position sharing method and a vehicle, and the method comprises the steps: determining an attention position of a first passenger outside the vehicle according to the motion information of the first passenger, and the first passenger is located in a first region of the vehicle; according to the attention position, a target image is displayed on a target window corresponding to a second passenger, the target image is used for displaying information corresponding to the attention position, the second passenger is located in a second area of the vehicle, and the second area is different from the first area.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent vehicle technology, and in particular to a method and a vehicle for sharing a location of interest. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, 5G communications, and big data, smart cars are becoming the focus of the global automotive industry. Smart cars are no longer a simple means of transportation, but are integrated with big data, artificial intelligence, etc., allowing passengers to gain more experience and enjoyment beyond driving. One of the experiences of riding a car is that you can enjoy the scenery outside the car window and discuss the scenery outside the window with fellow passengers. Then, the different positions of passengers in the car cause different fields of view. When one of the passengers starts a topic about the scenery outside the window, other passengers in the car may not pay attention to the scenery referred to by the topic initiator in time, resulting in poor communication. Summary of the invention

[0003] In view of this, an embodiment of the present disclosure provides a method and a vehicle for sharing a location of interest.

[0004] According to a first aspect of the present disclosure, a method for sharing a location of interest is provided, comprising:

[0005] determining a focus position of the first passenger outside the vehicle according to the motion information of the first passenger, the first passenger being located in a first area of ​​the vehicle;

[0006] A target image is displayed on a target window corresponding to a second passenger according to the focus position, the target image being used to display information corresponding to the focus position, the second passenger being located in a second area of ​​the vehicle, and the second area being different from the first area.

[0007] In some embodiments, the target image includes a logo image, and displaying the target image on a target window corresponding to the second passenger according to the focus position includes:

[0008] Determining a target position on the target vehicle window corresponding to the focus position;

[0009] The logo image is displayed at the target location.

[0010] In some embodiments, determining the target position on the target window corresponding to the focus position includes: determining the target position on the target window according to a line connecting the eye position of the second passenger and the focus position.

[0011] In some embodiments, determining a target position on the target vehicle window corresponding to the focus position includes:

[0012] Capturing an image of an area outside the vehicle to obtain an image of a scene outside the vehicle, wherein the focus position is located in the area outside the vehicle;

[0013] Displaying the exterior scene image on the target vehicle window;

[0014] The target position is determined according to a mapping position of the focus position in the external scene image.

[0015] In some embodiments, the target image includes a regional scene image, and displaying the target image on a target window corresponding to the second passenger according to the focus position includes:

[0016] Capturing an image of the area corresponding to the focus position to obtain a scene image of the area;

[0017] The regional scene image is displayed on the target vehicle window.

[0018] In some embodiments, displaying the regional scene image on the target vehicle window includes:

[0019] In response to the focus position being outside the visible area corresponding to the target vehicle window, the regional scene image is displayed on the target vehicle window.

[0020] In some embodiments, the action information includes a click position on the first vehicle window corresponding to the click action of the first passenger, and determining the focus position of the first passenger outside the vehicle based on the action information of the first passenger includes: determining the focus position based on the click position.

[0021] In some embodiments, the motion information includes line of sight information of the first passenger; and determining the focus position of the first passenger outside the vehicle based on the motion information of the first passenger includes: determining the focus position based on the line of sight information.

[0022] In some embodiments, determining the first passenger's focus position outside the vehicle according to the first passenger's motion information includes:

[0023] Obtaining voice information of the first passenger;

[0024] Processing the speech information to obtain semantic information;

[0025] The focus position is determined according to the semantic information and the action information.

[0026] According to a second aspect of the present disclosure, there is provided a vehicle, comprising:

[0027] An information collection device, used to collect movement information of the first passenger;

[0028] a determining device, for determining a focus position of the first passenger outside the vehicle according to the motion information, the first passenger being located in a first area of ​​the vehicle;

[0029] The target window corresponding to the second passenger is used to display a target image, and the target image is used to display information corresponding to the focus position; the second passenger is located in a second area of ​​the vehicle, and the second area is different from the first area. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 1 ;

[0031] Figure 2 A schematic diagram of a device for implementing a method for sharing a location of interest provided by an embodiment of the present disclosure;

[0032] Figure 3 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 2 ;

[0033] Figure 4 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 3 ;

[0034] Figure 5 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 4 ;

[0035] Figure 6 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 5 ;

[0036] Figure 7 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 6 ;

[0037] Figure 8 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 7 ;

[0038] Fig. 9 A schematic diagram of a method for sharing a location of interest provided in an embodiment of the present disclosure Figure 8 ;

[0039] Fig.10 FIG9 is a flow diagram of a method for sharing a location of interest provided by an embodiment of the present disclosure;

[0040] Fig.11 A schematic diagram of seats in a commercial vehicle provided in an embodiment of the present disclosure;

[0041] Fig.12 An overall schematic diagram of a vehicle provided in an embodiment of the present disclosure;

[0042] Fig.13 It is a partial schematic diagram of the interior of a vehicle provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the specific embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0044] In the following description, a large number of specific details are given to provide a more thorough understanding of the present disclosure. However, it is obvious to those skilled in the art that the present disclosure can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present disclosure, some technical features known in the art are not described; that is, all features of actual embodiments are not described here, and well-known functions and structures are not described in detail.

[0045] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be a limitation of the present disclosure. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0046] When riding in a vehicle, multiple passengers may discuss the scenery outside the car window. However, due to the different positions of different passengers in the car, they have different perceptions of the position and size of the scenery outside the car window. For example, when the front passenger says to look at the tree next to him, the tree may be located in front of his left or right for the back passenger, so the back passenger may not be able to accurately lock onto the tree mentioned by the front passenger, which may lead to poor communication between passengers and affect the driving experience.

[0047] In view of this, an embodiment of the present disclosure provides a viewpoint sharing method, which can, for example, share the viewpoint of a front-row passenger with a rear-row passenger, thereby making communication between the front-row and rear-row passengers smoother. Figure 1 A flowchart of a method for sharing a location of interest provided by an embodiment of the present disclosure is shown in FIG. Figure 1As shown, the method includes:

[0048] S100: determining a focus position of the first passenger outside the vehicle according to the motion information of the first passenger, the first passenger being located in a first area of ​​the vehicle;

[0049] S200: Displaying a target image on a target window corresponding to the second passenger according to the focus position, wherein the target image is used to display information corresponding to the focus position, and the second passenger is located in a second area of ​​the vehicle, which is different from the first area.

[0050] By way of example, the focus position may be the point of sight of the first passenger outside the vehicle, i.e., the gaze point. The first passenger's gaze point may fall on an area, such as a landscape or a street scene, or may fall on a specific object, such as a building, an animal, a flower, a tree, etc. By way of example, the focus position may be the position of a target object determined based on the point of sight, and the position of the target object may not coincide with the position of the viewpoint. By way of example, the target object may be the object closest to the point of sight. By way of example, the focus position may be a position selected by the user through operations such as clicking.

[0051] The first area and the second area may be two areas located on the same side of the vehicle and arranged side by side front and back, or they may be two areas located on different sides of the vehicle. The difference between the first area and the second area is reflected in the different fields of view of the passengers when they are located in the first area and the second area. By way of example, the first area and the second area may correspond to seats in the vehicle. For example, the first area and the second area are two seats in the vehicle, respectively close to the left and right windows, and these two seats may be in the same row or in different rows. For another example, the first area and the second area may also be two seats in the front and rear rows on the same side of the vehicle.

[0052] For example, the window near the first area is different from the target window near the second area. To distinguish it from the target window, the window near the first area is defined as the first window. In step S100, the focus position of the first passenger outside the first window is determined according to the action information of the first passenger. In step S200, a target image indicating the information corresponding to the focus position is displayed on the target window corresponding to the second passenger according to the focus position of the first passenger outside the first window. In this way, the information at the focus position of the first passenger is presented to the second passenger through the target window, so that when the first passenger talks about the information at the focus position, the second passenger can quickly know it, so as to facilitate the exchange of information between the first passenger and the second passenger, making the communication smoother.

[0053] In some embodiments, the action information of the first passenger includes the click position corresponding to the click action of the first passenger on the first window, such as Figure 2As shown, step S100 can be implemented in the following manner: S101, determining the focus position of the first passenger outside the vehicle according to the click position corresponding to the click action of the first passenger on the first vehicle window.

[0054] For example, the implementation method of step S101 includes: performing image capture on at least the visible area of ​​the first window to obtain a first projected image that overlaps with the scene outside the first window in the field of view of the first passenger, and displaying the first projected image on the first window; determining the focus position of the first passenger based on the mapping position of the click position on the first projected image.

[0055] For example, capturing an image of at least the visible area of ​​the first window to obtain a first projected image can be achieved in the following manner: capturing an image of the area outside the vehicle including the visible area of ​​the first window to obtain an image outside the vehicle; obtaining eye information of the first passenger, and obtaining a first projected image that overlaps with the scene outside the first window in the field of view of the first passenger based on the image outside the vehicle and the eye information of the first passenger.

[0056] Figure 3 A schematic diagram of a device for implementing a method for sharing a location of interest is shown below. Figure 3 For explanation. Figure 3 As shown, the vehicle is provided with an image acquisition device 110, which is, for example, a 360-degree camera or one or more 2D cameras. Each camera captures the scene in the external environment of the vehicle. By way of example, the 360-degree camera can be arranged on the roof of the vehicle to capture the scene around the vehicle body. By way of example, multiple cameras can be installed around the first window corresponding to the first passenger and the target window corresponding to the second passenger, or a 360-degree camera can be used to capture the scene in the area on both sides of the vehicle body including the visible area of ​​the window.

[0057] The image information (including pictures or images) collected by the image acquisition device 110 can be stored in the processor 120 and processed by the processor 120 to obtain a first projection image. For example, the processor 120 can be a central processor of the vehicle or a remote server, that is, the collected image information is sent to the remote server for processing to save the occupancy of the vehicle central processor and reduce power consumption.

[0058] An information collection device 130 using passenger motion information is provided inside the vehicle. The information collection device 130 can obtain eye information of the first passenger. Eye information includes eye position, etc. The processor can receive the eye position obtained by the information collection device, and based on the eye position, extract the first projection image from the scene image outside the vehicle. For example, when the processor processes the scene image outside the vehicle to obtain the first projection image, it should also consider the distance of the scenery outside the window relative to the first passenger, so that the relative position and size of each scene in the first projection image are consistent with the relative position and size of the real scene outside the window. That is, in the field of vision of the first passenger, the first projection image overlaps with the real scene outside the first window, achieving a virtual-real fusion effect.

[0059] For example, augmented reality (AR) technology can be used to display a first projected image on the first vehicle window that is integrated with the real scene outside the vehicle window. Augmented reality technology is a technology that superimposes and integrates virtual scenes or information with the real physical environment and interactively presents them to the user, thereby creating a virtual and real space sharing the same space.

[0060] The first projection image is displayed on the first window using augmented reality technology, so that the first passenger will not be disturbed by the first projection image when viewing the scenery outside the vehicle, thereby obtaining a better visual experience. Moreover, after seeing the scenery of interest, the first passenger can directly click on the window to determine the focus position without having to switch the viewpoint back and forth between the scenery outside the vehicle and the first window, thus obtaining a better sharing experience.

[0061] In some embodiments, the first projection image can be projected onto the first vehicle window using HUD (Head-Up Display) technology. For example, the first projection image can be projected onto the first vehicle window using thin film transistor liquid crystal display technology (TFT-LCD), digital light processing technology (DLP) and laser scanning technology. The first vehicle window is designed to be suitable as a display screen to carry the first projection image. The display device 140 includes the first vehicle window and other optoelectronic display devices for implementing HUD. In other embodiments, transparent display materials can also be used to display the first projection image on the first vehicle window. For example, glass can be bonded to a transparent OLED display screen to form a transparent vehicle window, which can not only show the wind direction outside the vehicle to the passengers, but also display the first projection image. In this embodiment, the display device 140 includes a first vehicle window integrated with a transparent OLED display screen.

[0062] When the first passenger enjoys the scenery outside the vehicle through the first window, when he sees something of interest, he can point out his focus position by clicking on the first window. The information collection device 130 in the vehicle can obtain the click position of the passenger on the first window and transmit it to the processor. The processor can determine the focus position of the first passenger according to the mapping position of the click position on the first projected image.

[0063] For example, a touch film can also be integrated on the vehicle window to obtain a transparent vehicle window assembly integrated with glass, a transparent OLED display and a touch film, which can realize both information display function and interaction function with passengers. For example, the touch film can be a transparent nano touch film. The touch film can obtain the click position of the passenger on the first vehicle window, and the processor can determine the focus position of the first passenger on the first projection image based on the click position fed back by the touch film and the first projection image.

[0064] In some embodiments, the motion information includes the first passenger's sight information, such as Figure 4 As shown, step S100 can be implemented in the following manner: S102: Determine the focus position of the first passenger outside the vehicle according to the sight line information of the first passenger.

[0065] For example, eye tracking technology can be used to determine the first passenger's focus position. Eye tracking technology captures the user's eye movement trajectory for analysis and calculates the eye's line of sight direction or viewpoint. In one specific embodiment, eye tracking technology can be used to obtain the first passenger's line of sight direction, and the first passenger's focus position is determined based on the intersection of the line of sight direction and the first projected image of the first window. In another specific embodiment, the first projected image may not be displayed on the first window, but the first passenger's focus position may be determined directly based on the line of sight direction and the image outside the vehicle obtained by the image acquisition device 110.

[0066] In some embodiments, step S100 can be implemented in the following manner: determine the focus position according to the eye position of the first passenger and the click position on the first window. It can be understood that when the first passenger sees an interesting scene outside the car and wants to point out the interesting scene to his fellow passengers, his finger will follow the position of his sight and point to the scene. At this time, his eye position, finger position and the interesting scene are in a straight line. If the finger falls on the first window, then his eye position, the click position of the finger on the window and the interesting scene are also in the same straight line. Based on this, the focus position can be determined in the image outside the car or the first projected image on the first window by obtaining the eye position of the first passenger and the click position on the first window, and then based on the straight line where the eye position and the click position are located.

[0067] In some embodiments, step S100 may also be performed in the following manner: in response to the voice information of the first passenger, the first passenger's focus position outside the vehicle is determined according to the action information of the first passenger within a preset period of time before and / or after the first passenger utters the voice.

[0068] It is understandable that in some cases, the first passenger may just stare out the window in a daze, and not talk about the scenery outside the window. In order to reduce the probability of misoperation, the voice information of the first passenger can be used as an instruction to control the information collection device 130 to obtain the action information of the first passenger. That is, when it is detected that the first passenger is speaking, the action information of the first passenger obtained by the information collection device 130 is processed. For example, the processor can only process the action information of the first passenger in the first period of time before the first passenger starts to speak. Before the first passenger speaks to discuss a certain scene, he must have seen the scene first, so the processor can determine the focus position according to the line of sight information of the first passenger before speaking. For example, the processor can only process the action information of the first passenger in the second period of time after the first passenger starts to speak. When the first passenger discusses a certain scene outside the car with a fellow passenger, in order to point out the scene to the fellow passenger, some guiding actions will be made, for example, the finger may be pointed to the scene, or the finger clicks on the first window to point to the scene, or the first passenger's head turns to the first window and the line of sight is fixed on the scene outside the car, and the sight of the fellow passenger is guided to the focus position through these actions. Then, the processor can determine the first passenger's focus position based on the position of the first passenger's finger on the first window after the first passenger utters the voice, or action information such as line of sight information. As another example, the processor can process the action information from the first period when the first passenger starts to utter the voice to the second period after the first passenger starts to utter the voice, so that by processing more action information, the first passenger's focus position can be determined more accurately.

[0069] In some embodiments, the processor may also detect a wake-up word in the voice information of the first passenger, and use the wake-up word as an instruction to control the information collection device 130 to obtain the action information of the first passenger. The wake-up word may include words in the voice information that guide fellow passengers to pay attention to the first passenger's focus position. For example, look, outside, outside the car, etc. The processor determines the first passenger's focus position based on the action information in a preset period before and after the first passenger issues the wake-up word.

[0070] In some embodiments, Figure 5 As shown, step S100 can also be implemented in the following manner: S103, obtaining voice information of the first passenger, S104, processing the voice information to obtain semantic information, S105, determining the focus position according to the semantic information and the action information.

[0071] For example, the voices of multiple passengers can be acquired through a recording device, and the voice information of the first passenger can be separated from the voices of multiple passengers through a voice separation device using voice separation technology. In a specific embodiment, recording devices can be set at multiple locations in the car, and the voice information of the first passenger can be obtained by acquiring the voice in the recording device in the first area where the first passenger is located. It can be understood that the first passenger is closest to the recording device in the first area, so the voice of the first passenger in the recording device in the first area is louder and clearer, and the noise is relatively small. When using voice separation to process the voice of the recording device, the voice information of the first passenger can be obtained more easily and accurately, and the time for voice separation processing can be shortened.

[0072] Next, the voice information of the first passenger is processed to obtain semantic information. For example, semantic information can be identified from the collected voice information through voice recognition technology. Specifically, when the voice of the first passenger includes a certain scene outside the window, the processor will recognize the voice information of the first passenger, determine the scene the first passenger is talking about, and search for the scene from the first projected image. For example, if the first passenger says, "The leaves of the ginkgo tree outside have turned yellow, so beautiful", then the processor can recognize from the voice of the first passenger that the ginkgo tree outside the window is currently being talked about, and then find the ginkgo tree through image recognition technology from the first projected image that overlaps with the real scene in the first passenger's field of view.

[0073] In some embodiments, semantic recognition technology may also be used when identifying voice information from voice information. Semantic recognition technology is smarter and more accurate than voice recognition technology. Semantic recognition technology can combine context, big data, etc. to make more accurate judgments. In this embodiment, semantic recognition technology can locate the target more accurately. For example, if the first passenger says, "Look, the ginkgo tree in the front is the largest and most beautiful," then the processor can not only identify the ginkgo tree through voice recognition technology, but also use semantic recognition technology to further select the largest ginkgo tree from the multiple ginkgo trees in the first passenger's field of view to provide more accurate recognition results. As described above, the first passenger's focus position can be determined based only on the first passenger's voice information.

[0074] In other embodiments, the position of interest can also be determined based on the voice information combined with the action information. The action information includes the click position on the first window corresponding to the click action of the first passenger mentioned above, and / or the line of sight information of the first passenger. It is understandable that in some cases, the voice information of the first passenger may not have a clear name of the scene, but some pronouns, such as that, this, etc., are used to express it, resulting in that the use of semantic information alone may not be able to locate the position of interest of the first passenger. At this time, the action information and the semantic information can be combined. It is understandable that when the first passenger uses pronouns to talk about the scenery outside the car, in order to make the passengers with him understand the scenery he is pointing to, he may use actions to guide the passengers with him to look at the scenery he is concerned about. At this time, the first passenger's position of interest can be determined by combining the action information and semantic information of the first passenger.

[0075] In some embodiments, the semantic information of the first passenger may be obtained first. When the semantic information indicates that the first passenger is talking about the scene outside the vehicle but the focus position cannot be determined, the motion information of the first passenger is obtained and the focus position is determined based on the motion information of the first passenger.

[0076] After determining the first passenger's location of interest, Figure 1 , Figure 3 , Figure 4 and Figure 5 As shown, step S200 is then performed: a target image is displayed on a target window corresponding to the second passenger according to the focus position, and the target image is used to display information corresponding to the focus position.

[0077] For example, the target window corresponding to the second passenger is different from the first window corresponding to the first passenger. That is, the target window is not the first window. The target window may be the window closest to the second passenger.

[0078] In the disclosed embodiment, the information corresponding to the first passenger's focus position is displayed on the target window corresponding to the second passenger through the target image, so that the second passenger can know the information that the first passenger is currently focusing on. Then, when the two communicate about the scenery outside the car, it can be guaranteed that they are communicating about the information at the same focus position, avoiding communication problems due to different focus points. In other words, the disclosed embodiment synchronizes the first passenger's viewpoint, which can also be called the focus point, to the second passenger in real time through the target window, so that the communication between the two is smoother, so as to obtain a better riding experience in the car.

[0079] In some embodiments, the target image includes a logo image, such as Figure 6 As shown, step S200 can be implemented in the following manner: S201, determining a target position on the target vehicle window corresponding to the focus position; S202, displaying a logo image at the target position.

[0080] Here, the target position corresponds to the focus position, which means that the target position points out the focus position on the target window. The target position can be located on the line connecting the eye position of the first passenger and the focus position, or not, as long as the focus position can be pointed out to the second passenger.

[0081] The identification image may mark the location of interest, or may mark the information at the location of interest. For example, the identification image may be a positioning icon used in map software, which is set at the target location. For another example, the identification image may be an outline frame or a bounding box, which circles the target object at the location of interest. For example, when the target object is a building, the building may be circled by a square bounding box or a circular bounding box, or by a contour frame that is the same as the outline of the building.

[0082] In this embodiment, the logo image indicates the first passenger's focus position on the target window, so that the second passenger can know the information of the focus position, which is convenient for the second passenger to communicate with the first passenger. For example, the logo image can be displayed on the target window using HUD technology, and a transparent OLED display screen can be integrated on the glass of the target window to display the logo image.

[0083] In some embodiments, Figure 7 As shown, step S201 can be implemented in the following manner: S2011, determining a target position at a target window according to a line connecting the eye position and the focus position of the second passenger.

[0084] In this embodiment, the scene outside the vehicle that the second passenger sees through the target vehicle window is required to include information at the focus position (such as the target object at the focus position).

[0085] For example, the first area and the second area are areas arranged side by side on the same side of the vehicle, that is, the second passenger and the first passenger sit on the same side of the vehicle. Specifically, the first passenger and the second passenger sit on the left side of the vehicle at the same time or on the right side of the vehicle at the same time. For example, the first passenger sits in the co-pilot seat of a family car, and the second passenger sits behind the co-pilot seat. In this case, most of the scenery in the first passenger's field of view can appear in the second passenger's field of view, but due to the different perspectives of the first passenger and the second passenger, the scene seen at the same time is not exactly the same relative to the two. In addition, when talking about distant scenery, it is difficult to provide accurate location or feature positioning of the target object; or, when talking about one of some scenery with similar features, it is also difficult to quickly locate the target object. For example, when the first passenger says on the highway, "Look, there are two birds on the eaves of the house in the distance", due to the large number of houses in the field of view, the second passenger may find it difficult to quickly locate the house that the first passenger is talking about.

[0086] When most of the scene in the first passenger's field of vision can appear in the second passenger's field of vision, the second passenger's view outside the vehicle through the target window can include information at the focus position. In this case, only the identification image can be displayed on the target window without displaying the information at the focus position. It should be understood that the target position and the focus position should overlap in the second passenger's field of vision, so that the location of the target object can be accurately pointed out by setting the identification image at the target position.

[0087] Based on this, the coordinate information of the eye position (i.e., eye position) of the second passenger and the coordinate information of the focus position can be obtained, and the target position can be determined according to the intersection of the line between the eye position and the focus position and the target window. For example, the coordinate information of the eye position of the second passenger can be obtained by an information acquisition device. The image outside the vehicle can be obtained by an image acquisition device, and the coordinate information of the focus position can be obtained in the image outside the vehicle. After obtaining the position information of the surface where the target window is located, the target position can be determined according to the coordinate information of the focus position, the coordinate information of the eye position of the second passenger, and the position information of the surface where the target window is located.

[0088] In other embodiments, Figure 8 As shown, step S201 can be implemented in the following ways: S2012, capturing images of the area outside the vehicle to obtain an image of the scene outside the vehicle, with the focus position being located in the area outside the vehicle; S2013, displaying the image of the scene outside the vehicle on the target vehicle window; S2014, determining the target position based on the mapping position of the focus position in the image of the scene outside the vehicle.

[0089] In this embodiment, the scenery including the area at the focus position in the area outside the vehicle is collected to obtain an image of the scene outside the vehicle. In the image of the scene outside the vehicle, not only the information at the focus position (for example, the target object) but also other scenery can be seen, so that the second passenger can see more scenery, which is conducive to understanding the communication content of the first passenger on the information at the focus position, and can make communication smoother.

[0090] In some embodiments, the image of the scene outside the vehicle may be a first projection image corresponding to the real scene in the field of vision of the first passenger. That is, the second passenger can see what the first passenger sees, and the focus position must have a mapping position in the first projection image, and the target position can be determined. It should be understood that if there is a focus position in the real scene seen by the second passenger through the window, the focus position and the mapping position on the target window and the eye position of the second passenger are not necessarily in a straight line.

[0091] In other embodiments, the image of the scene outside the vehicle may not be the first projection image, but an image obtained by photographing the area outside the vehicle including the area at the focus position from other angles. The image of the scene outside the vehicle only needs to clearly show the scenery in the area at the focus position.

[0092] In some other embodiments, the vehicle exterior scene image may be a second projected image corresponding to the real scene in the second passenger's field of view, and the second projected image is superimposed on the real scene in the second passenger's field of view. Fig. 9 As shown, step S201 can be implemented in the following manner: S2015, at least capture an image of the visible area of ​​the target window to obtain a second projection image overlapping with the scene in the second passenger's field of view; S2016, display the second projection image on the target window; S2017, in response to focusing on a visible area where the target window exists, determine the target position according to the mapping position of the focus position in the second projection image.

[0093] For example, the second projected image can be obtained by: capturing images of the area outside the vehicle including the visible area of ​​the target vehicle window to obtain an image outside the vehicle; obtaining the eye information of the second passenger, and obtaining a second projected image overlapping with the scene outside the target vehicle window in the field of view of the second passenger based on the image outside the vehicle and the eye information of the second passenger. For example, the eye information of the second passenger includes the eye position. For example, augmented reality technology can be used to merge the second projected image with the real scene in the field of view of the second passenger without causing a sense of disharmony.

[0094] As shown above, when the second passenger and the first passenger sit on the same side of the car, most of the scenery in the first passenger's field of view can appear in the second passenger's field of view. Therefore, a second mapping image overlapping with the real scene in the second passenger's field of view can be obtained, and the target position can be determined in the second mapping image. At this time, the second passenger's eye position, target position and focus position are on the same straight line. The advantage of this setting is that when the second mapping image appears on the car window, the scene in the eyes of the second passenger will not change, and the second passenger can quickly see the focus position. And it can reduce the physical discomfort caused by the second passenger's gaze switching back and forth between the real scene outside the car and the target window, for example, reducing the second passenger's feeling of motion sickness.

[0095] In some embodiments, augmented reality technology can be used to display a second mapping image overlapping with the real scene outside the vehicle on the target window corresponding to the second passenger for a long time. When the first passenger talks about the information of the location of interest outside the vehicle, the identification image can be quickly displayed on the second mapping image.

[0096] Continue to see Fig. 9In some cases, when the focus position is outside the visible area corresponding to the target window, that is, the focus position is not within the area seen by the second passenger through the target window, the method further includes: S2018, in response to the focus position being outside the visible area of ​​the target window, obtaining an adjusted second projection image based on the adjusted eye information of the second passenger, and determining the target position according to the mapping position of the focus position in the adjusted second projection image. For example, the eye information includes eye position.

[0097] In this embodiment, when the focus position is not within the area that the second passenger can see through the target window, the processor can control the information collection device to obtain the second passenger's action information. If the second passenger wants to continue communicating with the first passenger about the focus position, the second passenger can actively adjust his own position to change the eye position and thus change the visible area of ​​the target window until the focus position appears in the adjusted visible area, and then display the information of the focus position on the adjusted second projection image. If the second passenger does not move his own position, that is, does not change the eye position, it means that the second passenger may not want to continue communicating with the first passenger, and there is no need to display the identification image corresponding to the focus position on the target window. Here, it should be noted that the visible area of ​​the window refers to the area outside the car that the passenger sees through the window, and as the passenger's eye position moves, the visible area of ​​the window changes accordingly.

[0098] In this embodiment, when the focus position is outside the visible area corresponding to the target window, the action information of the second passenger is used as an instruction, and in response to the second passenger adjusting the eye position until the focus position appears in the adjusted visible area, the identification image is displayed on the adjusted second projection image. In response to the second passenger not adjusting the eye position until the focus position appears in the visible area, the identification image is not displayed. This setting method is more respectful of the wishes of the second passenger.

[0099] In some embodiments, the target image includes an area scene image, such as Fig.10 As shown, step S201 can be implemented in the following manner: S203, collecting images of the area corresponding to the focus position to obtain a regional scene image; S204, displaying the regional scene image on the target vehicle window.

[0100] The area corresponding to the focus position may be an area within a preset radius with the focus position as the center. The present disclosure does not limit the size of the preset radius, and it can be set according to actual needs. In different scenarios, the preset radius may remain unchanged or may vary. For example, depending on the distance of the focus position, the radius may be changed based on the principle of smaller at a distance and larger at a near distance. The area corresponding to the focus position may also be the area where the target object is located, that is, image acquisition is performed on the target object at the focus position.

[0101] Directly displaying the regional scene image on the target window can enable the second passenger to quickly obtain information about the location of interest to the first passenger. In addition, the amount of image data processed by the processor is small, which can speed up the display of the target image on the target window, and the processor consumes less computing power and power consumption.

[0102] In some embodiments, step 201 may be implemented in the following manner: in response to the focus position being outside the visible area corresponding to the target vehicle window, displaying a regional scene image on the target vehicle window.

[0103] For example, in combination with the above-mentioned embodiment of displaying the second projection image on the target window, step S2018 can be changed to: in response to the focus position being outside the visible area corresponding to the target window, the regional scene image is displayed on the target window. Specifically, in response to the focus position being within the visible area corresponding to the target window, the second projection image and the identification image are displayed on the target window; in response to the focus position being outside the visible area corresponding to the target window, the second projection is switched to the regional scene image. In this way, regardless of whether the focus position is within the visible area corresponding to the target window, the second passenger can see the information at the focus position without adjusting the eye position, which takes the convenience of operation of the second passenger into consideration.

[0104] As another example, this step can be combined with the above step S2011. Specifically, in response to the focus position being within the visible area corresponding to the target window, the target position is determined on the target window according to the line between the eye position of the second passenger and the focus position, and the identification image is displayed at the target position; in response to the focus position being outside the visible area corresponding to the target window, an image is captured of the area corresponding to the focus position to obtain a regional scene image, and the regional scene image is displayed on the target window.

[0105] In this embodiment, when the second passenger can see the focus position, the identification image is directly displayed, so that the second passenger can clearly see the information at the focus position, and the visual experience is better; when the second passenger cannot see the focus position, the regional scene image corresponding to the focus position is directly displayed, so that the second passenger can know the viewpoint of the first passenger, facilitating smooth communication between the two.

[0106] In some other embodiments, the target image may not be displayed in response to the focus position being within the visible area corresponding to the target window; the regional scene image may be displayed on the target window in response to the focus position being outside the visible area corresponding to the target window. That is, when the focus position is within the field of view of the second passenger, the focus position is not shared; when the focus position is outside the field of view of the second passenger, the regional scene image corresponding to the focus position is directly displayed. This can also help the second passenger know the focus point of the first passenger in some cases, but the experience is relatively poor.

[0107] In some embodiments, a viewpoint sharing mode selection button may be integrated in the vehicle, and the mode selection button may switch between multiple viewpoint sharing modes. The multiple viewpoint sharing modes may correspond to the above different embodiments, such as the above Figure 1 , Figure 2 , Figures 4 to 10 The embodiment shown. For example, the first viewpoint sharing mode can display the information of the focus position in the second projected image when the focus position is located in the visible area corresponding to the target window, and not display the information of the focus position when the focus position is located outside the visible area corresponding to the target window. The second viewpoint sharing mode can display the information of the focus position in the second projected image when the focus position is located in the visible area corresponding to the target window, and switch to the regional scene image when the focus position is located outside the visible area corresponding to the target window. The third viewpoint sharing mode can always use the method of displaying the regional scene image. In addition, the mode selection button can control whether the viewpoint sharing function is turned on or off. When the viewpoint sharing function is turned on, a viewpoint sharing mode can be selected for viewpoint sharing. When the viewpoint sharing function is not turned on, the target image is not displayed on the target window, and the function of the window glass is simply realized.

[0108] In some embodiments, the vehicle also includes a third passenger, and when the third passenger and the first passenger are located on different sides of the vehicle, the visible area of ​​the second target window corresponding to the third passenger does not include the position of interest. Then in one embodiment, an image can be collected for the area corresponding to the position of interest to obtain a regional scene image, and the regional scene image is displayed on the second target window. In another embodiment, an image of the scene outside the vehicle (i.e., the first projected image) overlapping with the real scene in the field of view of the first passenger is displayed on the second target window, the target position is determined according to the mapping position of the position of interest in the image of the scene outside the vehicle, and an identification image is displayed at the target position.

[0109] Fig.11 Schematic diagram of the seat arrangement in a commercial vehicle provided by the embodiment of the present disclosure. The following uses the method for sharing the location of interest provided by the embodiment of the present application in a commercial vehicle scenario as an example to explain a method for sharing the location of interest. Fig.11 As shown, the passengers in the vehicle include passenger A, passenger B and passenger C, wherein passenger A and passenger B sit in the front and rear seats on the same side, and passenger A and passenger C sit in the left and right seats of the same row respectively. Passenger A is close to the first window 210, passenger B is close to the second window 220, and passenger C is close to the third window 230.

[0110] When passenger A, passenger B and passenger C all turn on the viewpoint sharing function, augmented reality technology is used to project the scene outside the first window 210 corresponding to passenger A and superimpose it on the first window 210, to project the scene outside the second window 220 corresponding to passenger B and superimpose it on the second window 220, and to project the scene outside the third window 230 corresponding to passenger C and superimpose it on the third window 230, so that a combination of virtual and real scenes appear in the fields of vision of passengers A, passenger B and passenger C.

[0111] When passenger A actively clicks on the scene in the first window 210 with his finger, his focus position is determined. Then the processor can mark the focus position of passenger A in the scene in the second window 220 seen by passenger B. If the focus position is not displayed in the scene in the second window 220 seen by passenger B, passenger B can adjust his position until the focus position of passenger A is displayed on the second window; if passenger B does not adjust his position, the focus position of passenger A will not be displayed in the corresponding scene in the second window 220.

[0112] When passenger A actively clicks on the scene on the first window 210 with his finger, his focus position is determined. Then the processor can also share the scene on the first window 210 corresponding to passenger A and the marked focus position with the third window 230 corresponding to passenger C.

[0113] The method for sharing the focus location provided by the embodiment of the present disclosure can share the focus location of the first passenger with fellow passengers through the window near the fellow passengers. When the first passenger starts a topic about the scene at the focus location, the fellow passengers can quickly know and clearly understand the content of the discussion, thereby making communication between each other smoother and obtaining a better riding experience.

[0114] The present disclosure also provides a vehicle, Fig.12 This is the overall schematic diagram of the vehicle. Fig.13 It is a partial schematic diagram of the interior of the vehicle, such as Fig.12 and Fig.13 As shown, the vehicle 10 includes an information collection device 130, a determination device, and a target window 141 corresponding to the second passenger. The information collection device 130 is used to collect the motion information of the first passenger; the determination device is used to determine the focus position of the first passenger outside the vehicle based on the motion information, and the first passenger is located in the first area of ​​the vehicle; the target window 141 is used to display a target image, and the target image is used to display information corresponding to the focus position; the second passenger is located in the second area of ​​the vehicle, and the second area is different from the first area.

[0115] In some embodiments, the vehicle 10 includes a vehicle central processing unit (not shown in the figure), and the vehicle central processing unit can be used as the determining device. For example, the vehicle central processing unit is arranged below the dashboard.

[0116] For example, the motion information of the first passenger includes eye information such as the eye position and sight information of the first passenger and click position information corresponding to the click action of the first passenger on the first window 142. The information collection device 130 is also used to obtain the motion information of the second passenger. The motion information of the second passenger includes the eye information of the second passenger, etc. For example, the information collection device 130 includes a touch film for obtaining the click position of the first passenger. The information collection device 130 includes a camera (such as an infrared camera) for obtaining eye information.

[0117] For example, the vehicle 10 further includes an image acquisition device 120 for obtaining an image outside the vehicle, an image of a scene outside the vehicle, and an image of a regional scene. The image of the scene outside the vehicle includes a first projection image and a second projection image.

[0118] For example, the vehicle 10 further includes a display device 140 for displaying a target image, an image of an exterior scene, an image of an area scene, etc. The display device 140 may include a first window 142, a target window 141, and other optoelectronic display devices for implementing a HUD, or the display device 140 may include a first window 142 integrated with a transparent OLED display screen, a target window 141 integrated with a transparent OLED display screen, and the like.

[0119] For example, the vehicle 10 further includes a voice collection device for collecting voice information of passengers. The voice collection device may be one or more recording devices.

[0120] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method for sharing the location of interest in the embodiment of the present application.

[0121] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method for sharing a location of interest provided in the embodiment of the present application.

[0122] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0123] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0124] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0125] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for sharing a location of interest, the method comprising: determining a focus position of the first passenger outside the vehicle according to the motion information of the first passenger, the first passenger being located in a first area of ​​the vehicle; A target image is displayed on a target window corresponding to a second passenger according to the focus position, the target image being used to display information corresponding to the focus position, the second passenger being located in a second area of ​​the vehicle, and the second area being different from the first area.

2. The method according to claim 1, wherein the target image comprises a logo image, and displaying the target image on the target window corresponding to the second passenger according to the focus position comprises: Determining a target position on the target vehicle window corresponding to the focus position; The logo image is displayed at the target location.

3. The method according to claim 2, wherein determining a target position on the target vehicle window corresponding to the focus position comprises: A target position is determined at the target window according to a line connecting the eye position of the second passenger and the focus position.

4. The method according to claim 2, wherein determining a target position on the target vehicle window corresponding to the focus position comprises: Capturing an image of an area outside the vehicle to obtain an image of a scene outside the vehicle, wherein the focus position is located in the area outside the vehicle; Displaying the exterior scene image on the target vehicle window; The target position is determined according to the mapping position of the focus position in the image of the external scene.

5. The method according to claim 1, wherein the target image comprises a regional scene image, and displaying the target image on a target window corresponding to the second passenger according to the focus position comprises: Capturing an image of the area corresponding to the focus position to obtain a scene image of the area; The regional scene image is displayed on the target vehicle window.

6. The method according to claim 5, wherein displaying the regional scene image on the target vehicle window comprises: In response to the focus position being outside the visible area corresponding to the target vehicle window, the regional scene image is displayed on the target vehicle window.

7. The method according to claim 1, wherein the action information includes a click position on the first window corresponding to the click action of the first passenger, and the determining the focus position of the first passenger outside the vehicle according to the action information of the first passenger comprises: The focus position is determined according to the click position.

8. The method according to claim 1, wherein the motion information includes the sight line information of the first passenger; and determining the focus position of the first passenger outside the vehicle according to the motion information of the first passenger comprises: The focus position is determined according to the line of sight information.

9. The method according to claim 1, wherein determining the first passenger's focus position outside the vehicle according to the first passenger's motion information comprises: Obtaining voice information of the first passenger; Processing the speech information to obtain semantic information; The focus position is determined according to the semantic information and the action information.

10. A vehicle comprising: An information collection device, used to collect movement information of the first passenger; a determining device, for determining a focus position of the first passenger outside the vehicle according to the motion information, the first passenger being located in a first area of ​​the vehicle; A target window corresponding to the second passenger, used to display a target image, wherein the target image is used to display information corresponding to the focus position; The second passenger is located in a second area of ​​the vehicle, the second area being different from the first area.

Images