Method and device for indicating target outside vehicle through vehicle window projection, vehicle and storage medium

By projecting the indicator cursor on the car window, the problem of lack of physical reference when explaining the target outside the car is solved, the user experience is improved, and clear instructions for the target outside the car is achieved.

CN119078512APending Publication Date: 2024-12-06GREAT WALL MOTOR CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411372134.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-29
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, the vehicle machine lacks clear physical reference to the target when explaining the target outside the vehicle, resulting in poor user experience and it is difficult for the personnel in the vehicle to determine the specific things described by the vehicle machine.

Method used

By obtaining the eye position of the occupant in the car and the target position outside the car, it is determined whether the connection between the eye and the target has an intersection point with the car window, and project the indicator cursor at the intersection point to indicate the target outside the car.

Benefits of technology

Accurate instructions in the occupant's sight are realized, users' clarity and vehicle experience of the explanation object, and are easy to implement and low cost.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119078512A_ABST
    Figure CN119078512A_ABST
Patent Text Reader

Abstract

The invention is suitable for the technical field of vehicles, and provides a method and device for indicating a target outside a vehicle through vehicle window projection, the vehicle and a storage medium, and the method comprises the steps that the eye position of a passenger in the vehicle and the position of the target outside the vehicle needing to be indicated are obtained; according to the eye position and the position of the target outside the vehicle, a connecting line between the eyes of the passenger in the vehicle and the target outside the vehicle is determined, and whether the connecting line and a window of the vehicle have an intersection point or not is judged; and for the vehicle window having the intersection point with the connecting line, projecting an indication cursor used for indicating a target outside the vehicle to the position where the intersection point of the vehicle window is located. According to the invention, the user can see the target outside the vehicle along the indication cursor, the information of the target outside the vehicle is introduced in cooperation with the voice or screen of the vehicle machine, passengers can clearly know the explained object, and the problem that the prior art lacks a means for clearly indicating the target outside the vehicle referred by the vehicle machine is solved. The method is easy to implement and low in cost, and the vehicle using experience of the user can be well improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicles, and in particular relates to a method, a device, a vehicle and a storage medium for indicating an object outside a vehicle through vehicle window projection. Background Art

[0002] With the increasing intelligence of cars, the improvement of human-computer interaction, and the integration of in-car and out-car scenes, there is an increasing demand for car computers to explain specific matters outside the car. For example, when driving past popular attractions and landmark buildings, some car computers can automatically provide explanations of the attractions based on the user's location, covering background knowledge, local customs, etc.

[0003] The inventor of this application found that the vehicle computer generally uses text or voice to describe something outside the vehicle. However, both methods lack a clear and definite physical reference to the object, and require the occupants to match the object outside the vehicle according to the description of the vehicle computer. In many cases, the occupants may not know what the vehicle computer is describing, resulting in a poor experience. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method, device, vehicle and storage medium for indicating an external target by vehicle window projection, so as to solve the problem in the prior art of lacking clear indication of the external target indicated by the vehicle computer.

[0005] In a first aspect, an embodiment of the present invention provides a method for indicating an external target by projecting a vehicle window, comprising:

[0006] Obtain the eye position of the occupants in the vehicle and the position of the target outside the vehicle that needs to be indicated;

[0007] Determine a line between the eyes of the occupant in the vehicle and the target outside the vehicle according to the eye position and the position of the target outside the vehicle, and judge whether the line intersects with a window of the vehicle;

[0008] For a vehicle window that has an intersection with the connecting line, an indicating cursor for indicating the target outside the vehicle is projected onto the position of the intersection of the vehicle window.

[0009] In combination with the first aspect, in a possible implementation of the first aspect, obtaining the eye position of the occupant in the vehicle includes:

[0010] Acquire a cockpit image of the vehicle;

[0011] The eyes of the occupant are identified according to the cockpit image, and the spatial position of the center point of the eye in the vehicle cockpit is determined to obtain the eye position.

[0012] In combination with the first aspect, in a possible implementation of the first aspect, obtaining a position of an external target to be indicated includes:

[0013] Acquire an environment image outside the vehicle;

[0014] From the environment image, an external target to be indicated is identified, and a spatial position of a center point of the external target relative to the vehicle is determined to obtain the position of the external target.

[0015] In combination with the first aspect, in a possible implementation manner of the first aspect, identifying, from the environment image, an external target that needs to be indicated includes:

[0016] Acquire the conversation of the passengers in the vehicle, extract keywords from the conversation, and obtain target keywords;

[0017] According to the target keyword, an external target to be indicated is identified from the environment image.

[0018] In combination with the first aspect, in a possible implementation manner of the first aspect, the indicating cursor is an indicating frame; and projecting the indicating cursor for indicating the target outside the vehicle to the position of the intersection of the vehicle window includes:

[0019] Determining the size of the indication frame according to the size of the external target in the environmental image;

[0020] Based on the size of the indication frame, the indication frame is projected to the position of the intersection of the vehicle window; wherein the size of the external target is positively correlated with the size of the indication frame.

[0021] In combination with the first aspect, in a possible implementation manner of the first aspect, the method further includes:

[0022] Determine the number of indicator cursors on the car window;

[0023] For vehicle windows with a number of indicator cursors greater than 1, the identification of the corresponding vehicle occupant is displayed at each indicator cursor on the vehicle window.

[0024] In combination with the first aspect, in a possible implementation manner of the first aspect, after projecting an indication cursor for indicating the target outside the vehicle onto the position of the intersection of the vehicle window, the method further includes:

[0025] Determining whether the indicator cursor is in contact with the boundary of the vehicle window;

[0026] For a vehicle window where the indicator cursor contacts the boundary of the vehicle window, the projection of the indicator cursor is canceled.

[0027] In combination with the first aspect, in a possible implementation manner of the first aspect, after projecting an indication cursor for indicating the target outside the vehicle onto the position of the intersection of the vehicle window, the method further includes:

[0028] Acquiring the light intensity outside the vehicle;

[0029] According to the light intensity, the color and / or brightness of the indicator cursor is adjusted.

[0030] In a second aspect, an embodiment of the present invention provides a device for projecting an object outside a vehicle through a vehicle window, comprising:

[0031] An acquisition module, used to acquire the eye position of the occupant in the vehicle and the position of the target outside the vehicle that needs to be indicated;

[0032] A determination module, used to determine a line between the eyes of the occupant in the vehicle and the target outside the vehicle according to the eye position and the position of the target outside the vehicle, and to determine whether the line intersects with a window of the vehicle;

[0033] The projection module is used to project an indication cursor for indicating the target outside the vehicle to the position of the intersection of the vehicle window and the connecting line.

[0034] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the acquisition module is specifically used to:

[0035] Acquire a cockpit image of the vehicle;

[0036] The eyes of the occupant are identified according to the cockpit image, and the spatial position of the center point of the eye in the vehicle cockpit is determined to obtain the eye position.

[0037] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the acquisition module is specifically used to:

[0038] Acquire an environment image outside the vehicle;

[0039] From the environment image, an external target to be indicated is identified, and a spatial position of a center point of the external target relative to the vehicle is determined to obtain the position of the external target.

[0040] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the acquisition module is specifically used to:

[0041] Acquire the conversation of the passengers in the vehicle, extract keywords from the conversation, and obtain target keywords;

[0042] According to the target keyword, an external target to be indicated is identified from the environment image.

[0043] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the indicator cursor is an indicator box; and the projection module is specifically configured to:

[0044] Determining the size of the indication frame according to the size of the external target in the environmental image;

[0045] Based on the size of the indication frame, the indication frame is projected to the position of the intersection of the vehicle window; wherein the size of the external target is positively correlated with the size of the indication frame.

[0046] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the projection module is further used to:

[0047] Determine the number of indicator cursors on the car window;

[0048] For vehicle windows with a number of indicator cursors greater than 1, the identification of the corresponding vehicle occupant is displayed at each indicator cursor on the vehicle window.

[0049] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the projection module is further used to:

[0050] Determining whether the indicator cursor is in contact with the boundary of the vehicle window;

[0051] For a vehicle window where the indicator cursor contacts the boundary of the vehicle window, the projection of the indicator cursor is canceled.

[0052] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the projection module is further used to:

[0053] Acquiring the light intensity outside the vehicle;

[0054] According to the light intensity, the color and / or brightness of the indicator cursor is adjusted.

[0055] In a third aspect, an embodiment of the present invention provides a vehicle, comprising a memory, a controller, and a computer program stored in the memory and executable on the controller, wherein when the controller executes the computer program, the controller implements the steps of the method in the first aspect or any one of the implementations of the first aspect.

[0056] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a controller, the steps of the method in the above-mentioned first aspect or any one of the implementation methods of the first aspect are implemented.

[0057] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0058] When a vehicle needs to point out and explain a specific target outside the vehicle, the embodiment of the present invention obtains the eye position of the occupant in the vehicle and the position of the target outside the vehicle, determines whether the line between the eye of the occupant in the vehicle and the target outside the vehicle intersects with the vehicle window, and projects an indicator cursor for indicating the target outside the vehicle to the position of the intersection of the window with respect to the window that intersects with the line. In this way, in the occupant's line of sight, the indicator cursor can just indicate the target outside the vehicle, so that the user can see the target outside the vehicle by looking along the projection point, and the occupant can clearly know the object of the explanation. The embodiment of the present invention is easy to implement, low-cost, and can well improve the user's vehicle experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0060] Figure 1 is a schematic diagram of an application scenario of a method for indicating an object outside a vehicle through vehicle window projection provided by an embodiment of the present invention;

[0061] Figure 2 The following is a schematic diagram of the implementation process of the method for indicating an external target by window projection provided by an embodiment of the present invention. Figure 1 ;

[0062] Figure 3 Schematic diagram of the vehicle window projection provided by the embodiment of the present invention Figure 1 ;

[0063] Figure 4 Schematic diagram of the vehicle window projection provided by the embodiment of the present invention Figure 2 ;

[0064] Figure 5 The following is a schematic diagram of the implementation process of the method for indicating an external target by window projection provided by an embodiment of the present invention. Figure 2 ;

[0065] Figure 6 is a schematic diagram of the structure of a device for indicating an object outside a vehicle through vehicle window projection provided by an embodiment of the present invention;

[0066] Figure 7 It is a schematic diagram of the structure of a vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0068] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0069] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0070] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0071] The present application is described more clearly below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the effects of the present application, but are not intended to limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present application. These all fall within the scope of protection of the present application.

[0072] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0073] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0074] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0076] In addition, the “plurality” mentioned in the embodiments of the present application should be interpreted as two or more.

[0077] Figure 1 Schematic diagram of an application scenario of the method for indicating an external target through vehicle window projection provided by an embodiment of the present invention. Figure 1 , the application scenarios of this embodiment are introduced.

[0078] In some vehicles, there are scenarios where the car computer points out and explains specific matters outside the car to people. For example, scenario one: when driving past popular attractions or landmark buildings, some car computers or navigation can automatically provide explanations of the attractions based on the location of the user. Or scenario two: the vehicle reminds the driver in the car to pay attention to obstacles in a certain direction. Or scenario three: the vehicle instructs the driver which way to go. There are many similar scenarios, which will not be listed one by one in this embodiment. The present invention is mainly applied to scenario one, and the method of the present invention will be described in detail below using scenario one as an example.

[0079] In these scenarios, the car computer can generally only describe the target through voice or text. There is no way to directly indicate the target to the user through physical means. The user needs to look through the car window to find the target that matches the description of the car computer according to the description of the car computer. As a result, the user may not be able to clearly know what the car computer is referring to. To this end, the present application provides a method for indicating targets outside the vehicle through window projection. The method determines the target outside the vehicle to be recommended and the spatial position of the target through the camera outside the vehicle and the map navigation system, and confirms the spatial position of the occupant's eyes to be recommended through the camera inside the vehicle. A line is connected from the occupant's eyes to the target outside the vehicle, and the intersection of the line on the car window is determined. The projection system projects an indicator cursor to the intersection on the car window, so that the user can see the target outside the vehicle by following the indicator cursor. The information about the target outside the vehicle introduced by voice or text can enable the user to clearly know the object of the explanation.

[0080] like Figure 1As shown, from a hardware perspective, this method needs to utilize in-vehicle cameras, out-vehicle cameras, and a map navigation system to obtain the location information of the vehicle, out-of-vehicle targets, and users, analyze and process the information through a controller, and finally project an indicator mark at an appropriate position on the window through a projection system, thereby directly indicating to the user through the window what the current vehicle computer is describing.

[0081] The above-mentioned in-car camera is mainly used to capture images inside the cockpit, identify and determine the eye position of each person. There is generally one camera installed on the upper front windshield of the vehicle to monitor people in all seats in the car.

[0082] The above-mentioned external cameras are generally multiple in number, and the multiple external cameras are distributed in different directions of the vehicle (such as front, rear, left, and right), so as to monitor the 360-degree range around the vehicle and identify and confirm the spatial position of external targets.

[0083] The above-mentioned map navigation system can provide the geographic coordinates of the vehicle and external targets, so as to determine whether there are popular attractions or landmark buildings that can be explained near the vehicle, and can also determine the relative direction and distance between the vehicle and external targets.

[0084] The above-mentioned controllers may be vehicle controllers, body domain controllers, infotainment system controllers, etc. Generally, the vehicle controller is the core control component of new energy vehicles, and is mainly responsible for functions such as energy management, torque coordination and distribution, motor battery coordination management, safety monitoring, fault diagnosis, and brake energy recovery. It collects the driver's driving signals, receives data on the controller area network (CAN) bus, and makes decisions based on the vehicle's operating status information, and then sends instructions through the CAN bus to control the operation of each component controller to ensure the normal driving of the vehicle. The body domain controller is responsible for the control of the vehicle's body functions, including lighting, windows, door locks, seat control, etc. The infotainment system controller is mainly responsible for the vehicle's infotainment functions, including navigation, audio playback, video playback, etc. It collects various operating instructions from the driver and passengers, processes and responds to provide a rich infotainment experience. This embodiment does not limit which controller processes and executes the method of window projection to indicate targets outside the vehicle.

[0085] The above-mentioned projection system can install a small projection lamp at each window position to project images onto the window. A projection lamp is a device that uses the optical projection principle to project images or patterns onto walls, floors or other surfaces through a high-brightness light source. The principle is to decompose the image into three primary colors through an optical system, process them separately, and then mix and project them through a prism, thereby achieving high color brightness and soft color transition. The projected pattern, color, brightness, and size can all be adjusted.

[0086] In existing vehicles, in-vehicle cameras, external cameras, and map navigation systems are generally standard, so no additional installation is required. Only the projection system needs to be installed, which is low-cost.

[0087] Combine the following Figure 1 For application scenarios, refer to Figure 2 The method for indicating an object outside the vehicle by projecting a vehicle window according to an exemplary embodiment of the present application is described. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0088] Combine the following Figure 2 As shown, the method of this embodiment is described in detail:

[0089] Step S201, obtaining the eye position of the occupant in the vehicle and the position of the target outside the vehicle that needs to be indicated.

[0090] For example, in scenario 1, when driving, the scenic spot explanation function system of the vehicle computer can obtain the geographic coordinates of the vehicle in real time through the map navigation system, and can also detect whether there are scenic spots within a preset distance range around the vehicle, so as to determine whether the vehicle passes by the scenic spot. When it is determined that the vehicle passes by a scenic spot, the data of the scenic spot can be retrieved from the pre-stored database for explanation, or the data of the scenic spot can be searched and downloaded online for explanation. During the period from the beginning of the explanation to the end of the explanation, the scenic spot is the target outside the vehicle that needs to be indicated, and the start and end time of the projection in this embodiment is also during this period. Of course, in other embodiments, it can also be set to project as long as the intersection is detected, and the present invention is not limited thereto.

[0091] Here, the eye positions of the occupants inside the vehicle and the positions of the targets outside the vehicle that need to be indicated are both relative positions in space, rather than geographic coordinates.

[0092] For example:

[0093] Obtaining the eye position of an occupant in a vehicle includes: obtaining a cockpit image of the vehicle; identifying the eyes of the occupant according to the cockpit image, and determining the spatial position of the center point of the eye in the cockpit of the vehicle to obtain the eye position.

[0094] Obtaining the position of the target outside the vehicle that needs to be indicated includes: obtaining an environmental image outside the vehicle; identifying the target outside the vehicle that needs to be indicated from the environmental image, and determining the spatial position of the center point of the target outside the vehicle relative to the vehicle to obtain the position of the target outside the vehicle.

[0095] When there are multiple passengers in the vehicle, the passenger referred to in this method can be any passenger, that is, each passenger is detected. It can be understood that when the spatial position of the external target relative to the vehicle is known (it must have an angle, but may not include the distance), and the spatial position of the eye in the vehicle cabin, the line connecting the passenger in the space and the external target can be determined.

[0096] Here, the current mainstream target recognition algorithm can be used to recognize the eye position based on the cockpit image and the position of the target outside the vehicle based on the environment image.

[0097] Target recognition algorithms can be divided into algorithms based on image processing and deep learning. Algorithms based on image processing usually include steps such as filtering, edge detection, image enhancement, and image segmentation, which are used to preprocess and extract features from images. For example, convolutional neural networks are used for feature extraction, and then the target category and location are determined through a sliding window.

[0098] The target recognition algorithm based on deep learning uses the powerful feature extraction capability of convolutional neural networks to achieve efficient target detection and recognition by generating candidate regions and performing classification and regression. For example, the YOLO algorithm can quickly identify and locate multiple objects in real-time scenarios, with high speed and accuracy.

[0099] In terms of spatial position recognition, image processing algorithms can use lidar and camera data for high-precision map construction and positioning. For example, through binocular stereo vision technology, a system for detecting three-dimensional distance and orientation information of the target space can be designed, and the internal and external parameters of the image acquisition system can be calibrated using prefabricated three-dimensional calibration objects. In addition, algorithms based on deep learning can also be used for high-precision three-dimensional space measurement and positioning. The spatial position of the external target relative to the vehicle can be determined by the position of the external target in the image, as well as the installation angle and position of the camera. This embodiment will not be described in detail.

[0100] Among them, in order to further improve the accuracy of identifying targets outside the vehicle, this embodiment can also obtain the conversation between the passengers in the vehicle, extract keywords from the conversation, and obtain target keywords. Thus, based on the target keywords, the targets outside the vehicle that need to be indicated can be identified from the above-mentioned environmental images.

[0101] Here, this embodiment considers that the scenario where the vehicle computer points out and explains specific matters outside the vehicle to people often involves the conversation between the passengers in the vehicle. Usually, the conversation contains relevant information about the target outside the vehicle that needs to be indicated. Therefore, this embodiment obtains the conversation between the passengers in the vehicle, and based on the conversation, identifies the target outside the vehicle that needs to be indicated from the above-mentioned environmental image. Exemplarily, after obtaining the conversation between the passengers in the vehicle, this embodiment can obtain the geographical location of the vehicle from the navigation system, and then, according to the correspondence between the pre-stored geographical location and the keyword, determine the keyword corresponding to the geographical location of the vehicle, so that, according to the determined keyword, the above-mentioned conversation is keyword extracted to obtain the target keyword, and according to the target keyword, the target outside the vehicle that needs to be indicated is identified from the above-mentioned environmental image.

[0102] In this embodiment, the correspondence between the above-mentioned geographical locations and keywords can be understood as the correspondence between different geographical locations and popular attractions, landmark buildings, etc.

[0103] Step S202, determining the line between the eyes of the occupant in the vehicle and the target outside the vehicle based on the eye position and the target position outside the vehicle, and determining whether the line intersects with the vehicle window.

[0104] In this embodiment, the vehicle window to be judged may be a pre-defined vehicle window that can be projected. For example, when all vehicle windows can be projected, all vehicle windows are judged. In some embodiments, only the side windows can be used as vehicle windows that can be projected, and the front windshield is not projected because it affects the driver's sight, and the rear window is not projected because it is not convenient to watch. Therefore, it is only necessary to detect whether the connecting line has an intersection with each side window, and not to detect whether the connecting line has an intersection with the front windshield and the rear window.

[0105] It should be noted that the connecting line mentioned in this embodiment is a line segment, that is, it is determined whether the line segment has an intersection with the car window, rather than whether the straight line where the line segment is located has an intersection with the car window. Because if the target outside the car is on the left side of the vehicle, the target outside the car cannot be seen from the right window at all, and there is no need for projection indication. If the line segment is used for detection, the window that may have an intersection may only be the left window and there is only one, not the right window. If the straight line where the line segment is located is used for detection, it is very likely that there will be an intersection with the right window, which is unreasonable.

[0106] In addition, this embodiment detects the eye position of each passenger in the car. Due to the different positions of each passenger, at the same time, some passengers can see the target outside the car, and the line connecting their eyes and the target outside the car has an intersection with the car window. Some passengers are blocked by the car body or other obstacles and cannot see the target outside the car, and the line connecting their eyes and the target outside the car does not have an intersection with the car window. In this embodiment, as long as there is an intersection, an indicator cursor is projected, and each passenger can see the target outside the car through the corresponding indicator cursor. Moreover, the indicator cursor of each passenger may be on the same car window or on different car windows. For example, the front seat member sees the target outside the car through the left front car window, and the rear seat member sees the target outside the car through the left rear car window. Their sight lines will form intersections with different car windows, and their corresponding indicator cursors will be distributed on different car windows. Based on this requirement, the above-mentioned small projection lamp can be designed to be able to project multiple indicator cursors at different angles at the same time.

[0107] Step S203: For a vehicle window that has an intersection with the connecting line, project an indication cursor for indicating an object outside the vehicle to the position of the intersection of the vehicle window.

[0108] The present embodiment does not limit the form of the indicator cursor, which may be an indicator arrow, a red dot, an indicator frame, etc. If it is an indicator frame, its size may just surround the target outside the vehicle.

[0109] like Figure 3 As shown in the figure, AB is the line connecting the eye and the target outside the car, and a circular indicator cursor is projected at the intersection of the line AB and the car window. Figure 4 As shown, the eye sight can see the target outside the car through the indicator cursor on the car window, so as to achieve the effect of indicating the target outside the car. It can be understood that in this embodiment, the projection of the car window changes dynamically. As time goes by and the vehicle moves, the eye position and the position of the target outside the car will change, and the intersection of the line AB and the car window will also move, so the position of the indicator cursor will also move. In addition, the timing of the projection is usually determined by the time when the car computer explains the target outside the car. The projection is performed at the beginning of the explanation and the projection is terminated after the explanation. During the explanation, the intersection may not be detected, so there is also the appearance and disappearance of the indicator cursor. Based on this, the indicator cursor may appear from the edge or a certain position in the middle of the car window, move to the edge of the car window and disappear, appear at the edge of another car window after a short interval, and disappear after moving for a period of time.

[0110] It can be seen that when the vehicle has a need to indicate and explain a specific target outside the vehicle, the embodiment of the present invention obtains the eye position of the occupant in the vehicle and the position of the target outside the vehicle, determines whether the line connecting the eye position and the position of the target outside the vehicle intersects with the vehicle window, and projects one or more indicator cursors for indicating the target outside the vehicle to the position of the intersection of the window with the line when the vehicle needs to indicate a specific target outside the vehicle. In the occupant's line of sight, the indicator cursor can just indicate the target outside the vehicle, so that the user can see the target outside the vehicle by looking along the projection point, and with the information of the target outside the vehicle introduced by the voice or screen of the vehicle computer, the occupant can clearly know the object of the explanation. The embodiment of the present invention is easy to implement, low cost, and can well improve the user's vehicle experience.

[0111] Although the above embodiments describe how to project an indicator cursor on a car window to accurately indicate a target outside the car, they do not specifically design a strategy for projecting an indicator cursor, and there are still many problems in practical application. For example:

[0112] (1) What is the appropriate size of the indicator cursor? If it is too small, it may be difficult to notice; if it is too large, it may include too many objects and it may be difficult to distinguish which one is the target outside the vehicle.

[0113] (2) Whether the indicator cursor will illuminate the vehicle body part other than the window.

[0114] (3) How to adjust the color and brightness of the indicator cursor for the most comfortable setting.

[0115] To address these issues, this application also provides Figure 5 See below for the embodiment shown. Figure 5 , the details of this embodiment are described.

[0116] Step S501, obtaining the eye position of the occupant in the vehicle and the position of the target outside the vehicle that needs to be indicated.

[0117] The specific implementation of this step can be found in Figure 2 The description in the embodiments will not be repeated in this embodiment.

[0118] Step S502, determining the line between the eyes of the occupant in the vehicle and the target outside the vehicle based on the eye position and the target position outside the vehicle, and determining whether the line intersects with the vehicle window.

[0119] The specific implementation of this step can be found in Figure 2 The description in the embodiments will not be repeated in this embodiment.

[0120] Step S503, for the vehicle window that intersects with the connecting line, determine the size of the indicator frame according to the size of the external target in the environmental image, and project the indicator frame to the location of the intersection of the vehicle window; wherein the size of the external target is positively correlated with the size of the indicator frame.

[0121] In this embodiment, taking an indication frame as an example, the indication frame can be a circle, a rectangle or any other closed enclosing shape, and the size of the indication frame can be set to just enclose the object outside the vehicle. Since the size of the object outside the vehicle in the environment image is basically the same as what a person sees, the size of the indication frame is determined here by the size of the object outside the vehicle in the environment image.

[0122] Here, the size of the indicator box is also dynamic. As the vehicle moves, the distance between the vehicle and the target outside the vehicle is constantly changing. When the vehicle is getting closer to the target outside the vehicle, the target outside the vehicle appears larger and larger, so the indicator box also becomes larger and larger. You can set it to cancel the projection when the indicator box is larger than a certain threshold. At this time, there is no need to indicate which is the target outside the vehicle. For example, the indicator box is larger than half of the window. When the vehicle is getting farther and farther away from the target outside the vehicle, the target outside the vehicle appears smaller and smaller, so the indicator box also becomes smaller and smaller. You can set it to cancel the projection when the indicator box is smaller than a certain threshold. At this time, the indicator box and the target outside the vehicle are too small to see clearly. The specific threshold can be set according to actual needs.

[0123] Step S504, determining the number of indicator cursors on the vehicle window; for a vehicle window with a number of indicator cursors greater than 1, displaying the corresponding identification of the vehicle occupant at the location of the indicator cursor on the vehicle window.

[0124] according to Figure 1 In step S202 of the embodiment, it can be known that the number of indicator cursors on the vehicle window may be more than one. When there are two or more indicator cursors on a vehicle window, from the perspective of a certain passenger, the two indicator cursors indicate different things, and the passenger may not be able to distinguish which is his own indicator cursor, nor which is the object outside the vehicle. In this case, the corresponding passenger's identification can be marked at each indicator cursor.

[0125] The identification method is not limited in this embodiment, and can be, for example, the following methods:

[0126] Text marking: "Left front", "Left rear", "Right front", and "Right rear" are written in small characters after the indicator cursor. "Left front" refers to the occupant in the left front seat, "Left rear" refers to the occupant in the left rear seat, "Right front" refers to the occupant in the right front seat, and "Right rear" refers to the occupant in the left rear seat.

[0127] Color identification: Passengers in different seats are indicated by different colored cursors.

[0128] Shape identification: Occupants in different seats are indicated by different colored cursors.

[0129] etc...

[0130] In this embodiment, the occupant's identification can be determined based on the spatial position of the center point of the occupant's eyes in the vehicle cabin, such as the above-mentioned text identification "left front", "left rear", "right front", "right rear", etc., or color identification, with different spatial positions using indicator cursors of different colors.

[0131] In addition to marking the corresponding occupant at each indicator cursor on the window with a number of indicator cursors greater than 1, this step may also be replaced by:

[0132] The total number of indicator cursors on all vehicle windows is determined; if the total number of indicator cursors on all vehicle windows is greater than 1, the corresponding occupant identification is displayed at each indicator cursor.

[0133] This can more effectively indicate the corresponding windows and indicator cursors to the passengers.

[0134] Step S505, determining whether the indicating cursor is in contact with the boundary of the vehicle window; if so, canceling the projection of the indicating cursor.

[0135] In this embodiment, when the indicator cursor is projected to the intersection, the center point of the indicator cursor usually coincides with the intersection. Considering that the indicator cursor has a certain size, when the center point of the indicator cursor is too close to the edge of the window, part of the indicator cursor may fall outside the window, which is not very beautiful. Since when the center point of the indicator cursor is too close to the edge of the window, the target outside the vehicle will generally disappear or partially disappear from the window, so no indication is required. That is, it is determined whether the indicator cursor touches the boundary of the window; if so, the projection of the indicator cursor is canceled, so that part of the indicator cursor can be prevented from falling outside the window.

[0136] This step can also be replaced by measuring the short distance between the indicator cursor and the window boundary, but the shape and size of the indicator cursor need to be considered during the measurement. Since the algorithm is relatively complex, this method is not adopted in this embodiment, but this measurement method should also be included in the protection scope of the present invention.

[0137] Step S506, obtaining the light intensity outside the vehicle; and adjusting the color and / or brightness of the indicator cursor according to the light intensity.

[0138] In this embodiment, the color and / or brightness of the projection lamp can be adjusted according to the environment to achieve the best visual effect. This is because the ambient light has a certain influence on the projection effect. For example, in a dark environment, the projection effect will be clearer and brighter, while in an environment with the lights on, the brightness of the picture will be reduced and the color will be basically normal. In addition, the light conditions of the surrounding environment will also interfere with the effect of the projection lamp. In an environment with too strong light, the projection effect may be reduced. Therefore, although the projection lamp can adjust the color and brightness to adapt to the environment, the intensity and conditions of the ambient light will still affect the projection effect.

[0139] In this embodiment, a high-performance light sensor can be used, which includes an ambient light sensor and a color temperature sensor, which can accurately detect the intensity and color temperature of the ambient light. This sensor has performance advantages such as high sensitivity, ultra-short exposure, high-speed sampling and precise synchronization. The ambient light sensor can sense the light conditions of the surrounding environment and timely process the chip to adjust the required display backlight brightness. In the projection system, the ambient light sensor can detect the light intensity of the surrounding environment and automatically adjust the projection brightness and color temperature according to the changes in the ambient light. The projection system requires an intelligent adjustment mechanism to automatically adjust the projection brightness and color temperature according to the changes in the ambient light. For example, when the ambient light is dim, the projection system can increase the brightness to ensure that the image is clearly visible; when the ambient light is bright, the projection system can reduce the brightness to avoid overexposure of the image. In order to ensure the accuracy of the light sensor, it needs to be calibrated. A standard light source can be used to simulate light under different lighting conditions, including various light intensities and color temperatures indoors and outdoors, to provide a standard test environment for the ambient light sensor. Through calibration, the sensor performance can be verified to ensure its accuracy and stability in different environments. After the hardware design is completed, corresponding software and intelligent perception algorithms need to be developed to achieve intelligent control of the projection system and ensure that the projection system can automatically adjust according to changes in ambient light.

[0140] Through the method of indicating the target outside the vehicle through the vehicle window projection of this embodiment, on the one hand, when there is a need to point out and explain a specific target outside the vehicle, by projecting an indicator cursor for indicating the target outside the vehicle at a suitable position on the vehicle window, the indicator cursor can just indicate the target outside the vehicle in the sight of the occupant, so that the user can see the target outside the vehicle by looking along the projection point, and the information of the target outside the vehicle can be introduced by the voice or screen of the vehicle computer, so that the occupant can clearly know the object of the explanation. On the other hand, according to the information such as the size of the target outside the vehicle, the number of indicator cursors, and the change of the external light intensity, the form and size of the indicator cursor are dynamically adjusted to ensure the rationality and comfort of the indicator cursor, further improving the user experience.

[0141] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0142] Figure 6 FIG. 6 is a schematic diagram of the structure of a device 60 for projecting an object outside a vehicle through a vehicle window provided by an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0143] An acquisition module 61 is used to acquire the eye position of the occupant in the vehicle and the position of the target outside the vehicle that needs to be indicated;

[0144] A determination module 62 is used to determine the line between the eyes of the occupant in the vehicle and the target outside the vehicle according to the eye position and the position of the target outside the vehicle, and to determine whether the line intersects with the window of the vehicle;

[0145] The projection module 63 is used to project an indication cursor for indicating an object outside the vehicle onto the position of the intersection of the vehicle window with the connecting line.

[0146] As a possible implementation manner, the acquisition module 61 is specifically used for:

[0147] Acquire a vehicle's cabin image;

[0148] According to the cockpit image, the eyes of the occupant are identified, and the spatial position of the center point of the eye in the vehicle cockpit is determined to obtain the eye position.

[0149] As a possible implementation manner, the acquisition module 61 is specifically used for:

[0150] Acquire an environmental image outside the vehicle;

[0151] From the environment image, the external target to be indicated is identified, and the spatial position of the center point of the external target relative to the vehicle is determined to obtain the position of the external target.

[0152] As a possible implementation manner, the acquisition module 61 is specifically used for:

[0153] Obtain the conversations of the passengers in the car, extract keywords from the conversations, and obtain target keywords;

[0154] According to the target keywords, the external targets that need to be indicated are identified from the environment image.

[0155] As a possible implementation, the indicating cursor is an indicating box; the projection module 63 is specifically used for:

[0156] Determine the size of the indication frame according to the size of the target outside the vehicle in the environment image;

[0157] Based on the size of the indication frame, the indication frame is projected to the location of the intersection of the vehicle window; wherein the size of the target outside the vehicle is positively correlated with the size of the indication frame.

[0158] As a possible implementation, the projection module 63 is further used for:

[0159] Determine the number of indicator cursors on the car window;

[0160] For vehicle windows with a number of indicator cursors greater than 1, the identification of the corresponding vehicle occupant is displayed at each indicator cursor on the vehicle window.

[0161] As a possible implementation, the projection module 63 is further used for:

[0162] Determine whether the indicating cursor is in contact with the boundary of the vehicle window;

[0163] For a vehicle window where the indicator cursor contacts the boundary of the vehicle window, the projection of the indicator cursor is canceled.

[0164] As a possible implementation, the projection module 63 is further used for:

[0165] Get the light intensity outside the vehicle;

[0166] Adjust the color and / or brightness of the indicator cursor according to the light intensity.

[0167] When a vehicle needs to point out and explain a specific target outside the vehicle, the embodiment of the present invention obtains the eye position of the occupant in the vehicle and the position of the target outside the vehicle, determines whether the line connecting the eye position and the position of the target outside the vehicle intersects with the vehicle window, and if the line intersects with any window, projects an indicator cursor for indicating the target outside the vehicle to the position of the intersection. In the occupant's line of sight, the cursor can just indicate the target outside the vehicle, so that the user can see the target outside the vehicle by looking along the projection point, and with the information of the target outside the vehicle introduced by the voice or screen of the vehicle computer, the occupant can clearly know the object of the explanation. The embodiment of the present invention is easy to implement, low cost, and can well improve the user's vehicle experience.

[0168] Figure 7 FIG. 7 is a schematic diagram of a vehicle 70 provided in an embodiment of the present invention. The vehicle in this embodiment includes an in-vehicle camera, an out-vehicle camera, a map navigation system, and a projection system. Figure 7 As shown, the vehicle 70 of this embodiment further includes: a controller 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the controller 71, such as a program for projecting an object outside the vehicle through a vehicle window. When the controller 71 executes the computer program 73, the steps in the above-mentioned method for projecting an object outside the vehicle through a vehicle window are implemented, such as Figure 2 Alternatively, when the controller 71 executes the computer program 73, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of modules 61 to 63 are shown.

[0169] Exemplarily, the computer program 73 may be divided into one or more modules / units, which are stored in the memory 72 and executed by the controller 71 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 73 in the vehicle 70. For example, the computer program 73 may be divided into an acquisition module 61, a determination module 62, and a projection module 63 (a module in a virtual device), and the specific functions of each module are as follows:

[0170] An acquisition module 61 is used to acquire the eye position of the occupant in the vehicle and the position of the target outside the vehicle that needs to be indicated;

[0171] A determination module 62 is used to determine the line between the eyes of the occupant in the vehicle and the target outside the vehicle according to the eye position and the position of the target outside the vehicle, and to determine whether the line intersects with the window of the vehicle;

[0172] The projection module 63 is used to project an indication cursor for indicating an object outside the vehicle onto the position of the intersection of the vehicle window with the connecting line.

[0173] The vehicle 70 may include, but is not limited to, a controller 71 and a memory 72. Those skilled in the art will appreciate that Figure 7 This is only an example of the vehicle 70 and does not constitute a limitation of the vehicle 70 . The vehicle 70 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the vehicle 70 may also include input and output devices, network access devices, buses, etc.

[0174] The controller 71 may be a central processing unit (CPU), or other general controllers, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general controller may be a microcontroller or any conventional controller, etc.

[0175] The memory 72 may be an internal storage unit of the vehicle 70, such as a hard disk or memory of the vehicle 70. The memory 72 may also be an external storage device of the vehicle 70, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the vehicle 70. Further, the memory 72 may include both an internal storage unit and an external storage device of the vehicle 70. The memory 72 is used to store the computer program and other programs and data required by the vehicle 70. The memory 72 may also be used to temporarily store data that has been output or is to be output.

[0176] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0177] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0178] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0179] In the embodiments provided by the present invention, it should be understood that the disclosed devices / vehicles and methods can be implemented in other ways. For example, the device / vehicle embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0182] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the controller. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0183] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for indicating an object outside a vehicle by projecting a vehicle window, characterized in that: include: Obtain the eye position of the occupants in the vehicle and the position of the target outside the vehicle that needs to be indicated; According to the eye position and the position of the target outside the vehicle, a line between the eye of the occupant inside the vehicle and the target outside the vehicle is determined, and the line is determined to be Whether there is an intersection with the vehicle's window; For a vehicle window that has an intersection with the connecting line, an indicating cursor for indicating the target outside the vehicle is projected onto the position of the intersection of the vehicle window.

2. The method for indicating an object outside the vehicle by projecting a vehicle window as claimed in claim 1, characterized in that: Obtain the eye position of the occupants in the vehicle, including: Acquire a cockpit image of the vehicle; The eyes of the occupant are identified according to the cockpit image, and the spatial position of the center point of the eye in the vehicle cockpit is determined to obtain the eye position.

3. The method for indicating an object outside the vehicle by projecting a vehicle window as claimed in claim 1, characterized in that: Obtain the external target position that needs to be indicated, including: Acquire an environment image outside the vehicle; From the environment image, an external target to be indicated is identified, and a spatial position of a center point of the external target relative to the vehicle is determined to obtain the position of the external target.

4. The method for indicating an object outside the vehicle by projecting a vehicle window as claimed in claim 3, characterized in that: The step of identifying an external target to be indicated from the environment image includes: Acquire the conversation of the passengers in the vehicle, extract keywords from the conversation, and obtain target keywords; According to the target keyword, an external target to be indicated is identified from the environment image.

5. The method for indicating an object outside the vehicle by projecting a vehicle window as claimed in claim 3 or 4, characterized in that: The indicating cursor is an indicating frame; the indicating cursor projected toward the position of the intersection of the vehicle window for indicating the target outside the vehicle includes: Determining the size of the indication frame according to the size of the external target in the environmental image; Based on the size of the indication frame, the indication frame is projected to the position of the intersection of the vehicle window; wherein the size of the external target is positively correlated with the size of the indication frame.

6. The method for indicating an object outside the vehicle by projecting a vehicle window according to any one of claims 1 to 4, characterized in that: After projecting an indication cursor for indicating the target outside the vehicle onto the position of the intersection of the vehicle window, the method further includes: Determining the number of indicator cursors on the vehicle window; For vehicle windows with a number of indicator cursors greater than 1, the identification of the corresponding vehicle occupant is displayed at each indicator cursor on the vehicle window.

7. The method for indicating an object outside the vehicle by projecting a vehicle window according to any one of claims 1 to 4, characterized in that: After projecting an indication cursor for indicating the target outside the vehicle onto the position of the intersection of the vehicle window, the method further includes: Determining whether the indicator cursor is in contact with the boundary of the vehicle window; For a vehicle window where the indicator cursor contacts the boundary of the vehicle window, the projection of the indicator cursor is canceled.

8. A device for projecting a vehicle window to indicate an object outside the vehicle, characterized in that: include: An acquisition module, used to acquire the eye position of the occupant in the vehicle and the position of the target outside the vehicle that needs to be indicated; A determination module, used to determine a line between the eyes of the occupant in the vehicle and the target outside the vehicle according to the eye position and the position of the target outside the vehicle, and to determine whether the line intersects with a window of the vehicle; The projection module is used to project an indication cursor for indicating the target outside the vehicle to the position of the intersection of the vehicle window and the connecting line.

9. A vehicle comprising a memory, a controller, and a computer program stored in the memory and executable on the controller, characterized in that: When the controller executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a controller, the steps of the method according to any one of claims 1 to 7 are implemented.