Vehicle control method and device, vehicle, terminal, medium and program product

Abstract

The present disclosure relates to a vehicle control method, device, vehicle, terminal, medium and program product, the method comprising: determining pointing information of a terminal; in a case where it is determined according to the pointing information that the terminal points to a target device of a vehicle, outputting a pointing prompt on a display interface corresponding to the target device; and executing a function corresponding to a control instruction sent by the terminal according to the control instruction, the control instruction being generated according to an operation of the terminal. In this way, in various scenarios where it is difficult to directly manipulate a display screen, manipulation of any display screen can be conveniently completed. The convenience and user experience are improved.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle engineering technology, and in particular to a vehicle control method, device, vehicle, terminal, medium, and program product. Background Technology

[0002] Vehicle displays typically support touch control and voice control. However, touch control is less convenient in some scenarios, and voice control is not suitable for all scenarios, and in some scenarios, it cannot successfully obtain voice commands to complete the control. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a vehicle control method, device, vehicle, terminal, medium, and program product.

[0004] According to a first aspect of the present disclosure, a vehicle control method is provided, comprising: Determine the terminal's pointing information; If the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, a pointing prompt is output on the display interface corresponding to the target device. The system executes the function corresponding to the control command sent by the terminal, the control command being generated based on the operation on the terminal.

[0005] According to a second aspect of the present disclosure, a vehicle control method is provided, comprising: When the terminal is pointed at the target device of the vehicle, it receives the operation on the terminal. The control command corresponding to the operation is sent to the vehicle. The control command is used by the vehicle to execute the function corresponding to the control command when the display interface corresponding to the target device outputs a pointing prompt.

[0006] According to a third aspect of the present disclosure, a vehicle control device is provided, comprising: The determination module is configured to determine the terminal's pointing information; The display module is configured to output a pointing prompt on the display interface corresponding to the target device when the terminal is determined to be pointing to the target device of the vehicle based on the pointing information. The execution module is configured to perform a function corresponding to a control instruction sent by the terminal, the control instruction being generated based on the operation on the terminal.

[0007] According to a fourth aspect of the present disclosure, a vehicle control device is provided, comprising: The receiving module is configured to receive operations on the terminal when the terminal is pointed at a target device in the vehicle. The sending module is configured to send a control command corresponding to the operation to the vehicle. The control command is used by the vehicle to execute the function corresponding to the control command when the display interface corresponding to the target device outputs a pointing prompt.

[0008] According to a fifth aspect of the present disclosure, a vehicle is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement the method of any one of the first aspects.

[0009] According to a sixth aspect of the present disclosure, a terminal is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement the method of any one of the second aspects.

[0010] According to a seventh aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects, or, when executed by a processor, implements the steps of the method described in any one of the second aspects.

[0011] According to an eighth aspect of the present disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects, or, when executed by a processor, implements the steps of the method described in any one of the second aspects.

[0012] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: The vehicle determines the terminal's pointing information, and when it determines that the terminal is pointing at a target device on the vehicle based on the pointing information, it outputs a pointing prompt on the display interface corresponding to the target device; then it receives control commands sent by the terminal, and executes the functions corresponding to the control commands. In this way, for scenarios where it is difficult to directly operate the central control screen, such as when the front seats are reclined or rear passengers are seated, users can easily operate the vehicle's display screen through the terminal without moving or adjusting their posture, greatly improving the convenience and comfort of operation. Thus, through the interaction between the terminal and the vehicle, in various scenarios where it is difficult to directly control the display screen, it is possible to conveniently operate any display screen, improving convenience and user experience.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0015] Figure 1 This is a flowchart illustrating a vehicle control method on the vehicle side according to an exemplary embodiment.

[0016] Figure 2 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart for step S11.

[0017] Figure 3 This is a flowchart illustrating a vehicle control method on the terminal side according to an exemplary embodiment.

[0018] Figure 4 This is a flowchart illustrating a method for generating display instructions according to an exemplary embodiment.

[0019] Figure 5 This is a schematic diagram illustrating a vehicle control method according to an exemplary embodiment.

[0020] Figure 6 This is a block diagram illustrating a vehicle control device on the vehicle side according to an exemplary embodiment.

[0021] Figure 7 This is a block diagram illustrating a vehicle control device on the terminal side according to an exemplary embodiment.

[0022] Figure 8 This is a block diagram illustrating a vehicle according to an exemplary embodiment.

[0023] Figure 9 This is a block diagram illustrating a terminal device according to an exemplary embodiment. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0025] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0026] Before introducing the vehicle control method provided in the embodiments of this disclosure, let's first introduce the technical problems involved in the relevant scenarios of this disclosure. When the central control screen or PHUD (Panoramic Head-up Display) screen supports touch, although it reduces the space occupied compared to physical buttons, it is difficult for users to touch the display screen when the seat is reclined, resulting in the inability to achieve touch control. Therefore, the ease of use is relatively low. In scenarios where voice control is used, such as in noisy environments, voice commands cannot be accurately obtained, and voice control cannot be achieved.

[0027] In view of this, this disclosure provides a vehicle control method aimed at improving the convenience of controlling in-vehicle displays and enhancing user experience.

[0028] Figure 1 This is a flowchart illustrating a vehicle control method according to an exemplary embodiment, which can be applied to vehicles, such as... Figure 1 As shown, it includes the following steps.

[0029] In step S11, the terminal's pointing information is determined.

[0030] The terminal can be a mobile phone, tablet computer, remote control, or similar device equipped with technologies such as UWB (Ultra Wide Band) and NFC (Near Field Communication). The pointing information refers to the terminal's position and orientation relative to the display screen. This can be determined using image data captured by sensors on the terminal (such as gyroscopes, accelerometers, magnetometers, etc.) or a camera. In this embodiment, the terminal's motion state and surrounding environment information can be captured in real time using built-in sensors or a camera. After processing and analysis, the precise pointing information of the terminal relative to the vehicle's display screen can be calculated.

[0031] In this embodiment of the disclosure, the pointing information of the terminal can be determined by sensors, such as cameras and infrared sensors, to identify the position and direction of the terminal, or by wireless signals, such as Bluetooth and Wi-Fi Direct, to identify the position and direction of the terminal, and then the pointing information of the terminal can be obtained based on the position and direction.

[0032] The directional information can be determined by the terminal itself or by the vehicle based on sensors on the terminal. If the terminal determines the directional information, it needs to send the directional information to the vehicle system via, for example, Wi-Fi Direct, after determining that the target device is on the vehicle. Upon receiving the directional information, the vehicle system immediately displays a directional prompt on the target device's display interface.

[0033] If the vehicle determines the pointing information, it can automatically acquire the terminal's pose and thus determine whether the terminal is pointing at the target device on the vehicle. Assuming the vehicle has already determined the smartphone's pointing information relative to the central control screen, and that the pointing information indicates the terminal is pointing at the central control device's display, it can continuously monitor the smartphone's pose. As the smartphone moves or rotates, the cursor position is dynamically updated, always remaining at the position the smartphone is pointing at.

[0034] In step S12, if it is determined that the terminal is pointing to the target device of the vehicle based on the pointing information, a pointing prompt is output on the display interface corresponding to the target device.

[0035] The display interface can be the user interface of an in-vehicle display device, such as the display screen of the central control unit, or the projection display interface on the windshield of a PHUD (Panoramic Head-up Display).

[0036] In this embodiment, when the terminal is determined to be pointing at a target device on the vehicle based on the pointing information, a pointing prompt is output on the display interface corresponding to the target device. For example, a cursor can be displayed on the screen in real time, and the position of the cursor is consistent with the pointing of the terminal. Following the above embodiment, when the terminal is determined to be pointing at a target device on the vehicle based on the pointing information, a cursor is displayed on the central control screen, and the position of the cursor moves with the movement of the smartphone. Thus, the position of the cursor on the display screen can be controlled by moving the smartphone. For another example, suppose a user is using a smartphone as a terminal and wants to control the vehicle's PHUD. When the user points the smartphone at the PHUD or at the PHUD's projection interface on the windshield, the smartphone's camera or built-in sensors identify the smartphone's pointing and position. Subsequently, a cursor is displayed on the PHUD's projection interface on the windshield, and the position of the cursor moves with the movement of the smartphone. Thus, the position of the cursor on the projection interface can be controlled by moving the smartphone. By determining the terminal's pointing information and providing pointing prompts on the display interface, intuitive interaction between the terminal and the vehicle's display screen is achieved. This interaction method not only improves the convenience and flexibility of operation.

[0037] In step S13, the function corresponding to the control command sent by the terminal is executed. The control command is generated based on the operation on the terminal.

[0038] Control commands are instructions generated by the terminal and sent to the vehicle to control it to perform corresponding functions or operations. The user interface is the interface displayed on the terminal for user interaction and may include elements such as buttons, sliders, and menus. Vehicle control can be achieved by operating the vehicle system to control vehicle functions, such as adjusting volume, changing songs, and turning on the air conditioning.

[0039] In this embodiment of the disclosure, when the terminal is pointed at the target device in the vehicle, a communication connection can be established between the terminal and the vehicle according to a communication and control protocol. Then, when the user operates through the terminal's user interface (e.g., clicking a button, using a slider), the terminal generates corresponding control commands and sends them to the vehicle system via wireless signals such as Bluetooth, hotspot, or near-field communication. Upon receiving the control commands, the vehicle system executes corresponding vehicle control operations on the display interface corresponding to the target device, based on the content of the commands.

[0040] Continuing the example above, suppose a user wants to select and play a song on the central control screen. The user selects a song through their smartphone's user interface (such as a music app or a dedicated application) and clicks the "play" button. At this point, the smartphone generates a control command and sends it wirelessly to the vehicle system. Upon receiving the command, the vehicle system selects the corresponding song using the cursor on the central control screen and begins playback. In this way, the vehicle's music playback function can be successfully controlled via a smartphone.

[0041] The above-described technical solution involves the vehicle determining the terminal's pointing information and, if the terminal is pointing towards a target device on the vehicle based on this information, outputting a pointing prompt on the display interface corresponding to the target device. The vehicle then receives control commands from the terminal and executes the corresponding functions based on these commands. This allows users to easily control the vehicle's display screen via the terminal in scenarios where it is difficult to directly operate the central control screen, such as when the front seats are reclined or rear passengers are seated, without needing to move or adjust their posture. This significantly improves the convenience and comfort of operation. Through the interaction between the terminal and the vehicle, any display screen can be easily controlled even in situations where direct operation is difficult, thus enhancing convenience and user experience.

[0042] Optionally, see Figure 2 As shown, in step S11, determining the terminal's pointing information includes: In step S111, the first coordinates of the terminal in the vehicle body coordinate system are determined; The vehicle coordinate system can be a coordinate system centered on the vehicle, with its center point as the origin. The vehicle's forward direction is the positive X-axis, the positive Y-axis is perpendicular to the ground and points to the left of the vehicle, and the positive Z-axis is perpendicular to the XY plane and points upward. The first coordinate is the position coordinate of the terminal in the vehicle coordinate system.

[0043] In this embodiment of the disclosure, the precise position of the terminal relative to the vehicle body can be obtained by a sensor (such as LiDAR, camera combined with image recognition, or directly calculated by the installation position) and converted into coordinate values ​​in the vehicle body coordinate system.

[0044] In step S112, the attitude information of the terminal in the vehicle coordinate system is determined based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; The first rotation information refers to the overall rotation angle of the vehicle relative to the world coordinate system, which can include the vehicle's pitch, yaw, and roll angles, and can be represented by Euler angles relative to the world coordinate system. The second rotation information refers to the rotation angle of the terminal itself relative to the world coordinate system, which can also be represented by Euler angles, including pitch, yaw, and roll angles. Attitude information describes the terminal's orientation and tilt state in the vehicle's coordinate system and can be represented as a rotation matrix or quaternions.

[0045] In this embodiment, inertial sensors such as a compass, level, gyroscope, and accelerometer, combined with GPS data (for correcting long-term drift), are used to calculate the rotation information of the vehicle and the terminal. By combining the two, the final attitude of the terminal relative to the vehicle body coordinate system can be calculated.

[0046] In step S113, the pointing information of the terminal is determined based on the first coordinates, the attitude information, and the second coordinate set of the target device.

[0047] The second coordinate set is the set of coordinates of the target device. The second coordinate set may include one coordinate to represent the coordinates of the target device or the display interface corresponding to the target device, such as the center point coordinates of the target device or the display interface corresponding to the target device. The second coordinate set may also include multiple coordinates to represent the coordinates of the target device or the display interface corresponding to the target device.

[0048] In this embodiment, the attitude information of the terminal is transformed from the vehicle coordinate system to the world coordinate system. Then, the terminal's position in the world coordinate system (obtained through the first coordinate transformation) and the second coordinate set of the target device are combined to calculate the absolute position and orientation of the terminal in the world coordinate system through geometric transformation.

[0049] The above technical solution determines the terminal's position and orientation in the vehicle body coordinate system, and combines this with the coordinates of the display screen to ultimately determine the pointing information of the terminal (or a specific axis on it) relative to the external environment. This specific axis on the terminal can be, for example, the center line, the line containing the left frame, or the line containing the right frame.

[0050] Optionally, in step S111, determining the first coordinates of the terminal in the vehicle body coordinate system includes: Determine the distances between the multiple positioning modules on the vehicle and the terminal; The vehicle can be equipped with at least three positioning modules, for example, one each on the left A-pillar, right A-pillar, and left C-pillar. The positioning modules can measure the distance between the terminal and the positioning module using technologies such as TOF (Time of Flight), which measures the time it takes for a signal to travel between two terminals and the positioning module. Alternatively, they can use technologies such as TDOA (Time Difference of Arrival), which measures the time difference in signal propagation from the terminal to different positioning modules to determine the distance between the positioning module and the terminal. Finally, they can use TOA (Time of Arrival), which measures the time it takes for a signal to travel from the terminal to the positioning module to determine the distance between the positioning module and the terminal.

[0051] Based on the third coordinates of each positioning module and the corresponding distance, the first coordinate of the terminal in the vehicle coordinate system is determined, where the third coordinates are the pre-calibrated coordinates of the positioning modules in the vehicle coordinate system.

[0052] For example, the positioning module can be a UWB-based positioning module. Therefore, multiple positioning modules can be configured on the vehicle. For instance, a positioning module can be installed on the A-pillar and B-pillar of the vehicle. Each positioning module can be pre-calibrated in the spatial position of the vehicle body coordinate system. For example, if a positioning module is installed on the A-pillar and B-pillar of the vehicle, its spatial position in the vehicle body coordinate system can be calibrated as the third coordinate (x1, y1, z1) of positioning module a, the third coordinate (x2, y2, z2) of positioning module b, the third coordinate (x3, y3, z3) of positioning module c, and the third coordinate (x4, y4, z4) of positioning module d.

[0053] The coordinates of a terminal can be determined by using the third coordinate of positioning module a and the distance from positioning module a to the terminal. Similarly, the coordinates of a terminal can be determined by using the third coordinates of positioning modules b, c, and d and the distances from positioning modules b, c, and d to the terminal. For example, the coordinates of a terminal can be determined in reverse using the Euclidean distance method.

[0054] In this embodiment of the disclosure, the first coordinate of the terminal in the vehicle coordinate system is determined by the following formula:

[0055] In this system, the terminal's first coordinate in the vehicle body coordinate system is represented as (x, y, z), and d1 to d4 are the distances from positioning modules a, b, c, and d to the terminal, respectively. By solving the simultaneous equations and considering the height of the positioning modules on the vehicle, the terminal's first coordinate in the vehicle body coordinate system can be determined. This first coordinate can be used to represent the terminal's position within the vehicle.

[0056] Optionally, in step S113, determining the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device includes: Based on the first coordinates and the attitude information, determine the linear function of the terminal relative to the vehicle body; Among them, the linear function is used to describe the linear motion or direction of the terminal relative to the vehicle body.

[0057] In this embodiment, the position and attitude of the terminal in three-dimensional space are converted into a straight line relative to the vehicle body. A first coordinate system is used to determine the position of a reference point (e.g., the center point) on the terminal in the terminal coordinate system. Then, based on attitude information (pitch angle, yaw angle, roll angle), the reference point is transformed into the vehicle body coordinate system. Finally, through geometric analysis (e.g., vector analysis), the direction and slope of this straight line are determined, thereby obtaining a linear function.

[0058] For example: Assume the terminal is located in the back seat of the vehicle, with first coordinates (x0, y0, z0) (relative to the terminal itself). Attitude information indicates that the terminal has a pitch angle of 10 degrees and a yaw angle of 5 degrees relative to the vehicle body. Through coordinate transformation, (x0, y0, z0) can be converted to coordinates (x, y, z) in the vehicle body coordinate system. Then, using vector analysis, the direction and slope of the straight line from a point on the vehicle body (such as the center point of the dashboard) to (x, y, z) can be determined, resulting in the linear function y = mx + b (where x and y may represent coordinates in the vehicle body plane).

[0059] The pointing information of the terminal is determined based on the second coordinate set and the line function.

[0060] In this embodiment, the direction of the terminal is determined using a linear function and a second set of coordinates of the target device on the vehicle body or in the environment. By substituting the second set of coordinates into the linear function, the specific position or direction when the terminal points to that point can be calculated. If the second set of coordinates includes only one coordinate, the angle formed by the linear function and the coordinate in the second set can be determined. This angle is compared with a preset angle to determine the direction information of the terminal. This preset angle can be determined based on the size of the target device or the display interface corresponding to the target device. If the second set of coordinates includes multiple coordinates, the angle formed by the linear function and multiple coordinates in the second set can be determined. Each of these angles is compared with a preset angle to determine the direction information of the terminal. Only when multiple angles are within the preset angle range can it be determined that the terminal is pointing to the target device.

[0061] Continuing the example above, assume the second coordinate set is the coordinates (Xp, Yp) of the center point of the display screen (relative to the vehicle body coordinate system). Substituting (Xp, Yp) into the linear function y=mx+b, we can calculate a specific position or direction of the terminal in the vehicle body coordinate system if the terminal is pointing to point P. This position or direction can be used to determine the terminal's directional information, such as "pointing to the right of the display screen" or "pointing to the display screen," etc.

[0062] In this way, by combining the first coordinate, attitude information, and second coordinate set, and by using linear functions for geometric calculations, the pointing information of the terminal can be determined simply and conveniently.

[0063] Optionally, in step S112, determining the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal includes: Based on the first rotation information of the vehicle and the second rotation information of the terminal, the rotation matrix of the terminal in the vehicle body coordinate system is obtained; The vehicle's first rotation information is its rotation angle or rotation matrix relative to the world coordinate system in three-dimensional space, used to describe the vehicle's overall orientation. The terminal's second rotation information is its rotation angle or rotation matrix relative to the world coordinate system, used to describe the terminal's orientation or posture. The rotation matrix describes the terminal's rotation in three-dimensional space. It can be a 3x3 orthogonal matrix with a determinant of 1, and it satisfies the multiplication rules for rotation matrices.

[0064] In this embodiment, the rotation information of the vehicle and the rotation information of the terminal are combined to obtain the overall rotation matrix of the terminal in the vehicle coordinate system. This can be calculated using coordinate transformation and matrix multiplication. First, the rotation matrices (or rotation angles, then converted into rotation matrices) of the vehicle and the terminal are obtained separately. Then, through matrix multiplication, the rotation matrix of the terminal is applied to the vehicle coordinate system to obtain the rotation matrix of the terminal in the vehicle coordinate system.

[0065] For example, suppose the vehicle has a 30-degree rotation about the Z-axis (yaw angle) relative to the ground coordinate system, while the terminal has a 15-degree rotation about the X-axis (pitch angle) relative to the vehicle body coordinate system. First, rotation matrices can be created for each of these rotations. Then, by matrix multiplication, the terminal's rotation matrix is ​​multiplied by the vehicle's rotation matrix (or vice versa, depending on the rotation order and the definition of the coordinate system), thus obtaining the overall rotation matrix of the terminal in the vehicle body coordinate system.

[0066] The attitude information of the terminal in the vehicle body coordinate system is determined based on the rotation matrix.

[0067] In this embodiment, the terminal's attitude information, such as pitch, yaw, and roll angles (also known as Euler angles), can be extracted from the rotation matrix. Continuing the example above, the overall rotation matrix of the terminal in the vehicle coordinate system is obtained. Then, this rotation matrix can be converted into Euler angles using mathematical formulas to obtain the pitch, yaw, and roll angles of the terminal in the vehicle coordinate system. These angles can be used to describe the specific attitude of the terminal relative to the vehicle body, such as "the terminal tilts upward by 15 degrees and tilts to the right by 30 degrees," etc.

[0068] Optionally, the method includes: If, based on the pointing information, it is determined that the terminal is pointing to the target device of the vehicle, a prompting instruction is sent to the terminal, which is used by the terminal to prompt the display interface corresponding to the target device to output a pointing prompt.

[0069] The prompt command is a command sent by the vehicle to the terminal to display prompts or feedback on the terminal's user interface, such as displaying a cursor, vibration, sound, etc.

[0070] In this embodiment of the disclosure, after the vehicle determines that the terminal is pointing at the target device on the vehicle, the vehicle can send a prompt command to the terminal via wireless communication (such as Bluetooth, Wi-Fi, etc.) or wired connection (such as USB, HDMI, etc.). This command tells the terminal to display a cursor or other visual feedback on the user interface.

[0071] For example, if the vehicle determines that the terminal is pointing at the target device within the vehicle, it may send a prompt command to the terminal via Bluetooth, instructing the terminal to display a cursor on the screen. This command may include information such as the cursor type (e.g., arrow, dot, etc.), color, size, and initial position. This is to inform the user that control commands can be sent to the vehicle via the terminal, or to inform the terminal that a cursor has been displayed.

[0072] Optionally, in step S11, determining the terminal's pointing information includes: The system receives a display instruction sent by the terminal and determines the pointing information of the terminal based on the display instruction.

[0073] The display command is a command sent from the terminal to the vehicle to indicate the position of the cursor on the display screen.

[0074] In this embodiment of the disclosure, after receiving a display command, the vehicle responds to the display command and sends a command to the display screen. This command contains all the information required to display the cursor on the display screen, such as the cursor's type, color, size, and position.

[0075] For example, after receiving a display command, the vehicle responds to the display command and sends a command to the display screen. This command includes the precise position information of the cursor (such as X=100, Y=200) and its appearance (such as a red arrow).

[0076] In this embodiment of the present disclosure, when the vehicle receives a display command sent by the terminal, it can parse the display command to obtain the information carried in the display command. This information can be used to indicate which target device on the vehicle the terminal is pointing to, and the vehicle can then provide pointing prompts on the display interface of that target device. In this way, it is easier for the user to perceive that they have pointed to the display interface of one of the target devices.

[0077] Optionally, the pointing prompt includes: a cursor pointing prompt, wherein when it is determined that the terminal is pointing to a target device on the vehicle based on the pointing information, the pointing prompt is output on the display interface corresponding to the target device, including: When the terminal is pointed at the target device of the vehicle, a cursor is displayed on the display interface corresponding to the target device.

[0078] In this embodiment of the disclosure, a cursor can be displayed on the display interface corresponding to the target device to provide pointing prompts. Furthermore, as the terminal moves and rotates, the position of the cursor is dynamically updated, ensuring that the cursor always follows the movement of the terminal until the terminal no longer points at the target device. Thus, the cursor can be removed from the display interface of the target device.

[0079] Optionally, when it is determined that the terminal is pointing to a target device on the vehicle based on the pointing information, outputting a cursor pointing prompt on the display interface corresponding to the target device includes: If it is determined that the terminal is pointing to the target device according to the display instruction sent by the terminal, the display position information of the cursor pointing prompt on the display interface is determined; The display location information refers to the position where the cursor appears on the display screen, usually represented by coordinates (such as X, Y). This step is determined by the terminal and then sent to the vehicle.

[0080] Based on the display position information, the cursor pointing prompt is output on the display interface corresponding to the target device.

[0081] In this embodiment of the disclosure, after receiving a display command, the vehicle parses the command and displays the cursor on the screen according to the information in the command (such as the cursor type, color, size, and position). This typically involves technologies such as graphics rendering and display control.

[0082] For example, if the display receives an instruction containing the display position (X=100, Y=200) and appearance (red arrow), it will pass this information to the graphics rendering engine and draw a red arrow cursor at the corresponding position on the screen.

[0083] In summary, by comparing the terminal's pointing information with the target device's position information, it is determined whether the terminal is pointing at the display screen. Upon confirmation, a series of instructions are used to display a cursor on the display screen. This allows the user to see the cursor's position, facilitating the movement or operation of the terminal and enabling vehicle control.

[0084] Optionally, the directional prompt includes at least one of the following: a flashing display prompt, a background change prompt, a border prompt, and a pop-up prompt.

[0085] In this embodiment of the disclosure, the flashing display prompt refers to indicating that the terminal is pointing to a target device by flashing the display screen. One or more flashes remind the user which target device's display screen the terminal is currently pointing to. The background change prompt refers to indicating that the terminal is pointing to a target device by changing the background of the display screen. This is used to change the user's visual focus, thereby guiding the user to know which target device's display screen the terminal is currently pointing to. After the terminal is no longer pointing to the target device, the corresponding display screen of the target device returns to its original background. The border prompt refers to indicating a prompt by changing the border of the display screen. The pop-up prompt refers to opening an automatically pop-up window to display the pointing prompt. The pop-up window can be automatically closed after the pointing prompt is completed.

[0086] Optionally, the target device includes at least one of the following: a central control device, a panoramic head-up device, a combination instrument, and an in-vehicle entertainment device.

[0087] The central control unit, located on the vehicle's center console or near the dashboard, serves as the primary information display and control interface within the vehicle. Its display shows navigation information, vehicle status, entertainment system content, and allows the driver to control these functions via touch or voice. Therefore, vehicle functions such as music playback or volume adjustment can be controlled through the terminal.

[0088] The panoramic head-up device (PHD) is a transparent screen used to display vehicle information on the windshield in front of the driver's line of sight. It can display navigation instructions, vehicle speed, safety warnings, and other information within the driver's field of vision, reducing the risk of driver distraction and improving driving safety. Therefore, vehicle functions such as whether to display navigation can be adjusted via a terminal to achieve vehicle control. In this application, the terminal can be pointed at either the projection machine of the PHD located behind the instrument cluster or at the display interface of the PHD projected onto the windshield.

[0089] The instrument cluster is located on the dashboard in front of the driver. Its display interface shows basic vehicle information such as vehicle speed, engine speed, fuel level, and coolant temperature. Therefore, vehicle functions, such as whether to display instantaneous fuel consumption, can be adjusted via a terminal to achieve vehicle control.

[0090] The in-vehicle entertainment system is a display screen inside the vehicle used to play entertainment content such as music, videos, and games. It can be located in the rear-seat entertainment system and provide touchscreen control. Therefore, vehicle functions such as switching programs can be adjusted via the terminal, thus achieving vehicle control.

[0091] Optionally, the vehicle control includes at least one of the following: volume control, page switching control, power off control, screen scrolling control, mute control, and brightness adjustment control.

[0092] Volume control refers to adjusting the volume of the car audio system by adjusting the buttons on the display screen to change the loudness of audio content such as music and radio. Page switching control refers to switching between different information pages or function menus on the central control display screen or instrument panel display screen. For example, you can switch between the navigation page and the vehicle status page to view different information. Power off control refers to the function of turning off the power of the vehicle display screen, such as turning off the central control display screen and the car audio system.

[0093] Screen scrolling control refers to scrolling the content up and down or left and right on the central control display screen or in-vehicle entertainment display screen. Mute control refers to the function of temporarily turning off the sound of the in-vehicle audio system. Brightness adjustment control refers to adjusting the brightness level of the central control display screen, instrument panel display screen, or in-vehicle entertainment display screen.

[0094] This disclosure also provides a vehicle control method applied to a terminal, such as a mobile phone or tablet computer. See also... Figure 3 As shown, it includes: In step S21, when the terminal is pointed at the target device of the vehicle, an operation is received on the terminal; The operations can be performed on the user interface or generated through physical buttons on the terminal. The user interface is the interface on the terminal used for interaction with the user, including various virtual buttons, sliders, drop-down menus, and other elements. In this step, the user generates control commands through operations on the UI.

[0095] When the control application on the terminal is open, a cursor can be displayed on the display screen when the terminal is detected pointing at the vehicle's display screen.

[0096] In this embodiment of the disclosure, a communication connection (such as Bluetooth, Wi-Fi, or a dedicated vehicle communication protocol) is established between the terminal and the vehicle's display screen. The user can operate the terminal through its user interface (such as touching a specific location on the screen, swiping a finger, etc.), and these operations are recognized by the terminal and converted into corresponding control commands.

[0097] For example, suppose a user is using a smartphone to control a vehicle's display. The user touches the smartphone screen, and the smartphone, recognizing this as a single tap, sends the corresponding control command to the vehicle via Bluetooth. Upon receiving the signal, the vehicle performs a tap on the display using the cursor. For instance, if the cursor is hovering over the virtual "previous track" button, the user can then switch to the next track. In this way, the user can control the vehicle using their smartphone.

[0098] In step S22, a control command corresponding to the operation is sent to the vehicle. The control command is used by the vehicle to execute the function corresponding to the control command when the display interface corresponding to the target device outputs a pointing prompt.

[0099] In this embodiment, the terminal sends control commands generated by the user's operations on the UI to the vehicle. Upon receiving the control commands, the vehicle parses the information within them and executes corresponding vehicle control operations on the target device's display interface based on this information. These operations may include volume adjustment, page switching, power off, etc.

[0100] The aforementioned technical solution addresses scenarios where direct control of the central control screen is difficult, such as when the front seats are reclined or rear passengers are seated. Users can easily control the vehicle's display screen via a terminal without moving or adjusting their posture, significantly improving ease of operation and comfort. Thus, through interaction between the terminal and the vehicle, control of any display screen can be conveniently achieved in various situations where direct operation is difficult, enhancing convenience and user experience.

[0101] Optionally, see Figure 4 As shown, the method further includes: In step S51, the pointing information of the terminal is determined; Among these, directional information refers to the position and orientation of the terminal relative to the display screen. This can be determined through image data captured by sensors on the terminal (such as gyroscopes, accelerometers, magnetometers, etc.) or cameras.

[0102] In this embodiment, the terminal can capture its motion state and surrounding environment information in real time using built-in sensors or a camera. After processing and analysis, the precise pointing information of the terminal relative to the vehicle's display screen can be calculated.

[0103] In step S52, if it is determined that the terminal is pointing to a target device on the vehicle based on the pointing information, a display instruction is generated, which is used by the vehicle to determine the pointing information of the terminal.

[0104] In this embodiment of the present disclosure, when the vehicle receives a display command sent by the terminal, it can parse the display command to obtain the information carried in the display command. This information can be used to indicate which target device on the vehicle the terminal is pointing to, and the vehicle can then provide pointing prompts on the display interface of that target device. In this way, it is easier for the user to perceive that they have pointed to the display interface of one of the target devices.

[0105] Optionally, the pointing prompt includes a cursor pointing prompt. In step S52, when it is determined that the terminal is pointing to the target device on the vehicle based on the pointing information, generating a display instruction includes: If the target device of the vehicle is determined to be pointed to by the terminal based on the pointing information, the display position information of the cursor pointing prompt on the display interface is determined; In this embodiment of the disclosure, after the terminal obtains the pointing information, it can combine information such as the position and size of the vehicle's display screen and the relative positional relationship between the terminal and the display screen to determine the specific position of the cursor on the display interface of the target device.

[0106] Continuing the example above, suppose the vehicle's display screen is a rectangular area of ​​fixed size, and the relative position between the smartphone and the display screen is known (e.g., positioning via wireless communication technologies such as Bluetooth or Wi-Fi). When the smartphone is pointed at the display screen, the exact position on the display screen (e.g., center, top left corner, etc.) is calculated based on the smartphone's sensor data and the display screen's position and size information.

[0107] In step S53, the display instruction is generated based on the display location information. The display instruction is used by the vehicle to output the cursor pointing prompt on the display interface corresponding to the target device based on the display location information when the vehicle receives the display instruction.

[0108] In this embodiment of the disclosure, the display command not only instructs the display screen to display the cursor, but also includes the specific position information where the cursor should be displayed on the display screen. Then, the display command is sent to the vehicle, and upon receiving the command, the vehicle moves the cursor to the designated position on the display screen according to the position information in the command.

[0109] Continuing the example above, suppose the terminal has calculated that the smartphone is pointing to the center of the screen. Then, a display command containing this location information is generated and sent to the vehicle. Upon receiving the command, the vehicle moves the cursor to the center of the screen. This way, the user can see the cursor and move it on the screen as the smartphone moves.

[0110] Optionally, determining the pointing information of the terminal includes: Determine the first coordinates of the terminal in the vehicle body coordinate system; In this embodiment of the disclosure, the precise position of the terminal relative to the vehicle body can be obtained by a sensor (such as LiDAR, camera combined with image recognition, or directly calculated by the installation position) and converted into coordinate values ​​in the vehicle body coordinate system.

[0111] Based on the first rotation information of the vehicle and the second rotation information of the terminal, the attitude information of the terminal in the vehicle coordinate system is determined, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; In this embodiment, inertial sensors such as a compass, level, gyroscope, and accelerometer, combined with GPS data (for correcting long-term drift), are used to calculate the rotation information of the vehicle and the terminal. By combining the two, the final attitude of the terminal relative to the vehicle body coordinate system can be calculated.

[0112] The pointing information of the terminal is determined based on the first coordinates, the attitude information, and the second coordinate set of the target device.

[0113] In this embodiment, the attitude information of the terminal is transformed from the vehicle coordinate system to the world coordinate system. Then, the terminal's position in the world coordinate system (obtained through the first coordinate transformation) and the second coordinate set of the target device are combined to calculate the absolute position and orientation of the terminal in the world coordinate system through geometric transformation.

[0114] In this embodiment of the disclosure, either the vehicle or the terminal can determine the terminal's pointing information. This step is performed by the terminal to determine the terminal's pointing information. For detailed embodiments, please refer to the execution steps on the vehicle side, which will not be repeated here.

[0115] Optionally, determining the first coordinates of the terminal in the vehicle coordinate system includes: Determine the distances between the multiple positioning modules on the vehicle and the terminal; The distance between a terminal and a positioning module can be measured using various methods, such as Time of Flight (TOF) technology, which utilizes the time it takes for a signal to travel between two terminals and a positioning module. Alternatively, the distance can be determined by measuring the time difference in signal propagation from the terminal to different positioning modules, for example, using Time Difference of Arrival (TDOA). Finally, the distance can be determined by Time of Arrival (TOA), which uses the time it takes for a signal to travel from the terminal to the positioning module.

[0116] Based on the third coordinates of each positioning module and the corresponding distance, the first coordinate of the terminal in the vehicle coordinate system is determined, where the third coordinates are the pre-calibrated coordinates of the positioning modules in the vehicle coordinate system.

[0117] In this embodiment of the disclosure, either the vehicle or the terminal can determine the first coordinate of the terminal in the vehicle coordinate system. This step is the step performed by the terminal to determine the first coordinate of the terminal in the vehicle coordinate system. For detailed embodiments, please refer to the execution steps on the vehicle side, which will not be repeated here.

[0118] Optionally, determining the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device includes: Based on the first coordinates and the attitude information, the linear function of the terminal relative to the vehicle body is determined; In this embodiment, the position and attitude of the terminal in three-dimensional space are converted into a straight line relative to the vehicle body. A first coordinate system is used to determine the position of a reference point (e.g., the center point) on the terminal in the terminal coordinate system. Then, based on attitude information (pitch angle, yaw angle, roll angle), the reference point is transformed into the vehicle body coordinate system. Finally, through geometric analysis (e.g., vector analysis), the direction and slope of this straight line are determined, thereby obtaining a linear function.

[0119] The pointing information of the terminal is determined based on the second coordinate set and the line function.

[0120] In this embodiment of the disclosure, the direction of the terminal is determined using a linear function and a second set of coordinates of the target device on the vehicle body or in the environment. By substituting the second set of coordinates into the linear function, the specific position or direction when the terminal points to that point can be calculated.

[0121] Optionally, determining the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal includes: Based on the first rotation information of the vehicle and the second rotation information of the terminal, the rotation matrix of the terminal in the vehicle body coordinate system is obtained; In this embodiment, the rotation information of the vehicle and the rotation information of the terminal are combined to obtain the overall rotation matrix of the terminal in the vehicle coordinate system. This can be calculated using coordinate transformation and matrix multiplication. First, the rotation matrices (or rotation angles, then converted into rotation matrices) of the vehicle and the terminal are obtained separately. Then, through matrix multiplication, the rotation matrix of the terminal is applied to the vehicle coordinate system to obtain the rotation matrix of the terminal in the vehicle coordinate system.

[0122] The attitude information of the terminal in the vehicle body coordinate system is determined based on the rotation matrix.

[0123] In this embodiment, the terminal's attitude information, such as pitch, yaw, and roll angles (also known as Euler angles), can be extracted from the rotation matrix. Continuing the example above, the overall rotation matrix of the terminal in the vehicle coordinate system is obtained. Then, this rotation matrix can be converted into Euler angles using mathematical formulas to obtain the pitch, yaw, and roll angles of the terminal in the vehicle coordinate system. These angles can be used to describe the specific attitude of the terminal relative to the vehicle body, such as "the terminal tilts upward by 15 degrees and tilts to the right by 30 degrees," etc.

[0124] Optionally, the method further includes: The system receives a prompt instruction sent by the vehicle, which is sent to the terminal by the vehicle after determining, based on the pointing information, that the terminal is pointing to the target device of the vehicle. In this embodiment of the disclosure, after the vehicle determines that the terminal is pointing at the display screen, the vehicle can send a prompt command to the terminal via wireless communication (such as Bluetooth, Wi-Fi, etc.) or wired connection (such as USB, HDMI, etc.). This command tells the terminal to display a cursor or other visual feedback on the user interface.

[0125] According to the prompting instruction, a directional prompt is output on the display interface corresponding to the target device.

[0126] In this embodiment of the present disclosure, after receiving a prompt instruction, the terminal can provide a directional prompt on the display interface corresponding to the target device through, for example, changes in sound and light or displaying text content on the user interface.

[0127] Optionally, the operation includes at least one of the following: clicking, swiping, or continuous touch.

[0128] Clicking can include single or double clicks, or it can be a single or double click on the left or right mouse button after the user has defined a left or right mouse button. Swiping can be done by the user sliding their finger across the screen, either airborne or by touch, causing the cursor to move across the display. Sustained touch can be a long press, which can then be used to control the corresponding vehicle on the display.

[0129] See Figure 5 As shown, a detailed embodiment of the vehicle control method of this disclosure is illustrated. This embodiment uses a UWB positioning module as an example. When the terminal is configured with UWB and enters the vehicle's UWB recognition range, the terminal positioning module can obtain the distance between the vehicle's positioning module and the terminal's positioning module. This distance can be obtained using a gyroscope or magnetic sensor, etc., to acquire the Euler angles of the vehicle and the terminal in the world coordinate system. The display screen on the vehicle can include a central control display screen and a PHUD display screen. The terminal positioning module can be configured on the vehicle or on the terminal, and the operation module is configured on the terminal.

[0130] Furthermore, based on the calibration coordinates of the vehicle's positioning module in the vehicle's coordinate system and the distance between the vehicle's positioning module and the terminal's positioning module, the terminal's position relative to the vehicle's coordinate system can be determined, and this position can be represented by coordinates. Simultaneously, based on the Euler angles of the vehicle and terminal in the world coordinate system, the Euler angle transformation angle of the terminal relative to the vehicle's coordinate system can be determined.

[0131] Then, based on the terminal's position relative to the vehicle body coordinate system and the Euler angle transformation angle, the terminal's pose is determined, where pose refers to the terminal's position and orientation. This allows us to determine the position of the terminal's central axis, and by combining this with the position of the display screen, we can determine whether the terminal is pointing towards the display screen.

[0132] When the terminal is pointed at the display screen, a cursor, such as a mouse cursor, can be displayed on the screen. The position of the terminal on the display screen is the cursor's position, and the cursor can move on the display screen as the terminal moves. Furthermore, when the terminal receives user actions, such as single-clicking or double-clicking, these actions can be executed using the cursor on the display screen.

[0133] This disclosure also provides a vehicle control device, see [link to relevant documentation] Figure 6 As shown, it includes: The determination module 110 is configured to determine the terminal's pointing information; Display module 120 is configured to output a pointing prompt on the display interface corresponding to the target device when the terminal is determined to be pointing to the target device of the vehicle based on the pointing information. The execution module 130 is configured to execute a function corresponding to a control instruction sent by the terminal, the control instruction being generated based on the operation on the terminal.

[0134] Optionally, the determining module 110 is configured to: determine the first coordinates of the terminal in the vehicle coordinate system; determine the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; and determine the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device.

[0135] Optionally, the determining module 110 is configured to: determine the distance between the multiple positioning modules on the vehicle and the terminal; and determine the first coordinate of the terminal in the vehicle coordinate system based on the third coordinate of each positioning module and the corresponding distance, wherein the third coordinate is the pre-calibrated coordinate of the positioning module in the vehicle coordinate system.

[0136] Optionally, the determining module 110 is configured to: determine the linear function of the terminal relative to the vehicle body based on the first coordinates and the attitude information; and determine the pointing information of the terminal based on the second coordinate set and the linear function.

[0137] Optionally, the determining module 110 is configured to: obtain the rotation matrix of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal; and determine the attitude information of the terminal in the vehicle coordinate system based on the rotation matrix.

[0138] Optionally, the determining module 110 is configured to: when it is determined that the terminal is pointing to the target device of the vehicle based on the pointing information, send a prompting instruction to the terminal, wherein the prompting instruction is used by the terminal to prompt the display interface corresponding to the target device to output a pointing prompt.

[0139] Optionally, the determining module 110 is configured to: receive a display instruction sent by the terminal, and determine the pointing information of the terminal according to the display instruction.

[0140] Optionally, the pointing prompt includes a cursor pointing prompt, wherein the display module 110 is configured to output a cursor pointing prompt on the display interface corresponding to the target device when the terminal is determined to be pointing to the target device of the vehicle based on the pointing information.

[0141] Optionally, the display module 110 is configured to determine the display position information of the cursor pointing prompt on the display interface when the terminal is determined to be pointing to the target device according to the display instruction sent by the terminal; and to output the cursor pointing prompt on the display interface corresponding to the target device according to the display position information.

[0142] Optionally, the target device includes at least one of the following: a central control device, a panoramic head-up device, a combination instrument, and an in-vehicle entertainment device.

[0143] This disclosure also provides a vehicle control device, see [link to relevant documentation] Figure 7 As shown, it includes: The receiving module 210 is configured to receive operations on the terminal when the terminal is pointed at a target device in the vehicle. The sending module 220 is configured to send a control command corresponding to the operation to the vehicle. The control command is used by the vehicle to execute the function corresponding to the control command when the display interface corresponding to the target device outputs a pointing prompt.

[0144] Optionally, the receiving module 210 is configured to: determine the pointing information of the terminal; and, if it is determined from the pointing information that the terminal is pointing to a target device on the vehicle, generate a display instruction, the display instruction being used by the vehicle to obtain the pointing information of the terminal.

[0145] Optionally, the pointing prompt includes a cursor pointing prompt. The receiving module 210 is configured to determine the display position information of the cursor pointing prompt on the display interface when the terminal is determined to be pointing to the target device of the vehicle based on the pointing information; and to generate the display instruction based on the display position information. The display instruction is used by the vehicle to output the cursor pointing prompt on the display interface corresponding to the target device based on the display position information when the display instruction is received.

[0146] Optionally, the receiving module 210 is configured to: determine the first coordinates of the terminal in the vehicle coordinate system; determine the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; and determine the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device.

[0147] Optionally, the receiving module 210 is configured to: determine the distance between the multiple positioning modules on the vehicle and the terminal; and determine the first coordinate of the terminal in the vehicle coordinate system based on the third coordinate of each positioning module and the corresponding distance, wherein the third coordinate is the pre-calibrated coordinate of the positioning module in the vehicle coordinate system.

[0148] Optionally, the receiving module 210 is configured to: determine the linear function of the terminal relative to the vehicle body based on the first coordinates and the attitude information; and determine the pointing information of the terminal based on the second coordinate set and the linear function.

[0149] Optionally, the receiving module 210 is configured to: obtain the rotation matrix of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal; and determine the attitude information of the terminal in the vehicle coordinate system based on the rotation matrix.

[0150] Optionally, the receiving module 210 is configured to: receive a prompt instruction sent by the vehicle, the prompt instruction being sent by the vehicle to the terminal when the vehicle determines, based on the pointing information, that the terminal is pointing to the target device of the vehicle; and, based on the prompt instruction, output a pointing prompt on the display interface corresponding to the target device.

[0151] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0152] This disclosure also provides a vehicle, including: Processor; memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement any of the vehicle-side methods described in the foregoing embodiments.

[0153] This disclosure also provides a terminal, including: Processor; memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement any of the terminal-side methods described in the foregoing embodiments.

[0154] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the vehicle-side methods described in the foregoing embodiments, or, when executed by a processor, implements the steps of any of the terminal-side methods described in the foregoing embodiments.

[0155] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described on the vehicle side in the foregoing embodiments, or, when executed by a processor, implements the steps of any of the methods described on the terminal side in the foregoing embodiments.

[0156] Figure 8 This is a block diagram illustrating a vehicle 600 according to an exemplary embodiment. For example, vehicle 600 can be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicle. Vehicle 600 can be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0157] Reference Figure 8 The vehicle 600 may include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. The vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of the vehicle 600 can be interconnected via wired or wireless means.

[0158] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, and a navigation system, etc.

[0159] The perception system 620 may include several sensors for sensing information about the environment surrounding the vehicle 600. For example, the perception system 620 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0160] The decision control system 630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0161] The drive system 640 may include components that provide powered motion to the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0162] Some or all of the functions of vehicle 600 are controlled by computing platform 650. Computing platform 650 may include at least one processor 651 and memory 652, processor 651 can execute instructions 653 stored in memory 652.

[0163] Processor 651 can be any conventional processor, such as a commercially available CPU. Processors may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems-on-chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0164] The memory 652 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0165] In addition to instruction 653, memory 652 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 652 can be used by computing platform 650. In this embodiment of the present disclosure, processor 651 can execute instruction 653 to complete all or part of the steps of the above-described vehicle-side vehicle control method.

[0166] Figure 9 This is a block diagram illustrating a device 800 for vehicle control according to an exemplary embodiment. For example, device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0167] Reference Figure 9 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output interface 812, a sensor component 814, and a communication component 816.

[0168] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0169] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0170] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 800.

[0171] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0172] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0173] Input / output interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0174] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0175] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0176] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the vehicle control method on the terminal side described above.

[0177] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to complete the aforementioned vehicle control method on the terminal side. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0178] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the aforementioned vehicle control method on the terminal side when executed by the programmable device.

[0179] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0180] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A vehicle control method, characterized in that, include: Determine the terminal's pointing information; If the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, a pointing prompt is output on the display interface corresponding to the target device. According to the control command sent by the terminal, execute the function corresponding to the control command, which is generated based on the operation on the terminal; The information indicating the location of the terminal includes: The first coordinates of the terminal in the vehicle coordinate system are determined; the attitude information of the terminal in the vehicle coordinate system is determined based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; the pointing information of the terminal is determined based on the first coordinates, the attitude information and the second coordinate set of the target device. Determining the first coordinate of the terminal in the vehicle body coordinate system includes: The distance between the multiple positioning modules on the vehicle and the terminal is determined; based on the third coordinates of each positioning module and the corresponding distance, the first coordinate of the terminal in the vehicle coordinate system is determined, wherein the third coordinates are the pre-calibrated coordinates of the positioning modules in the vehicle coordinate system; Determining the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device includes: Based on the first coordinates and the attitude information, the linear function of the terminal relative to the vehicle body is determined; based on the second coordinate set and the linear function, the pointing information of the terminal is determined. The step of determining the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal includes: Based on the first rotation information of the vehicle and the second rotation information of the terminal, the rotation matrix of the terminal in the vehicle body coordinate system is obtained; based on the rotation matrix, the attitude information of the terminal in the vehicle body coordinate system is determined. The pointing prompt includes: a cursor pointing prompt, wherein when it is determined that the terminal is pointing to a target device on the vehicle based on the pointing information, the pointing prompt is output on the display interface corresponding to the target device, including: If the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, a cursor pointing prompt is output on the display interface corresponding to the target device.

2. The method according to claim 1, characterized in that, The method includes: If the terminal is determined to be pointing to the target device of the vehicle based on the pointing information, a prompting instruction is sent to the terminal, which prompts the terminal to output a pointing prompt on the display interface corresponding to the target device.

3. The method according to claim 1, characterized in that, The information indicating the location of the terminal includes: The system receives a display instruction sent by the terminal and determines the pointing information of the terminal based on the display instruction.

4. The method according to claim 1, characterized in that, When the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, the step of outputting a cursor pointing prompt on the display interface corresponding to the target device includes: If it is determined that the terminal is pointing to the target device according to the display instruction sent by the terminal, the display position information of the cursor pointing prompt on the display interface is determined; Based on the display position information, the cursor pointing prompt is output on the display interface corresponding to the target device.

5. The method according to any one of claims 1-4, characterized in that, The target device includes at least one of the following: a central control unit, a panoramic head-up device, a combination instrument panel, and an in-vehicle entertainment system.

6. A vehicle control method, characterized in that, include: Determine the terminal's pointing information; If the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, a display instruction is generated, which is used by the vehicle to obtain the pointing information from the terminal. When the terminal is pointed at the target device of the vehicle, it receives the operation on the terminal. The control command corresponding to the operation is sent to the vehicle. The control command is used by the vehicle to execute the function corresponding to the control command when the display interface corresponding to the target device outputs a pointing prompt. The information indicating the location of the terminal includes: The first coordinates of the terminal in the vehicle coordinate system are determined; the attitude information of the terminal in the vehicle coordinate system is determined based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; the pointing information of the terminal is determined based on the first coordinates, the attitude information and the second coordinate set of the target device. Determining the first coordinate of the terminal in the vehicle body coordinate system includes: The distance between the multiple positioning modules on the vehicle and the terminal is determined; based on the third coordinates of each positioning module and the corresponding distance, the first coordinate of the terminal in the vehicle coordinate system is determined, wherein the third coordinates are the pre-calibrated coordinates of the positioning modules in the vehicle coordinate system; Determining the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device includes: Based on the first coordinates and the attitude information, the linear function of the terminal relative to the vehicle body is determined; based on the second coordinate set and the linear function, the pointing information of the terminal is determined. The step of determining the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal includes: Based on the first rotation information of the vehicle and the second rotation information of the terminal, the rotation matrix of the terminal in the vehicle body coordinate system is obtained; based on the rotation matrix, the attitude information of the terminal in the vehicle body coordinate system is determined. The pointing prompt includes a cursor pointing prompt. When the pointing information determines that the terminal is pointing at a target device on the vehicle, generating a display instruction includes: When the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, the display position information of the cursor pointing prompt on the display interface is determined; based on the display position information, the display instruction is generated, and the display instruction is used by the vehicle to output the cursor pointing prompt on the display interface corresponding to the target device based on the display position information when the display instruction is received.

7. The method according to claim 6, characterized in that, The method further includes: The system receives a prompt instruction sent by the vehicle, which is sent to the terminal by the vehicle after determining, based on the pointing information, that the terminal is pointing to the target device of the vehicle. According to the prompting instruction, a directional prompt is output on the display interface corresponding to the target device.

8. A vehicle control device, characterized in that, include: The determination module is configured to determine the terminal's pointing information; The display module is configured to output a pointing prompt on the display interface corresponding to the target device when the terminal is determined to be pointing to the target device of the vehicle based on the pointing information. An execution module is configured to perform a function corresponding to a control instruction sent by the terminal, the control instruction being generated based on an operation on the terminal. The information indicating the location of the terminal includes: The first coordinates of the terminal in the vehicle coordinate system are determined; the attitude information of the terminal in the vehicle coordinate system is determined based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; the pointing information of the terminal is determined based on the first coordinates, the attitude information and the second coordinate set of the target device. Determining the first coordinate of the terminal in the vehicle body coordinate system includes: The distance between the multiple positioning modules on the vehicle and the terminal is determined; based on the third coordinates of each positioning module and the corresponding distance, the first coordinate of the terminal in the vehicle coordinate system is determined, wherein the third coordinates are the pre-calibrated coordinates of the positioning modules in the vehicle coordinate system; Determining the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device includes: Based on the first coordinates and the attitude information, the linear function of the terminal relative to the vehicle body is determined; based on the second coordinate set and the linear function, the pointing information of the terminal is determined. The step of determining the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal includes: Based on the first rotation information of the vehicle and the second rotation information of the terminal, the rotation matrix of the terminal in the vehicle body coordinate system is obtained; based on the rotation matrix, the attitude information of the terminal in the vehicle body coordinate system is determined. The pointing prompt includes: a cursor pointing prompt, wherein when it is determined that the terminal is pointing to a target device on the vehicle based on the pointing information, the pointing prompt is output on the display interface corresponding to the target device, including: If the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, a cursor pointing prompt is output on the display interface corresponding to the target device.

9. A vehicle control device, characterized in that, include: The determination module is configured to determine the terminal's pointing information; If the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, a display instruction is generated, which is used by the vehicle to obtain the pointing information from the terminal. The receiving module is configured to receive operations on the terminal when the terminal is pointed at a target device in the vehicle. The sending module is configured to send a control command corresponding to the operation to the vehicle. The control command is used by the vehicle to execute the function corresponding to the control command when the display interface corresponding to the target device outputs a pointing prompt. The information indicating the location of the terminal includes: The first coordinates of the terminal in the vehicle coordinate system are determined; the attitude information of the terminal in the vehicle coordinate system is determined based on the first rotation information of the vehicle and the second rotation information of the terminal, wherein the first rotation information and the second rotation information are rotation information relative to the world coordinate system; the pointing information of the terminal is determined based on the first coordinates, the attitude information and the second coordinate set of the target device. Determining the first coordinate of the terminal in the vehicle body coordinate system includes: The distance between the multiple positioning modules on the vehicle and the terminal is determined; based on the third coordinates of each positioning module and the corresponding distance, the first coordinate of the terminal in the vehicle coordinate system is determined, wherein the third coordinates are the pre-calibrated coordinates of the positioning modules in the vehicle coordinate system; Determining the pointing information of the terminal based on the first coordinates, the attitude information, and the second coordinate set of the target device includes: Based on the first coordinates and the attitude information, the linear function of the terminal relative to the vehicle body is determined; based on the second coordinate set and the linear function, the pointing information of the terminal is determined. The step of determining the attitude information of the terminal in the vehicle coordinate system based on the first rotation information of the vehicle and the second rotation information of the terminal includes: Based on the first rotation information of the vehicle and the second rotation information of the terminal, the rotation matrix of the terminal in the vehicle body coordinate system is obtained; based on the rotation matrix, the attitude information of the terminal in the vehicle body coordinate system is determined. The pointing prompt includes a cursor pointing prompt. When the pointing information determines that the terminal is pointing at a target device on the vehicle, generating a display instruction includes: When the terminal is determined to be pointing to a target device on the vehicle based on the pointing information, the display position information of the cursor pointing prompt on the display interface is determined; based on the display position information, the display instruction is generated, and the display instruction is used by the vehicle to output the cursor pointing prompt on the display interface corresponding to the target device based on the display position information when the display instruction is received.

10. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement the method of any one of claims 1-5.

11. A terminal, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement the method of any one of claims 6-7.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5, or when executed by a processor, the computer program implements the steps of the method according to any one of claims 6-7.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-5, or, when executed by a processor, implements the steps of the method according to any one of claims 6-7.

Images