Interaction method and related device

Abstract

An interaction method and a related device, applicable to the fields of interactive technology, intelligent vehicle technology, etc. The state of a control object can be adjusted by means of motion information of a hand of a user, such as a gesture movement, and an interactive action is visually used for expressing a user instruction, such that the user can comfortably and freely adjust the state of the control object according to the user's own needs, thereby reducing the number of physical buttons in a vehicle and reducing costs. By projecting a first interface and a second interface by means of a projection device, the user can visually perceive the states of the control object before and after adjustment, and interaction can be jointly completed by means of gestures together with projection, thereby implementing timely feedback, improving interaction efficiency, and enhancing user experience. Moreover, interface display is implemented by means of projection, so that even a rear-seat passenger can conveniently and quickly adjust the state of a control object, and visually perceive the state of the control object after adjustment, thereby improving user experience, reducing the number of display screens, and reducing costs.

Description

An interactive method and related apparatus Technical Field

[0001] This application relates to the fields of interactive technology and intelligent vehicle technology, and in particular to an interactive method and related device. Background Technology

[0002] With the development of intelligent vehicle technology, people's demands for vehicle comfort, safety, convenience and other in-vehicle functions are constantly increasing. More and more intelligent in-vehicle devices are being integrated into vehicles, such as in-vehicle refrigerators, in-vehicle air conditioners, in-vehicle audio systems, cabin ambient lighting, central control screens, projectors, rear-seat LCD screens, and light field screens. These are gradually being widely used in large, medium and small passenger cars, commercial vehicles and engineering vehicles, comprehensively empowering the improvement of cabin performance and the riding experience.

[0003] However, current user interaction designs with in-vehicle devices are primarily based on physical button controls, touchscreen controls, and voice interaction. With physical buttons, it's impossible to provide physical buttons for all functions. With touchscreen controls, there aren't enough screens available, and some functions may be located deep within the interface hierarchy; even with screen controls, they may not be easily found, making the interaction inconvenient. Some vehicles support voice interaction, which eliminates the need for significant physical movements, but the information system experiences delays in processing and controlling voice commands, resulting in a poor user experience.

[0004] Therefore, how to achieve efficient interaction and improve the interactive experience is an urgent problem to be solved. Summary of the Invention

[0005] This application provides an interaction method and related device that can adjust the state of a controlled object through the user's hand movement information, such as gestures, allowing the user to comfortably and freely adjust the state of the controlled object. The interface is projected through a projection device so that the user can intuitively perceive the adjusted state of the controlled object, thereby improving interaction efficiency and enhancing the user experience.

[0006] In a first aspect, this application provides an interaction method, the method comprising: projecting a first interface onto a first surface using a projection device, acquiring first motion information of a user's hand, adjusting the state of a controlled object based on the first motion information, and projecting a second interface onto the first surface using the projection device. The first surface is located in the cabin of a vehicle, the first interface indicates the initial state of the controlled object, and the second interface indicates the adjusted state of the controlled object.

[0007] Optionally, this method can be applied to a vehicle, for example, executed by a computing device or a module (including software and / or hardware modules) within the vehicle. For ease of description, the following explanation uses a processing device as the executing entity. The vehicle also includes a projection device for projecting images (including video).

[0008] Optionally, the user's first hand movement information includes the movement of air gestures or the user's hand sliding or clicking on the first surface.

[0009] In this embodiment, the user can adjust the state of the controlled object through hand movements. The interactive actions intuitively express the user's instructions, allowing the user to comfortably and freely adjust the state of the controlled object according to their own needs. This can reduce the number of physical buttons in the vehicle and lower costs.

[0010] Furthermore, by projecting the interface using a projection device, users can intuitively perceive the initial and adjusted states of the controlled object. Interaction is achieved through a combination of hand movements and projection, enabling timely feedback, which improves interaction efficiency and enhances user experience. Moreover, in this application, the state of the controlled object can be quickly adjusted through hand movements, minimizing the time cost of user operation. For example, it significantly reduces the time spent searching for and manipulating the controlled object on the interface, thus improving interaction efficiency and convenience.

[0011] In addition, since the number of displays in a vehicle is usually limited, this application uses projection to display the interface, so that even rear passengers can easily and quickly adjust the state of the controlled object and intuitively perceive the adjusted state of the controlled object. This can significantly improve the user experience, reduce the number of displays, and lower costs.

[0012] In summary, this application combines hand movement adjustment with projection technology to achieve highly efficient interaction and enhance the user experience.

[0013] In one possible implementation of the first aspect, the method further includes: determining the user's operational intent, wherein the controlled object is the object targeted by the user's operational intent. The controlled object is either an adjustable device of the vehicle or an adjustable function of the vehicle.

[0014] Vehicles typically have numerous adjustable functions and devices, but the button positions for these functions and the locations of these devices are usually fixed, making the user experience less than intuitive when adjusting them. In the above embodiment, the computing device selects the desired operation based on the user's intention and uses hand movements and projection technology to adjust the adjustable functions or devices within the vehicle. This allows the user to freely adjust the desired operation under comfortable conditions and with minimal effort.

[0015] For example, rear passengers can freely control the vehicle's air conditioning temperature using hand gestures and projection technology without having to get up from their seats, improving convenience and enhancing user safety. Of course, the air conditioning temperature setting can be replaced with other adjustable functions.

[0016] In another possible implementation of the first aspect, the first motion information includes the amplitude and direction of the first motion of the user's hand, the adjustment amount of the state of the controlled object is related to the amplitude of the first motion, and the adjustment direction of the state of the controlled object is related to the direction of the first motion.

[0017] In the above embodiments, the user can express the adjustment amount of the state of the controlled object by the range of hand movements, and express the adjustment direction of the state of the controlled object by the direction of hand movements, so that the user can freely adjust the state of the controlled object according to their own needs.

[0018] In another possible implementation of the first aspect, the first movement includes one or more of the following movements: hand translation, hand rotation, degree of hand opening or closing, or degree of finger pinching, etc.

[0019] In another possible implementation of the first aspect, the adjustable device includes at least one of a sunroof, air conditioning, seat, trunk, ambient lighting, or vehicle system.

[0020] In another possible implementation of the first aspect, the adjustable functions include one or more of the following functions: audio and video playback volume, audio and video playback progress, display window scaling, display brightness, air conditioning temperature, air conditioning airflow, seat back height, seat heating level, seat ventilation level, seat movement, seat opening and closing, seat massage level, 360° surround view angle, window height, trunk opening, sunroof opening, ambient light brightness, ambient light color, fragrance concentration, or radio channels, etc.

[0021] In another possible implementation of the first aspect, projecting a first interface onto a first surface using a projection device includes: determining the first surface, determining projection parameters based on the properties of the first surface, and projecting the first interface onto the first surface using the projection device according to the projection parameters. Here, the first surface is the surface of a first carrier object, which is the object carrying the projected image. Further, the first carrier object is located in the cabin of a vehicle, and the first carrier object includes a user's body part or a target object.

[0022] For example, the projection parameters include at least one of the following: the position of the first interface, the size of the first interface, the brightness of the first interface, the contrast of the first interface, the sharpness of the first interface, and the color of the first interface. Optionally, the attributes of the first surface may also include the color, size, position, brightness, etc. of the first surface.

[0023] In the above embodiments, the positions, sizes, colors, etc. of different first carrier objects may be different. The computing device can determine the projection parameters according to the properties of the first surface, so that the display effect of the projected first interface is better and the user experience is enhanced.

[0024] In yet another possible implementation of the first aspect, determining the first surface includes: determining the first surface based on the user's operational intent.

[0025] In the above embodiments, the computing device determines the first surface based on the user's operating intention, so that the position and size of the projection match the user's operating intention, thereby improving the display effect of the projected interface and enhancing the user experience.

[0026] In another possible implementation of the first aspect, the position of the first surface is related to the position of the controlled object.

[0027] In the above embodiments, the first surface of the projection can be the surface of the device to which the controlled object belongs. When the user adjusts the controlled object, they can also intuitively see the state of the projected controlled object before and after adjustment on the surface of the device to which the controlled object belongs, which makes the display effect of the projected interface better and enhances the user experience.

[0028] In another possible implementation of the first aspect, the first surface is located at the position touched by the user's hand.

[0029] In the above embodiments, the first surface of the projection can be the location touched by the user's hand. The user can conveniently and quickly adjust the state of the controlled object by clicking or swiping the interface projected on the first surface. Moreover, the user can intuitively see the state of the controlled object before and after adjustment, resulting in a better display effect of the projected interface and an enhanced user experience.

[0030] In another possible implementation of the first aspect, the position of the first interface remains unchanged relative to the position of the first surface.

[0031] In the above embodiments, the position of the interface remains unchanged relative to the position of the first surface. For example, if the first surface is the surface of the user's hand, if the user's hand moves, the computing device can make the projected first interface move with the movement of the user's hand, so that the display effect of the projected interface is better and the user experience is enhanced.

[0032] In yet another possible implementation of the first aspect, the method further includes: determining a second surface based on the user's line of sight, and projecting a second interface onto the second surface using a projection device.

[0033] Similarly, the second surface is the surface of the second carrier object, which is the object that carries the projected image. Further, the second carrier object is located in the vehicle's cabin and includes a user's body part or a target object. Optionally, the second carrier object and the first carrier object can be different objects; for example, the first carrier object is the user's hand, and the second carrier object is a table in the vehicle's cabin. Even more optionally, the second carrier object and the first carrier object can be different locations of the same object; for example, the first carrier object is the left side of the inner surface of the vehicle door, and the second carrier object is the right side of the inner surface of the vehicle door.

[0034] In the above embodiments, the user's line of sight may change. At this time, the position of the projected interface can be adjusted according to the change of the user's line of sight, so that the projected interface changes with the change of the user's line of sight, thereby improving the display effect of the projected interface and enhancing the user experience.

[0035] In another possible implementation of the first aspect, the method further includes: adjusting the position of the second interface according to the relative relationship between the user's head posture and the position of the first surface.

[0036] The first surface may be tilted, vertical, or horizontal, and the user's head posture may be looking up or down. For example, when the first surface is vertical and the user's head posture is looking up, the position of the second interface needs to be adjusted upwards; when the first surface is vertical and the user's head posture is looking down, the position of the second interface needs to be adjusted downwards. In this way, the computing device adjusts the position of the second interface according to the relative relationship between the user's head posture and the position of the first surface, resulting in a better display effect of the projected interface and an enhanced user experience.

[0037] In another possible implementation of the first aspect, the method further includes: acquiring second motion information of the user's hand, and adjusting the second interface based on the second motion information.

[0038] In another possible implementation of the first aspect, the second motion information includes the amplitude and direction of the second motion of the user's hand, the position adjustment amount of the second interface is related to the amplitude of the second motion, and the adjustment direction of the position of the second interface is related to the direction of the second motion.

[0039] For example, the second movement involves long-pressing and dragging the second interface. When the user long-presses and drags the second interface, its position moves accordingly. When the user stops dragging and releases the button, the second interface stops moving. In this way, the computing device can intuitively adjust the position of the second interface based on the user's second movement, resulting in a better display effect of the projected interface and an enhanced user experience.

[0040] In another possible implementation of the first aspect, the second motion information includes the amplitude and direction of the third motion of the user's hand, the amount of size adjustment of the second interface is related to the amplitude of the third motion, and the direction of size adjustment of the second interface is related to the direction of the third motion.

[0041] For example, the third movement is pinching between fingers. When the user zooms in or out of the second interface by pinching, the size of the second interface changes accordingly. When the user stops pinching, the size of the second interface stops changing. In this way, the computing device can intuitively adjust the size of the second interface based on the user's third movement, resulting in a better display effect of the projected interface and an enhanced user experience.

[0042] In another possible implementation of the first aspect, the second motion information includes the amplitude and direction of the fourth motion of the user's hand, the angle adjustment amount of the second interface is related to the amplitude of the fourth motion, and the adjustment direction of the angle of the second interface is related to the direction of the fourth motion.

[0043] For example, the fourth motion is the rotation of the hand in space. When the user rotates their hand, the angle of the second interface can change accordingly; when the user stops rotating their hand, the angle of the second interface stops changing. In this way, the computing device can intuitively adjust the angle of the second interface based on the user's fourth motion, resulting in a better display effect of the projected interface and an enhanced user experience.

[0044] In yet another possible implementation of the first aspect, the method further includes: acquiring brightness, and adjusting at least one of brightness, contrast, and sharpness of the second interface based on the brightness.

[0045] For example, when the ambient brightness or the brightness of the first surface in the vehicle is high, the brightness, contrast, and / or sharpness of the projected second interface can be adjusted accordingly to ensure that the user cannot see the projected second interface clearly due to it being too dark or blurry. When the ambient brightness is low, the brightness, contrast, and / or sharpness of the projected second interface can be adjusted accordingly to prevent the user from experiencing excessive glare when viewing the projected second interface, and also to save energy.

[0046] In another possible implementation of the first aspect, projecting a first interface onto a first surface via a projection device includes: projecting the first interface onto the first surface via the projection device when a first condition is met. The first condition includes at least one of the following: a user's hand touches the first surface; a user's voice input instructs the user to activate a gesture control function; or a user's wake-up gesture is detected.

[0047] In another possible implementation of the first aspect, the method further includes: when a second condition is met, exiting the gesture adjustment function and outputting a stop command to the projection device. The stop command instructs the projection device to stop projecting the second interface, and the second condition includes at least one of the following: the duration for which the user's hand leaves the second interface exceeds a duration threshold; the user's voice input indicates exiting the gesture adjustment function; and the user's exit gesture is detected.

[0048] When the first condition is met, the computing device projects the first interface; when the second condition is met, the projection stops. This eliminates the need to keep the projection device constantly on, thus saving energy. Furthermore, since most of the first and second conditions involve user interaction, the interaction between the user and the projection device is enhanced, better meeting the user's actual needs and improving the user experience.

[0049] In another possible implementation of the first aspect, when the user is in a first posture, the first surface is the surface of the user's first hand, the first posture includes a lying posture, and the first motion information of the user's hand is the motion information of the user's second hand.

[0050] When a user is lying down, it is inconvenient to get up to adjust the state of the controlled object. The computing device projects the interface onto the surface of the user's first hand, and the user can adjust the state of the controlled object through the user's second hand. This allows the user to freely adjust the controlled object under comfortable conditions and at a small cost, without having to get up from the seat, which improves the convenience of adjustment and helps to ensure the user's personal safety.

[0051] In another possible implementation of the first aspect, the vehicle includes an interactive device, and the method further includes: outputting a first prompt via the interactive device. The first prompt is used to indicate the adjusted state of the controlled object, and the output method of the first prompt includes at least one of interface prompts, light prompts, voice announcements, and vibration prompts.

[0052] Computing devices can also output prompts through interactive devices, allowing the prompts to be perceived at the user's subconscious level, such as in peripheral vision, panoramic vision, or subconscious awareness. This can reduce the effort users spend obtaining feedback and reduce their cognitive load.

[0053] Secondly, this application provides a processing apparatus connected to a projection device. The processing apparatus includes: a processing unit for projecting a first interface onto a first surface located in the cabin of a vehicle via the projection device, the first surface indicating an initial state of a controlled object; an acquisition unit for acquiring first motion information of a user's hand; and the processing unit further for adjusting the state of the controlled object based on the first motion information and projecting a second interface onto the first surface via the projection device, the second interface indicating the adjusted state of the controlled object.

[0054] In one possible implementation of the second aspect, the processing unit is further configured to determine the user's operational intent, the control object being the object targeted by the user's operational intent, the control object belonging to the adjustable device of the vehicle, or the control object belonging to the adjustable function of the vehicle.

[0055] In another possible implementation of the second aspect, the first motion information includes the amplitude and direction of the first motion of the user's hand, the adjustment amount of the state of the controlled object is related to the amplitude of the first motion, and the adjustment direction of the state of the controlled object is related to the direction of the first motion.

[0056] In another possible implementation of the second aspect, the first movement includes one or more of the following movements: hand translation, hand rotation, degree of hand opening or closing, or degree of finger pinching, etc.

[0057] In another possible implementation of the second aspect, the adjustable device includes at least one of a sunroof, air conditioning, seat, trunk, ambient lighting, or vehicle system.

[0058] In another possible implementation of the second aspect, the adjustable functions include one or more of the following functions: audio and video playback volume, audio and video playback progress, display window scaling, display brightness, air conditioning temperature, air conditioning airflow, seat back height, seat heating level, seat ventilation level, seat movement, seat opening and closing, seat massage level, 360° surround view, window height, trunk opening, sunroof opening, ambient light brightness, ambient light color, fragrance concentration, or radio channels, etc.

[0059] In another possible implementation of the second aspect, the processing unit is further configured to: determine a first surface; determine projection parameters based on the properties of the first surface; and project a first interface onto the first surface via a projection device according to the projection parameters. The first surface is the surface of a first carrier object located in the vehicle's cabin, and the first carrier object includes a user's body part or a target object. The projection parameters include at least one of the following: the position of the first interface, the size of the first interface, the brightness of the first interface, the contrast of the first interface, the sharpness of the first interface, and the color of the first interface.

[0060] In another possible implementation of the second aspect, the processing unit is further configured to determine the first surface based on the user's operational intent.

[0061] In another possible implementation of the second aspect, the position of the first surface is related to the position of the controlled object, or the first surface is located at the position touched by the user's hand.

[0062] In another possible implementation of the second aspect, the position of the first interface remains unchanged relative to the position of the first surface.

[0063] In another possible implementation of the second aspect, the processing unit is further configured to: determine a second surface based on the user's line of sight, and project a second interface onto the second surface via a projection device. The second surface is the surface of a second carrier object located in the vehicle's cabin, and the second carrier object includes a part of the user's body or a target object.

[0064] In another possible implementation of the second aspect, the processing unit is further configured to adjust the position of the second interface according to the relative relationship between the user's head posture and the position of the first surface.

[0065] In another possible implementation of the second aspect, the acquisition unit is further configured to acquire second motion information of the user's hand, and the processing unit is further configured to adjust the second interface based on the second motion information.

[0066] In another possible implementation of the second aspect, the second motion information includes the amplitude and direction of the second motion of the user's hand, the position adjustment amount of the second interface is related to the amplitude of the second motion, and the adjustment direction of the position of the second interface is related to the direction of the second motion.

[0067] In another possible implementation of the second aspect, the second motion information includes the amplitude and direction of the third motion of the user's hand, the amount of size adjustment of the second interface is related to the amplitude of the third motion, and the direction of size adjustment of the second interface is related to the direction of the third motion.

[0068] In another possible implementation of the second aspect, the second motion information includes the amplitude and direction of the fourth motion of the user's hand, the angle adjustment amount of the second interface is related to the amplitude of the fourth motion, and the adjustment direction of the angle of the second interface is related to the direction of the fourth motion.

[0069] In another possible implementation of the second aspect, the acquisition unit is further configured to acquire brightness, and the processing unit is further configured to adjust at least one of the brightness, contrast, and sharpness of the second interface according to the brightness.

[0070] In another possible implementation of the second aspect, the processing unit is further configured to project the first interface onto the first surface via the projection device when the first condition is met. The first condition includes at least one of the following: the user's hand touches the first surface; the user's voice input instructs the user to activate the gesture adjustment function; or the user's wake-up gesture is detected.

[0071] In another possible implementation of the second aspect, the processing unit is further configured to exit the gesture adjustment function and output a stop command to the projection device when the second condition is met. The stop command instructs the projection device to stop projecting the second interface, and the second condition includes at least one of the following: the duration for which the user's hand leaves the second interface exceeds a duration threshold; the user's voice input indicates an exit from the gesture adjustment function; or the user's exit gesture is detected.

[0072] In another possible implementation of the second aspect, when the user is in a first posture, the first surface is the surface of the user's first hand, the first posture includes a lying posture, and the first motion information of the user's hand is the motion information of the user's second hand.

[0073] In another possible implementation of the second aspect, the processing device is connected to an interactive device, and the processing unit is further configured to output a first prompt through the interactive device. The first prompt is used to indicate the adjusted state of the controlled object, and the output method of the first prompt includes at least one of interface prompts, light prompts, voice announcements, and vibration prompts.

[0074] Thirdly, this application provides a computing device including a processor and a communication interface. The communication interface is used for outputting and / or outputting data (including instructions), and / or for receiving and / or sending data. When the processor executes program instructions in memory, it implements the method described in the first aspect or any possible embodiment of the first aspect.

[0075] Fourthly, this application provides a computing device, including a processor and a memory. The memory stores a computer program, and the processor invokes the computer program to implement the method described in the first aspect or any possible embodiment of the first aspect.

[0076] Optionally, the computing device described in the third or fourth aspect above includes a chip, or the computing device includes a chip system of multiple chips.

[0077] Alternatively, the computing device may be a computing platform.

[0078] Fifthly, this application provides a vehicle that includes a projection device, the processing device described in the second aspect, or the computing device described in the third aspect, or the computing device described in the fourth aspect. The vehicle is used to implement the method described in the first aspect or any possible embodiment of the first aspect.

[0079] In a sixth aspect, embodiments of this application provide a computer-readable storage medium for storing a computer program, the computer program including instructions for performing the methods described in the first aspect or any possible implementation of the first aspect.

[0080] In a seventh aspect, this application provides a computer program product, including program instructions or executable computer program code, which, when executed by at least one processor, implements the method described in the first aspect or any possible implementation thereof.

[0081] The solutions provided in the second to seventh aspects above are used to implement or cooperate with the methods provided in the first aspect above, and therefore can achieve the same or corresponding beneficial effects as the first aspect, which will not be elaborated here. Attached Figure Description

[0082] The accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0083] Figure 1 is a schematic diagram of the architecture of a vehicle provided in an embodiment of this application;

[0084] Figure 2 is a schematic diagram of a projection scene provided in an embodiment of this application;

[0085] Figure 3 is a schematic diagram of an interactive scenario provided in an embodiment of this application;

[0086] Figure 4 is a flowchart illustrating an interaction method provided in an embodiment of this application;

[0087] Figure 5 is a schematic diagram of a first surface provided in an embodiment of this application;

[0088] Figure 6A is a schematic diagram of a first interface provided in an embodiment of this application;

[0089] Figure 6B is a schematic diagram of a control control provided in an embodiment of this application;

[0090] Figure 7 is a schematic diagram of three hand gesture movements provided in the embodiments of this application;

[0091] Figure 8 is a schematic diagram of another gesture movement provided in an embodiment of this application;

[0092] Figure 9 is a schematic diagram of three more hand gestures provided in the embodiments of this application;

[0093] Figure 10 is a schematic diagram of a first interface and a second interface provided in an embodiment of this application;

[0094] Figure 11 is a schematic diagram of another gesture movement provided in an embodiment of this application;

[0095] Figure 12 is a schematic diagram of adjusting the second interface according to an embodiment of this application;

[0096] Figure 13 is a schematic diagram of another adjustment of the second interface provided in an embodiment of this application;

[0097] Figure 14 is a schematic diagram of another adjustment of the second interface provided in an embodiment of this application;

[0098] Figure 15 is a schematic diagram of another adjustment of the second interface provided in an embodiment of this application;

[0099] Figure 16 is a schematic diagram of another adjustment of the second interface provided in an embodiment of this application;

[0100] Figure 17 is a schematic diagram of another interactive scenario provided in an embodiment of this application;

[0101] Figure 18 is a schematic diagram of a processing device provided in an embodiment of this application;

[0102] Figure 19 is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0103] The following is an exemplary description of the systems and scenarios in which this application may be applied. Please refer to Figure 1, which is a schematic diagram of a vehicle architecture provided in an embodiment of this application. The vehicle 100 includes a computing device 101, a gesture detection device 102, a control object 103, and a projection device 104. The various devices included in the vehicle 100 are described below by way of example:

[0104] The computing device 101 is a device with computing capabilities, which may include hardware modules with computing capabilities and / or software modules with computing capabilities. Examples are given below based on both hardware and software implementations.

[0105] As an example of hardware implementation, computing device 101 may include at least one processor, which is a circuit with processing capabilities. In one implementation, the processor may be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, microcontroller unit (MCU), graphics processing unit (GPU), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits, which may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement a response function. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In some implementations, the computing device 101 includes at least one processor integrated as a system-on-chip (SOC), which is commonly referred to as an SOC by those skilled in the art. The SOC may include at least one processor, and when the SOC includes multiple processors, the types of processors can be different, such as including a CPU and an NPU.

[0106] For example, computing device 101 includes, but is not limited to, mobile data center (MDC), domain controller (DC), electronic control unit (ECU), vehicle integrated / integration unit (VIU), etc. Alternatively, computing device 101 may also include one or more of the following: handheld terminal, wearable terminal, vehicle, or network device. Among these, network device includes, for example, a gateway (GW).

[0107] As an example of software implementation, computing device 101 may include software functional units. As another example of a software functional unit, computing device 101 may include computing instances, such as virtual machines, containers, and the cloud. A virtual machine is a computer system with complete hardware system functionality simulated by software and running in an isolated environment. A container is an isolated environment obtained by packaging applications and their dependencies. The cloud is a software platform using application virtualization technology, enabling one or more software applications to be developed and run in an independent virtualized environment. Optionally, the cloud may be deployed on a public cloud, private cloud, or hybrid cloud. As yet another example of a software functional unit, computing device 101 may include code that can run on a computing instance, wherein the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container.

[0108] In this application, the computing device 101 is capable of processing the user's hand movement information (or gesture information). The user's hand movement information is provided by the gesture detection device 102, or the user's hand movement information is obtained by processing the data collected by the gesture detection device 102.

[0109] The computing device 101 is connected to the gesture detection device 102. The gesture detection device 102 is capable of detecting the user's hand movement information, such as the user's gestures. For example, the computing device 101 can process the data collected by the gesture detection device 102 to obtain the user's hand position, hand movement information, etc. For example, the gesture detection device 102 may include one or more of the following: a visual sensor, a radar sensor, and a wireless communication device.

[0110] Vision sensors include image sensors (often called cameras). Optionally, the camera can be a monocular camera, a binocular camera, or other different types of cameras; this is not limited here. An image sensor may include multiple photosensitive units, which can sense light signals and form an image. Examples of photosensitive units include complementary metal-oxide-semiconductor (CMOS), charge-coupled devices (CCD), and Live MOS. Image sensors are typically classified based on the type of photosensitive units they contain. For example, an image sensor with CMOS photosensitive units is called a CMOS image sensor (CIS). A CIS may include one or more of the following: a color sensor, a monosensor, etc. A color sensor may include a red-green-blue channel sensor (RGB sensor).

[0111] Radar sensors are devices that detect objects by emitting detection signals and receiving the echoes returned from targets in space. Based on the echo processing, radar sensors can obtain relevant information about the targets in the perceived space, such as distance, position, angle, velocity, reflectivity, reflection intensity, image, color, texture, and material, among other things. Depending on the type of detection signal emitted, radar sensors can be categorized into millimeter-wave radar, centimeter-wave radar, ultrasonic radar, and lidar, among others.

[0112] A wireless communication device is a device capable of receiving wireless signals, and optionally also capable of transmitting wireless signals. After being transmitted into the object space, the wireless signal travels through multiple transmission paths to reach the receiving device. Based on the wave characteristics of the received wireless signal, objects in the object space can be sensed. For example, based on the basic model of Fresnel zones, the movement of objects in space can be sensed through the waveform of the received wireless signal, thus enabling the detection of the user's hand movements. The wireless technologies used in the wireless communication device include one or more of the following: SparkLink (or NearLink), wireless local area network (WLAN), Bluetooth, Zigbee, radio frequency identification (RFID), ultra-wideband (UWB) technology, communication technologies based on long term evolution (LTE), 5th generation mobile networks (or 5th generation wireless systems, 5th-Generation, abbreviated as 5G or 5G technology), global system for mobile communications (GSM), general packet radio service (GPRS), or universal mobile telecommunications system (UMTS), etc.

[0113] The computing device 101 is capable of adjusting the controlled object 103. The computing device 101 is connected to the controlled object 103. It should be understood that the controlled object 103 can be an adjustable device, such as at least one of a sunroof, air conditioning, seat, trunk, ambient lighting, or in-vehicle system. The controlled object 103 can also be an adjustable function, which can be a hardware function, such as the sunroof opening degree, air conditioning temperature, air conditioning airflow, seat back height, seat heating level, seat ventilation level, seat movement, seat opening / closing, seat massage level, window height, trunk opening degree, ambient lighting brightness, ambient lighting color, fragrance concentration, or radio channels. Adjustable functions can also be software functions, such as audio / video playback progress, audio / video playback volume, display window scaling, or display brightness. For example, the computing device 101 can adjust the state of the controlled object 103 based on the user's hand movement information. As another example, the computing device 101 can generate a first interface or a second interface and send the first interface or the second interface to the projection device 104, which then displays the first interface and the second interface to the user. The first interface indicates the initial state of the controlled object 103, and the second interface indicates the adjusted state of the controlled object 103.

[0114] Projection device 104 is a device with projection capability, capable of projecting images and / or videos. Exemplarily, projection device 104 can project an interface onto a first surface. The first surface is located in the vehicle's cabin and is the surface that carries an object (for ease of description, it can be referred to as the surface of the first carrying object). Here, the first carrying object refers to an object (including living objects) capable of carrying images or interfaces. Since the first carrying object is located in the vehicle's cabin, the first surface is located in the vehicle's cabin. Exemplarily, the first carrying object can be a user (including user body parts) or in-vehicle equipment (or a target object). For example, the first carrying object includes the user's hand or leg, and the target object can be any object in the vehicle's cabin within the user's line of sight. As shown in Figure 2, projection device 104 can be installed on the vehicle's roof. Projection device 104 can be used to display configuration information of functions in the vehicle, such as displaying the initial state of control object 103 and the adjusted state of control object 103. For example, in Figure 2, the projection device 104 can project the interface onto the surface of the device to which the controlled object 103 belongs, i.e., the surface of the first carrier object, where the first carrier object is the device to which the controlled object 103 belongs, and the first surface is the surface of the device to which the controlled object 103 belongs. This application does not strictly limit the number and placement of the projection devices 104 in the applicable scenario. The placement of the projection devices 104 can be the headrest (or backrest) of the rear seat of a vehicle, the armrest of the seat, on the door, or the top of the cabin, etc.

[0115] Optionally, the vehicle 100 also includes an interaction device 105, which includes one or more of the following: a display device, a lighting device, an audio playback device, and a haptic output device. The display device includes, but is not limited to, a screen (including a light field screen, HUD, etc.), a button panel, or a smart entity (such as a smart robot). The lighting device includes single lamps such as function indicator lights, ambient lights, or general lighting, or a light strip or light array formed by multiple lamps. The audio playback device is such as a speaker or a device with speaker functionality. The haptic output device is used to transmit tactile information to the user, such as from the steering wheel, motor (which may be integrated into other devices), air vents, seat belts, seats, seat armrests, or floor mats.

[0116] The interactive device 105 is used to output prompts to the user. The prompts are used to indicate the adjusted state of the controlled object 103. Optionally, the output method of the prompts may include at least one of visual output, auditory output, and tactile output. Visual output includes at least one of interface prompts, light prompts, voice announcements, and vibration prompts.

[0117] Optionally, the vehicle 100 also includes a voice acquisition device 106, and a computing device 101 is connected to the voice acquisition device 106, capable of processing voice information. The voice information is provided by the voice acquisition device 106, or it is obtained by processing data collected by the voice acquisition device 106. The voice acquisition device 106 is a device capable of sensing sound, such as a microphone. The computing device 101 can determine the user's operational intent based on the voice information collected by the voice acquisition device 106, where the user's operational intent pertains to an adjustable function that the user wants to adjust. For example, if the voice information is "open the window," then the window is an adjustable function that the user wants to adjust.

[0118] Optionally, the computing device 101 can also determine the user's operational intention based on the user's hand information obtained by the gesture detection device 102. For example, if the user's hand points to the car window, then the car window is the adjustable function that the user wants to adjust. Furthermore, different adjustable functions can correspond to different gestures. When the gesture indicated by the user's hand information is the same as the gesture corresponding to a certain adjustable function, that adjustable function is the one the user wants to adjust.

[0119] Optionally, the multiple modules included in vehicle 100 can be set independently, integrated together, or partially designed independently and partially integrated. For example, the control object 103 can be a function implemented by computing device 101. Another example is that gesture detection device 102 is integrated with computing device 101. Yet another example is that interaction device 105, gesture detection device 102, and control object 103 are integrated together. Yet another example is that computing device 101, projection device 104, and interaction device 105 are integrated into one device, while gesture detection device 102 and control object 103 are integrated into another device.

[0120] It should be understood that the vehicle shown in Figure 1 is merely an example. In actual applications, components in the vehicle can be added or removed as needed. For example, vehicle 100 may also include devices supporting vehicle movement and control, such as a driving system and a braking system, which are not all described here. Figure 1 should not be construed as a limitation on the embodiments of this application. The vehicle 100 described above may be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, or handcart, etc. The embodiments of this application do not impose any special limitations.

[0121] In this embodiment, the projection device 104 can project a first interface onto a first surface. The first interface is used to indicate the initial state of the controlled object 103. The gesture detection device 102 can acquire the user's hand movement information and send it to the computing device 101. The computing device 101 can adjust the state of the controlled object 103 according to the user's hand movement information. The projection device 104 can also project a second interface onto the first surface. The second interface is used to indicate the adjusted state of the controlled object 103. Thus, the user can adjust the state of adjustable functions through hand movements. The interactive actions intuitively express the user's commands, allowing the user to comfortably and freely adjust the state of adjustable functions according to their own needs. This reduces the number of physical buttons in the vehicle and lowers costs.

[0122] Furthermore, by projecting the interface using a projection device, users can intuitively perceive the initial and adjusted states of the controlled object. Interaction is achieved through a combination of hand movements and projection, enabling timely feedback, which improves interaction efficiency and enhances user experience. Moreover, in this application, the state of the controlled object can be quickly adjusted through hand movements, minimizing the time cost of user operation. For example, it significantly reduces the time spent searching for and manipulating the controlled object on the interface, thus improving interaction efficiency and convenience.

[0123] In addition, since the number of displays in a vehicle is usually limited, this application uses projection to display the interface, so that even rear passengers can easily and quickly adjust the state of the controlled object and intuitively perceive the adjusted state of the controlled object. This can significantly improve the user experience, reduce the number of displays, and lower costs.

[0124] In summary, this application combines hand movement adjustment with projection technology to achieve highly efficient interaction and enhance the user experience.

[0125] The vehicle architecture has been introduced above. The following section introduces the human-vehicle interaction scenario as an application scenario example of this application.

[0126] In one implementation example, referring to Figure 3, when a user (such as a driver or passenger) needs to interact with the vehicle 100, the gesture detection device 102 can be used to acquire the movement of the user's air gestures or the user's hand movements such as sliding or clicking on the surface of the first carrier object, i.e., the movement information of the user's hand, thereby adjusting the state of adjustable devices in the vehicle or adjusting the state of adjustable functions in the vehicle. The projection device 104 can project the interface (e.g., a first interface and a second interface) onto the surface of the first carrier object. The first interface, the second interface, and the first carrier object are described in the foregoing related descriptions. In the above implementation example, the user is located inside the cabin of the vehicle 200.

[0127] The methods of the embodiments of this application will be described below.

[0128] Please refer to Figure 4, which is a flowchart illustrating an interaction method provided in an embodiment of this application. Optionally, this method is applied to a vehicle, such as the vehicle 100 shown in Figure 1, and is executed by the vehicle 100 or the computing device 101 within the vehicle 100, for example. For ease of description, the following description uses a computing device as the executing entity.

[0129] The interaction method shown in Figure 4 may include one or more steps from S401 to S404. It should be understood that, for ease of description, the steps are described in the order of S401 to S404, but this embodiment does not limit the execution order, execution time, or number of executions of the above one or more steps. Steps S401 to S404 are as follows:

[0130] Step S401: The computing device projects the first interface onto the first surface through the projection device.

[0131] A computing device is a device with computing capabilities, including processors (such as chips or integrated circuits), chip systems (such as SOCs), devices integrating processors, smart terminals integrating multiple devices, network devices, etc. Examples of computing devices integrating processors include mobile data centers (MDCs), domain controllers (DCs), electronic control units (ECUs), and vehicle integrated / integration units (VIUs). Examples of smart terminals integrating multiple devices include handheld terminals, wearable terminals, and vehicles. Examples of network devices include gateways (GWs). For example, the computing device can be computing device 101 in the vehicle 100 shown in Figure 1.

[0132] A projection device is a device with projection capability, capable of projecting images and / or videos. For example, the projection device may be the projection device 104 in the vehicle 100 shown in FIG. 1 or FIG. 3, or the projection device 104 shown in FIG. 2.

[0133] The first surface is located within the vehicle's cabin and is the surface that carries an object (for ease of description, it can be referred to as the surface of the first carrying object). Here, the first carrying object refers to an object (including living objects) capable of carrying images or interfaces. Since the first carrying object is located within the vehicle's cabin, the first surface is also located within the vehicle's cabin. Exemplarily, the first carrying object can be a user (including user body parts) or an in-vehicle device (or target object). For example, the first carrying object includes the user's hand or leg, and the target object can be any object within the user's line of sight in the vehicle's cabin. For example, in Figure 2, the first carrying object is the device to which the control object 103 belongs, and the first surface is the surface of the device to which the control object 103 belongs.

[0134] Optionally, the first surface can be predefined. For example, the projection device projects an image to a designated location, such as the surface of a first carrier object, such as a seat back, a car table, or a screen.

[0135] Alternatively, the first surface may be calculated by a computing device based on the input information. Several possible methods for determining the first surface are described below:

[0136] Method 1: The computing device can determine the first surface based on the user's operational intent. Several possible designs for determining the first surface based on user operational intent are described below:

[0137] Design 1: The computing device can determine the user's operational intent based on the voice information collected by the voice acquisition device, and determine the first surface based on the user's operational intent. The voice acquisition device is a device capable of sensing sound, such as the voice acquisition device 106 shown in Figure 1. The object targeted by the user's operational intent is the adjustable function the user wants to adjust, also known as the control object. The control object belongs to the vehicle's adjustable equipment, or it belongs to the vehicle's adjustable function, such as the control object 103 shown in Figure 1. For example, if the voice information is "open the car window," then the car window is the adjustable function the user wants to adjust; that is, the car window or the opening and closing of the car window is the control object. In this case, the first surface can be the surface of the car window. As another example, if the voice information is "It's stuffy in the car," the computing device, through semantic analysis, suggests that the user may need ventilation. In this case, the function related to ventilation is the adjustable function the user wants to adjust; that is, functions related to ventilation, such as the fresh air system or sunroof, are the control objects. In this case, the first surface can be the surface of the fresh air system or sunroof, etc.

[0138] Optionally, artificial intelligence (AI) technology can be used in the speech recognition process. For example, models trained using reinforcement learning, deep learning, and neural networks can be used during speech analysis. For instance, a speech recognition model can be configured within the computing device. This model can analyze the semantics of the user's input speech information and identify the user's operational intent. For example, this speech recognition model is trained based on samples, including multiple speech messages and their corresponding operational intents. In short, the computing device can recognize the input speech information based on this speech recognition model, thereby determining the user's operational intent.

[0139] Design 2: The computing device can also determine the user's operational intent based on the user's hand information obtained by the gesture detection device, and determine the first surface based on the user's operational intent. The gesture detection device is a device capable of detecting the movement of the user's hand, such as the gesture detection device 102 shown in Figure 1. For example, if the user's hand points to a car window, then the car window is an adjustable function that the user wants to adjust, i.e., the car window is the control object. In this case, the first surface can be the surface of the car window. Exemplarily, the computing device can identify the three-dimensional spatial position of the user's hand based on an image of the user's hand, and combine the three-dimensional spatial position of the user's hand with the positions of objects inside the vehicle cabin to determine the user's operational intent. For example, if the distance between the user's hand and an object is less than a first distance threshold, the object is determined to be the target of the user's operational intent, i.e., the adjustable device (i.e., the control object) that the user wants to adjust. Optionally, the first distance threshold is a preset value, such as 5cm, 10cm, etc. For example, different adjustable functions can correspond to different gestures. When the user's hand gesture matches the gesture corresponding to a certain adjustable function, such as gesture A corresponding to the adjustable function "vehicle air conditioning temperature," then the user's hand gesture is A, meaning that the adjustable function "vehicle air conditioning temperature" is the function the user wants to adjust, i.e., the air conditioning temperature or air conditioning is the controlled object. In this case, the first surface can be the air vent of the vehicle's air conditioning system.

[0140] The two designs above are merely examples; in actual implementations, other designs may exist to determine the user's operational intent. For instance, the computing device can also determine the user's operational intent based on one or more of the following: the line-of-sight area, head movements, and the relative relationship between the user's position and objects in the vehicle's cabin. These will not be listed here. For example, the computing device can construct three-dimensional spatial coordinates based on the relative relationship between the user's position and objects in the vehicle's cabin, and determine the user's operational intent. For instance, when the distance between the user's position and an object is less than a second distance threshold, the object is determined to be the target of the user's operational intent; that is, the object is an adjustable device (i.e., a controllable object) that the user wants to adjust. Optionally, the second distance threshold is a preset value, such as 5cm, 10cm, etc.

[0141] Method 2: The computing device determines the first surface based on the position of the controlled object. That is, the first surface is also related to the position of the controlled object. The controlled object belongs to an adjustable device of the vehicle, or it belongs to an adjustable function of the vehicle. For example, the first surface may be the surface of the device to which the controlled object belongs, or it may be near the location of the device to which the controlled object belongs. As shown in Figure 5, the first surface may be the surface of the device to which the controlled object belongs. Alternatively, in Figure 5, if the distance between object C and the device to which the controlled object belongs is closer than the distance between object B and the device to which the controlled object belongs, the first surface may be the surface of object C. In this way, when the user adjusts the controlled object, they can intuitively see the projected state of the controlled object before and after adjustment near the location of the device to which the controlled object belongs, resulting in a better display effect of the projected interface and an enhanced user experience.

[0142] Method 3: The computing device determines the first surface based on the position of the user's hand. For example, the first surface is located at the position touched by the user's hand. In Figure 5, the user's hand touches object D, and the first surface could also be located on the surface of object D. The projected first surface can be the position touched by the user's hand, allowing the user to conveniently and quickly adjust the state of the controlled object by clicking or swiping the interface projected on the first surface. Furthermore, the user can intuitively see the state of the projected controlled object before and after adjustment, resulting in a better display effect of the projected interface and an enhanced user experience.

[0143] These various positions can be combined; for example, a computing device can determine the first surface by combining the user's operating intention and the user's hand position.

[0144] In some possible implementations, the computing device can determine projection parameters based on the properties of the first surface, and project a first interface onto the first surface using a projection device according to the projection parameters. For example, the properties of the first surface include at least one of the following: position, size, color, brightness, etc. The projection parameters include at least one of the following: position of the first interface, size of the first interface, brightness of the first interface, contrast of the first interface, sharpness of the first interface, color of the first interface, etc. Different first surfaces may have different positions, sizes, colors, etc., therefore the computing device can determine the projection parameters based on the properties of the first surface, resulting in a better display effect of the projected first interface and enhanced user experience.

[0145] For ease of understanding, Table 1 shows the relationship between some properties of the first surface and some projection parameters.

[0146] As shown in Table 1, the surface area of ​​the projected interface is larger than that of the hand. When the first surface is the hand, the first interface is smaller; when the first surface is the desktop, the first interface is larger. Since the surface of the hand and the desktop are both light-colored, when the first surface is both the hand and the desktop, a darker color can be used for the first interface to ensure clear display. A higher brightness can also be used to prevent the projected interface from being too dark or blurry, thus enhancing the user experience. Conversely, when the first surface is a dark desktop, a lighter color can be used for the first interface to ensure clear display. A lower brightness can be used to prevent color distortion and avoid overly glaring viewing, thus enhancing the user experience.

[0147] In some possible implementations, the computing device may generate a first interface and send projection parameters and the first interface to a projection device so that the projection device projects the first interface onto a first surface. The first interface is used to indicate the initial state of the controlled object.

[0148] Please refer to Figure 6A. The projection device can project and display a first interface, which includes a control 601, which is a slider assembly. The control 601 includes a slider 6011, a sliding track 6012, and a current value 6013 (optional). The slider 6011 is used to indicate a value or a range of values. The sliding track 6012 is used to move the slider along a path based on changes in available values. The current value 6013 is used to further display the value indicated by the current position of the slider 6011. It should be understood that the current value 6013 is optional. Combining Figure 6A and the aforementioned process description of the controlled object, the object targeted by the user's operation intention, i.e., the air conditioning temperature, can be projected and displayed on the first interface through the projection device to indicate the initial state of the controlled object. As shown in Figure 6A, the control 601 can be displayed on the first interface. The control 601 can indicate the initial state of the controlled object, i.e., the current air conditioning operating temperature (e.g., 24.5℃).

[0149] Furthermore, the control 601 can also indicate the initial state of the controlled object within the adjustable range. Referring to Figure 6A, for example, the position of the slider 6011 on the sliding track 6012 can indicate the initial state of the controlled object within the adjustable range. For instance, the adjustable range of an air conditioner temperature is between 18-35°C, and the two ends of the sliding track represent the maximum and minimum values ​​of the adjustable range, respectively. Therefore, the position of the slider of the control 601 can indicate that the current temperature is within the 18-35°C range.

[0150] Of course, the circular ring shown here is only one possible implementation. In another possible design, the control 601 can be replaced with other forms of controls. For example, Figure 6B shows a progress bar-type control 601, including a slider 6011, a sliding track 6012, and a current value 6013 (optional). The control 601 can indicate the initial state of the controlled object, i.e., the current air conditioner operating temperature (e.g., 24.5℃). In addition to the slider control, the initial state of the controlled object can also be displayed through other controls, such as notification bar controls, cards, etc., which will not be illustrated here.

[0151] In one possible implementation, the position of the first interface remains constant relative to the first surface. If the first surface is the surface of a movable object, such as a tabletop or a user's hand, the position of the first interface can move along with the position of the first surface. For example, the computing device can adjust the position of the first interface based on the position movement information of the first surface, so that the position of the first interface relative to the first surface remains constant. Optionally, the position movement information includes the direction and magnitude of the position movement of the first surface. Thus, if the first surface is the surface of a user's hand, and the user's hand moves, the position of the first surface changes, and the computing device can make the first interface move along with the movement of the user's hand, resulting in a better display effect of the projected interface and an enhanced user experience.

[0152] In one possible implementation, when the first condition is met, the computing device executes step S401. The first condition includes at least one of the following: the user's hand touching the first surface, the user's voice input indicating to activate the gesture adjustment function, and the detection of the user's wake-up gesture. The wake-up gesture may be a pre-set specific gesture used to indicate activating the gesture adjustment function. For example, when the user's voice input is "activate gesture adjustment function," the computing device projects the first interface onto the first surface via a projection device.

[0153] Step S402: The computing device acquires the first movement information of the user's hand.

[0154] The user's first hand movement information is provided by a gesture detection device. This gesture detection device is used to sense the user's hand movement and may include a visual sensor, a radar sensor, a wireless communication device, etc. A related description can be found in the description of the vehicle 100 shown in Figure 1. It should be understood that the gesture detection device may be separate from the computing device, or it may be integrated into the computing device. Accordingly, the computing device acquiring the user's first hand movement information includes the computing device receiving the user's first hand movement information provided by the gesture detection device, or the computing device itself collecting the user's first hand movement information.

[0155] The user's first hand movement information includes one or more of the following: an image of the user's gesture, a video of the gesture, or the gesture recognition result. For example, taking a visual sensor as the gesture detection device, the visual sensor can capture multiple images of the user's gestures, such as multiple images stored independently or multiple frames of gesture images stored as video. These multiple gesture images can be provided to the computing device as the user's first hand movement information. Alternatively, the visual sensor can process multiple gesture images to obtain the hand's position, the amplitude of the hand's movement, and the direction of the movement. The first hand movement information provided to the computing device by the visual sensor includes the amplitude and direction of the hand's movement recognized by the visual sensor.

[0156] The first motion information instructs the user's hand to perform a first movement. This first motion information includes the amplitude and direction of the first hand movement. For ease of explanation, the user making the gesture is referred to as the first user, and this first motion information may optionally be associated with the first user's identity information. The first movement includes one or more of the following movements: hand translation, hand rotation, the degree of hand opening or closing, or the degree of finger pinching. For ease of understanding, several possible first movements (also called gesture movements) are listed below:

[0157] The first gesture movement involves the translation of the hand in space. Please refer to Figures 7(a), (b), and (c). The gesture can move along a certain direction in space. The amplitude of the first movement is the distance the hand moves, and the direction of the movement is the direction in which the hand moves.

[0158] The second hand gesture involves the rotation of the hand in space. Please refer to Figure 8. The hand rotates in a certain direction. The amplitude of the first movement is the angle through which it has rotated, and the direction of the movement is the direction of rotation (e.g., clockwise, counterclockwise, etc.).

[0159] The third type of hand gesture involves flexible changes in the hand. Referring to Figures 9(a), (b), and (c), the hand can be considered a flexible object, and the fingers can move flexibly. The movement of a hand gesture can be manifested through the movement of the fingers. As shown in Figure 9(a), two fingers can be pinched together. The amplitude of the first movement includes the change in distance between the two fingers, and the direction of the gesture is either opening or closing. As shown in Figure 9(b), fingers can be twisted. The amplitude of the first movement includes the twisting angle of the gesture, and the direction of the movement is the twisting direction. As shown in Figure 9(c), multiple fingers can be opened or closed. The amplitude of the first movement includes the change in distance between the fingers, and the direction of the gesture is either opening or closing.

[0160] Of course, the above gestures are just examples, and other gestures may exist in the actual implementation.

[0161] Step S403: The computing device adjusts the state of the controlled object according to the first motion information.

[0162] The first motion information includes the amplitude and direction of the user's first hand movement.

[0163] For example, the amount of adjustment to the state of the controlled object is related to the amplitude of the first motion.

[0164] The greater the amplitude of the first movement, the greater the adjustment amount to the controlled object. Referring to Figure 10, taking the air conditioner temperature as an example, the initial state of the air conditioner temperature is 24.5℃. When the gesture moves a distance d1, the state of the air conditioner temperature is updated to 27.5℃, and the adjustment amount of the air conditioner temperature (27.5℃-24.5℃=)3℃ is related to the distance d1 moved.

[0165] Furthermore, the direction of adjusting the state of the controlled object is related to the direction of the first movement. Referring to Figure 10, when the gesture moves to the right, it indicates an increase in temperature; similarly, when the gesture moves to the left, it indicates a decrease in temperature.

[0166] Optionally, the computing device can determine an adjustment command based on the amplitude and direction of the first movement, and send the adjustment command to the controlled object, thereby adjusting the state of the controlled object. The adjustment command includes the amount of adjustment to the state of the controlled object and the direction of adjustment to the state of the controlled object.

[0167] Optionally, the user's first hand movement includes sliding or clicking on the first surface or interface. For example, referring to Figure 6A, the user's hand can click and drag or slide the slider 6011 to adjust the air conditioner temperature. The greater the amplitude of the user's hand clicking and dragging or sliding the slider 6011, the greater the adjustment of the air conditioner temperature. The direction of the user's hand clicking and dragging or sliding the slider 6011 is related to the direction of air conditioner temperature adjustment. For example, dragging or sliding the slider 6011 counterclockwise can lower the air conditioner temperature, while dragging or sliding the slider 6011 clockwise can increase the air conditioner temperature.

[0168] In some possible implementations, the first user's gesture movement can be performed multiple times, triggering multi-stage adjustments to the controlled object through multiple back-and-forth movements. Referring to Figure 11, the first user's gesture movement includes three phases. In the first phase (effective movement segment 1), the first user makes a first gesture and moves it to the right by a distance d1. This gesture triggers an adjustment to the state of the controlled object, but may not achieve the desired state. Therefore, in the second phase (return segment 1), the user retracts the first gesture (e.g., makes a second gesture), no longer maintaining the first gesture. During this phase, the user's gesture movement no longer triggers adjustments to the state of the controlled object. In the third phase (effective movement segment 2), the first user continues to make the first gesture and moves it to the right by a distance d2. This gesture continues to trigger adjustments to the state of the controlled object. By superimposing multiple movements, a larger adjustment range can be achieved.

[0169] Step S404: The computing device projects the second interface onto the first surface via a projection device.

[0170] The second interface is used to indicate the adjusted state of the controlled object. Referring to Figure 10, taking the air conditioner temperature as an example, the initial state of the air conditioner temperature is 24.5℃. When the gesture moves a distance d1, the adjusted air conditioner temperature is updated to 27.5℃. Thus, the adjusted state of the controlled object is related to its initial state and the first movement, providing feedback to the user with minimal latency. Since the gesture expresses the adjustment amount of the controlled object, the second interface projected by the projection device provides feedback on the adjusted state, indicating the adjustment amount and assisting the user in completing the adjustment action more accurately and intuitively through gestures, thereby improving the user experience.

[0171] Similar to step S401 above, the computing device can generate a second interface and send projection parameters and the second interface to the projection device so that the projection device projects the second interface onto the first surface. Further examples are omitted here. For instance, the computing device can receive the adjusted state feedback from the controlled object, generate a second interface based on the adjusted state feedback, and send projection parameters and the second interface to the projection device so that the projection device projects the second interface onto the first surface. This updates the interface projected by the projection device, allowing the user to promptly and intuitively perceive the adjusted state of the controlled object, significantly improving the user experience.

[0172] In one possible implementation, the computing device can determine the second surface based on the user's line of sight and project a second interface onto the second surface using a projection device. Similarly, the second surface, located within the vehicle's cabin, is a surface that carries an object (for ease of description, it can be referred to as the surface carrying the second object). Here, the second object carrying an object refers to an object (including living objects) capable of carrying an image or interface. Since the second object carrying an object is located within the vehicle's cabin, the second surface is also located within the vehicle's cabin. Exemplarily, the second object carrying an object can be a user (including user body parts) or an in-vehicle device (or target object). For example, the second object carrying an object could include the user's hand or leg, and the target object could be any object within the user's line of sight in the vehicle's cabin.

[0173] Optionally, the second supporting object and the first supporting object can be different objects. For example, the first supporting object is the user's hand, and the second supporting object is a table in the vehicle's cabin. Alternatively, the second supporting object and the first supporting object can be different locations of the same object. For example, the first supporting object is the left side of the inner surface of the vehicle door, and the second supporting object is the right side of the inner surface of the vehicle door.

[0174] Referring to Figure 12, when the user's first line of sight is towards the back of the driver's seat, the computing device determines the first surface to be the surface of the driver's seat back and projects a second interface onto this surface using a projection device. When the user's line of sight changes, for example, to a second line of sight facing the right-side door of the vehicle, the computing device determines the second surface to be the inner surface of the right-side door and projects a second interface onto this surface. In this way, the computing device can adjust the position of the projected interface according to changes in the user's line of sight, resulting in a better display effect and enhanced user experience.

[0175] Since the first surface may be tilted, vertical, or horizontal, and the user's head posture may be looking up or down, for example, when the first surface is vertical and the user's head posture is looking up, the position of the second interface needs to be adjusted upwards; when the first surface is vertical and the user's head posture is looking down, the position of the second interface needs to be adjusted downwards.

[0176] As another possible implementation, the computing device can adjust the position of the second interface based on the relative relationship between the user's head posture and the position of the first surface. Referring to Figure 13, the computing device determines the first surface to be the inner surface of the right-side door of the vehicle, and the user's initial head posture is the first posture. Based on the first posture and the inner surface of the right-side door, the computing device determines the position of the second interface as the first position; that is, the computing device projects the second interface at the first position using a projection device. When the user's head posture changes, for example, to the second posture, the computing device determines the position of the second interface as the second position based on the second posture and the inner surface of the right-side door; that is, the computing device needs to adjust the position of the second interface upwards and projects it at the second position using a projection device. In this way, the computing device adjusts the position of the second interface according to the relative relationship between the user's head posture and the position of the first surface, resulting in a better display effect of the projected interface and enhancing the user experience.

[0177] As another possible implementation, the computing device can also acquire second motion information of the user's hand and adjust the second interface based on the second motion information, such as adjusting the position, size, angle, etc. of the second interface.

[0178] Similarly, the computing device can acquire the second motion information of the user's hand according to the implementation method in step S402 described above. For example, the second motion information of the user's hand is provided by a gesture detection device. Acquiring the second motion information of the user's hand by the computing device includes the computing device receiving the second motion information of the user's hand provided by the gesture detection device, or the computing device itself collecting the second motion information of the user's hand. The second motion information of the user's hand includes one or more of the following: an image of the user's gesture, a video of the gesture, or a gesture recognition result.

[0179] The following are some possible examples of adjusting the second interface based on the second motion information:

[0180] Example 1: The second motion information includes the amplitude and direction of the user's hand movement. This second motion is used to adjust the position of the second interface. For example, the adjustment amount of the second interface's position is related to the amplitude of the second motion, and the adjustment direction is related to the direction of the second motion. Referring to Figure 14, the second motion is a translation of the second interface in space while holding down the second interface. The amplitude of the second motion is the distance the hand moves, and the direction of the second motion is the direction the hand moves. For instance, when the user holds down the second interface and drags it to the right, the position of the second interface can move with the user's second motion. When the user stops dragging and releases the second interface, the second interface stops moving. If the distance of the second motion is d3, then the distance the second interface moves is also d3. In this way, the computing device can intuitively adjust the position of the second interface based on the user's second motion, resulting in a better display effect of the projected interface and an enhanced user experience.

[0181] Example 2: The second motion information includes the amplitude and direction of the user's third hand movement, which is used to adjust the size of the second interface. For example, the amount of size adjustment of the second interface is related to the amplitude of the third movement, and the direction of adjustment is related to the direction of the third movement. Referring to Figure 15, the third movement is pinching between the fingers; the amplitude of the third movement is the change in distance between the two fingers, and the direction is either the opening or closing direction of the two fingers. For instance, when the user's third movement changes from a distance of n1 between the two fingers to n2 in the opening direction, the second interface is enlarged, and its size increases. When the user stops enlarging or shrinking the second interface by pinching, the size change stops. In this way, the computing device can intuitively adjust the size of the second interface based on the user's third movement, resulting in a better display effect of the projected interface and an enhanced user experience.

[0182] Example 3: The second motion information includes the amplitude and direction of the user's fourth hand movement, which is used to adjust the angle of the second interface. For example, the adjustment amount of the second interface angle is related to the amplitude of the fourth movement, and the adjustment direction of the second interface angle is related to the direction of the fourth movement. Referring to Figure 16, the fourth movement is the rotation of the hand in space. The hand rotates in a certain direction, the amplitude of the fourth movement is the angle rotated, and the direction of the fourth movement is the direction of rotation (e.g., clockwise, counterclockwise, etc.). For example, when the user's fourth movement rotates clockwise by an angle of m, the second interface also rotates clockwise, and the angle of the second interface can change according to the user's fourth movement. When the user stops rotating their hand, the angle of the second interface stops changing. In this way, the computing device can intuitively adjust the angle of the second interface based on the user's fourth movement, resulting in a better display effect of the projected interface and an enhanced user experience.

[0183] Of course, the above gestures are just examples. In the actual implementation, there may be other gestures used to adjust the second interface.

[0184] In some possible scenarios, users may be sleeping or taking a nap in the vehicle's cabin. In such situations, it may be inconvenient for the user to get up and adjust the controlled objects. In such cases, a second interface or a first interface could be projected onto the surface of the user's first hand, allowing the user to adjust the controlled objects through the movement of their second hand. For example, when the user is in a first posture (including a lying position), the first surface is the surface of the user's first hand, and the first movement information of the user's hand becomes the movement information of the user's second hand. Referring to Figure 17, when the user is lying on the rear seat of the vehicle, the first surface is the surface of the user's left hand (i.e., the user's left hand is the first hand). The computing device can project the interface onto the surface of the user's left hand using the projection device 104. The computing device can obtain the movement information of the user's right hand (i.e., the user's right hand is the second hand) and adjust the state of the controlled objects based on this information. For example, the user can click or drag controls on the interface displayed on the surface of their left hand to adjust the air conditioning temperature, volume, seat tilt, massage intensity, etc. In this way, computing devices enable users to freely adjust the objects they are operating on in a comfortable environment at a low cost, without having to get up from their seats, thus improving the convenience of adjustment and helping to ensure the personal safety of users.

[0185] Optionally, if the computing device detects a user in a reclining position and any occupant making a hand gesture when the nap alarm sounds, the alarm will be delayed and will not sound at that moment. Optionally, the duration of the alarm delay can be a value pre-selected by the user or a pre-configured fixed value. This eliminates the need for the user to get up to turn off the alarm, reducing disruption to the user's sleep and enhancing the user experience.

[0186] In some possible implementations, the ambient brightness in the vehicle cabin may change. The computing device can acquire the brightness and adjust at least one of the brightness, contrast, and sharpness of the second interface based on the brightness. Optionally, the brightness includes at least one of the ambient brightness in the vehicle cabin and the brightness of the first surface. For example, when the ambient brightness in the vehicle cabin or the brightness of the first surface is high, the brightness, contrast, and / or sharpness of the projected second interface can be adjusted to a corresponding degree to increase it, so that the user cannot see the projected second interface clearly because it is too dark or blurry. When the ambient brightness is low, the brightness, contrast, and / or sharpness of the projected second interface can be adjusted to a corresponding degree to decrease it, preventing the user from feeling overly glared at the projected second interface and saving energy.

[0187] Optionally, the computing device can also output prompts to display the adjusted state of the controlled object via other interactive devices, which are referred to as primary prompts. For example, the output methods of primary prompts include at least one of interface prompts, light prompts, voice announcements, vibration prompts, etc. In this way, the computing device enables the prompts to be perceived at the user's subconscious level, such as in peripheral vision, glancing around, or subconsciously, which can reduce the effort users spend to obtain feedback and reduce the user's cognitive load.

[0188] In one possible implementation, when the second condition is met, the computing device exits the gesture adjustment function and outputs a stop command to the projection device. The stop command instructs the projection device to stop projecting the second interface. The second condition includes at least one of the following: the user's hand leaving the second interface for a duration exceeding a duration threshold; the user's voice input instructing the user to exit the gesture adjustment function; or the user's exit gesture being detected. Optionally, the distance the user's hand leaves the second interface may be greater than a third distance threshold, which is a preset value, such as 10cm or 15cm. Thus, when the second condition is met, the computing device stops projection without continuously keeping the projection interface on, saving energy. Furthermore, since most of the second conditions involve user interaction, the interactivity between the user and the projection device is enhanced, better meeting the user's actual needs and improving the user experience.

[0189] In the embodiment shown in Figure 4, the computing device can adjust the state of the controlled object through the user's hand movements. The user's interactive actions intuitively express the user's instructions, allowing the user to comfortably and freely adjust the state of the controlled object according to their own needs. This can reduce the number of physical buttons in the vehicle and lower costs.

[0190] Furthermore, the computing device projects the interface through a projection device, allowing the user to intuitively perceive the initial and adjusted states of the controlled object. Interaction is completed through a combination of hand movements and projection, enabling timely feedback, which improves interaction efficiency and enhances the user experience. Moreover, in this application, the state of the controlled object can be quickly adjusted through hand movements, minimizing the user's time cost for operation. For example, it significantly reduces the time spent searching for and manipulating the controlled object on the interface, thus improving interaction efficiency and convenience.

[0191] In addition, since the number of displays in a vehicle is usually limited, this application uses projection to display the interface, so that even rear passengers can easily and quickly adjust the state of the controlled object and intuitively perceive the adjusted state of the controlled object. This can significantly improve the user experience, reduce the number of displays, and lower costs.

[0192] In summary, this application combines hand movement adjustment with projection technology to achieve highly efficient interaction and enhance the user experience.

[0193] The foregoing has described the application scenarios and methods provided by the embodiments of this application. The apparatus of the embodiments of this application is provided below. It is understood that the various apparatuses provided in the embodiments of this application, such as multimodal interaction devices, computing devices, chips, etc., include hardware structures, software units, or combinations of hardware and software structures to perform the functions described in the above method embodiments. Those skilled in the art should readily recognize that the apparatus and modules within it can be implemented in hardware or a combination of hardware and computer software, in conjunction with the various functions described in the embodiments disclosed herein. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different apparatus implementation methods to implement the aforementioned method embodiments in different usage scenarios, and different implementation methods of the apparatus should not be considered beyond the scope of the embodiments of this application.

[0194] Several possible devices are listed below.

[0195] Please refer to Figure 18, which is a schematic diagram of a processing device provided in an embodiment of this application. The processing device 180 is connected to a projection device and may include a processing unit 1801 and an acquisition unit 1802. The processing device 180 may be an independent device, such as the computing device 101 shown in Figure 1. Alternatively, the processing device 180 may also be a software module and / or hardware module in an independent device, such as a chip in the computing device 101.

[0196] The processing device 180 is used to implement the aforementioned interaction method, such as the interaction method in the embodiment shown in FIG4. The processing unit 1801 is used to implement one or more of the aforementioned data processing, instruction execution, and other related operations, such as determining, adjusting, generating, deciding, and judging; the acquisition unit 1802 is used to implement one or more of the aforementioned operations, such as acquiring and receiving.

[0197] In one possible implementation, processing unit 1801 is used to project a first interface onto a first surface located in the vehicle's cabin via a projection device, the first interface indicating the initial state of the controlled object. Acquisition unit 1802 is used to acquire first motion information of the user's hand. Processing unit 1801 is further used to adjust the state of the controlled object based on the first motion information, and to project a second interface onto the first surface via the projection device, the second interface indicating the adjusted state of the controlled object.

[0198] In another possible implementation, the processing unit 1801 is further configured to determine the user's operating intention, the control object being the object targeted by the user's operating intention, the control object being an adjustable device of the vehicle, or the control object being an adjustable function of the vehicle.

[0199] In another possible implementation, the first motion information includes the amplitude and direction of the first motion of the user's hand, the adjustment amount of the state of the controlled object is related to the amplitude of the first motion, and the adjustment direction of the state of the controlled object is related to the direction of the first motion.

[0200] In another possible implementation, the first movement includes one or more of the following movements: hand translation, hand rotation, degree of hand opening or closing, or degree of finger pinching, etc.

[0201] In yet another possible implementation, the adjustable device includes at least one of a sunroof, air conditioning, seat, trunk, ambient lighting, or in-vehicle system.

[0202] In another possible implementation, the adjustable functions include one or more of the following: audio and video playback volume, audio and video playback progress, display window scaling, display brightness, air conditioning temperature, air conditioning airflow, seat back height, seat heating level, seat ventilation level, seat movement, seat opening and closing, seat massage level, 360° surround view, window height, trunk opening, sunroof opening, ambient light brightness, ambient light color, fragrance concentration, or radio channels, etc.

[0203] In another possible implementation, the processing unit 1801 is further configured to: determine a first surface; determine projection parameters based on the properties of the first surface; and project a first interface onto the first surface using a projection device based on the projection parameters. The first surface is the surface of a first carrier object located in the vehicle's cabin, and the first carrier object includes a user's body part or a target object. The projection parameters include at least one of the following: the position of the first interface, the size of the first interface, the brightness of the first interface, the contrast of the first interface, the sharpness of the first interface, and the color of the first interface.

[0204] In another possible implementation, the processing unit 1801 is further configured to determine the first surface based on the user's operational intent.

[0205] In another possible implementation, the position of the first surface is related to the position of the controlled object, or the first surface is located at the position touched by the user's hand.

[0206] In yet another possible implementation, the position of the first interface remains unchanged relative to the position of the first surface.

[0207] In another possible implementation, the processing unit 1801 is further configured to: determine a second surface based on the user's line of sight, and project a second interface onto the second surface using a projection device. The second surface is the surface of a second carrier object located in the vehicle's cabin, and the second carrier object includes a part of the user's body or a target object.

[0208] In another possible implementation, the processing unit 1801 is further configured to adjust the position of the second interface according to the relative relationship between the user's head posture and the position of the first surface.

[0209] In another possible implementation, the acquisition unit 1802 is further configured to acquire second motion information of the user's hand, and the processing unit 1801 is further configured to adjust the second interface based on the second motion information.

[0210] In another possible implementation, the second motion information includes the amplitude and direction of the second motion of the user's hand, the position adjustment amount of the second interface is related to the amplitude of the second motion, and the adjustment direction of the position of the second interface is related to the direction of the second motion.

[0211] In another possible implementation, the second motion information includes the amplitude and direction of the third motion of the user's hand, the amount of size adjustment of the second interface is related to the amplitude of the third motion, and the direction of size adjustment of the second interface is related to the direction of the third motion.

[0212] In another possible implementation, the second motion information includes the amplitude and direction of the fourth motion of the user's hand, the angle adjustment of the second interface is related to the amplitude of the fourth motion, and the direction of the angle adjustment of the second interface is related to the direction of the fourth motion.

[0213] In another possible implementation, the acquisition unit 1802 is further configured to acquire brightness, and the processing unit is further configured to adjust at least one of the brightness, contrast, and sharpness of the second interface according to the brightness.

[0214] In another possible implementation, the processing unit 1801 is further configured to project a first interface onto the first surface via a projection device when a first condition is met. The first condition includes at least one of the following: a user's hand touches the first surface; a user's voice input instructs the user to activate the gesture adjustment function; or a user's wake-up gesture is detected.

[0215] In another possible implementation, the processing unit 1801 is further configured to exit the gesture adjustment function and output a stop command to the projection device when the second condition is met. The stop command instructs the projection device to stop projecting the second interface, and the second condition includes at least one of the following: the duration for which the user's hand leaves the second interface exceeds a duration threshold; the user's voice input indicates an exit from the gesture adjustment function; or the user's exit gesture is detected.

[0216] In another possible implementation, when the user is in a first posture, the first surface is the surface of the user's first hand, the first posture includes a lying posture, and the first motion information of the user's hand is the motion information of the user's second hand.

[0217] In another possible implementation, the processing device 180 is connected to the interactive device, and the processing unit 1801 is further configured to output a first prompt through the interactive device. The first prompt is used to indicate the adjusted state of the controlled object, and the output method of the first prompt includes at least one of interface prompt, light prompt, voice broadcast, and vibration prompt.

[0218] The specific operations performed by the above-mentioned processing device 180 can also be found in the description of the embodiment shown in Figure 4.

[0219] Please refer to Figure 19, which is a schematic diagram of the structure of a computing device provided in an embodiment of this application. The computing device 190 is a device with processing capabilities. The device here can be a physical device, such as a server (e.g., a rack server), a host, etc., or it can be a virtual device, such as a virtual machine, a container, etc.

[0220] As shown in Figure 19, the computing device 190 includes a processor 1901, a memory 1902, and one or more programs, and may include a communication interface 1903. It should be understood that this application does not limit the number of processors and memories in the computing device 190.

[0221] Processor 1901 is a module that performs calculations and may include a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), a micro controller unit (MCU), or one or more integrated circuits for controlling the execution of programs in the above schemes.

[0222] Memory 1902 provides storage space, in which application data, user data, operating system, and computer programs can be optionally stored. Memory 1902 may include read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0223] The memory 1902 can exist independently and be connected to the processor 1901 via a bus. Alternatively, the memory 1902 can be integrated with the processor 1901.

[0224] The communication interface 1903 is used to provide information input or output to the at least one processor. And / or, the communication interface 1903 can be used to receive data transmitted externally and / or transmit data externally. The communication interface 1903 can be a wired link interface, such as an Ethernet cable, or a wireless link interface (Bluetooth, general wireless transmission, and other wireless communication technologies, etc.). Optionally, the communication interface 1903 may also include a transmitter (such as a radio frequency transmitter, antenna, etc.) or a receiver coupled to the interface.

[0225] In this embodiment, one or more programs are stored in the memory 1902 in the form of program code and configured to be executed by the processor 1901. The programs include instructions for implementing the steps in the interactive method shown in FIG4. That is, the memory 1902 stores executable instructions, and the processor 1901 executes the executable instructions to implement the steps in the interactive method shown in FIG4. In other words, the memory 1902 stores instructions for executing the interactive method shown in FIG4.

[0226] This application embodiment also provides a vehicle, which includes a projection device and the aforementioned processing device 180 or computing device 190. The vehicle is used to implement the aforementioned interaction method, such as the interaction method shown in FIG4.

[0227] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. The computer program instructions are used to implement the aforementioned interaction method, such as the interaction method shown in FIG4.

[0228] This application also provides a computer-readable storage medium. This computer-readable storage medium is used to store a computer program, the computer program including instructions for implementing the aforementioned interactive method, such as the interactive method shown in FIG4.

[0229] The computer-readable storage medium can be any available medium that can be stored by an information interaction device and / or computing device, or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media, or semiconductor media (e.g., solid-state drives).

[0230] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0231] In this application, "at least one" in the embodiments refers to one or more items, and "more than one" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or multiple. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0232] Furthermore, unless otherwise stated, the use of ordinal numbers such as "first" and "second" in the embodiments of this application is for distinguishing multiple objects and is not for limiting the order, sequence, priority or importance of multiple objects.

[0233] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. An interaction method, characterized in that, include: A first interface is projected onto a first surface located in the vehicle's cabin using a projection device. The first interface is used to indicate the initial state of the controlled object. Obtain the user's first hand movement information; Adjust the state of the controlled object according to the first motion information; The projection device projects a second interface onto the first surface, and the second interface is used to indicate the adjusted state of the controlled object.

2. The method according to claim 1, characterized in that, The method further includes: The user's operational intent is determined, and the control object is the object targeted by the user's operational intent. The control object belongs to the adjustable device of the vehicle, or the control object belongs to the adjustable function of the vehicle.

3. The method according to claim 1 or 2, characterized in that, The first motion information includes the amplitude and direction of the first motion of the user's hand. The adjustment amount of the state of the controlled object is related to the amplitude of the first motion, and the adjustment direction of the state of the controlled object is related to the direction of the first motion.

4. The method according to any one of claims 1-3, characterized in that, The projection of the first interface onto the first surface via the projection device includes: A first surface is defined, which is the surface of a first carrier object located in the cabin of the vehicle. The first carrier object includes a body part of the user or a target object. Based on the properties of the first surface, projection parameters are determined, and the projection parameters include at least one of the following: the position of the first interface, the size of the first interface, the brightness of the first interface, the contrast of the first interface, the sharpness of the first interface, and the color of the first interface. The first interface is projected onto the first surface using the projection device according to the projection parameters.

5. The method according to claim 4, characterized in that, Determining the first surface includes: The first surface is determined based on the user's operational intent.

6. The method according to any one of claims 1-5, characterized in that, The position of the first surface is related to the position of the controlled object; Alternatively, the first surface may be located at the position touched by the user's hand.

7. The method according to any one of claims 1-6, characterized in that, The position of the first interface remains unchanged relative to the position of the first surface.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: Based on the user's line of sight, a second surface is determined. The second surface is the surface of a second carrier object located in the vehicle's cabin. The second carrier object includes the user's body parts or a target object. The second interface is projected onto the second surface by the projection device.

9. The method according to any one of claims 1-6, characterized in that, The method further includes: The position of the second interface is adjusted according to the relative relationship between the user's head posture and the position of the first surface.

10. The method according to any one of claims 1-6, characterized in that, The method further includes: Obtain the second motion information of the user's hand; Based on the second motion information, adjust the second interface.

11. The method according to claim 10, characterized in that, The second motion information includes the amplitude and direction of the second motion of the user's hand. The position adjustment amount of the second interface is related to the amplitude of the second motion, and the adjustment direction of the position of the second interface is related to the direction of the second motion.

12. The method according to claim 10 or 11, characterized in that, The second motion information includes the amplitude and direction of the third motion of the user's hand. The size adjustment of the second interface is related to the amplitude of the third motion, and the adjustment direction of the size of the second interface is related to the direction of the third motion.

13. The method according to any one of claims 10-12, characterized in that, The second motion information includes the amplitude and direction of the fourth motion of the user's hand. The angle adjustment amount of the second interface is related to the amplitude of the fourth motion, and the adjustment direction of the angle of the second interface is related to the direction of the fourth motion.

14. The method according to any one of claims 1-13, characterized in that, The method further includes: Obtain brightness; Adjust at least one of the brightness, contrast, and sharpness of the second interface according to the brightness.

15. The method according to any one of claims 1-14, characterized in that, The projection of the first interface onto the first surface via the projection device includes: When the first condition is met, the first interface is projected onto the first surface via the projection device. The first condition includes at least one of the following: the user's hand touches the first surface, the user's voice input instructs the user to activate the gesture adjustment function, and the user's wake-up gesture is detected.

16. The method according to any one of claims 1-15, characterized in that, The method further includes: When the second condition is met, the gesture adjustment function is exited and a stop command is output to the projection device. The stop command is used to instruct the projection device to stop projecting the second interface. The second condition includes at least one of the following: the duration for which the user's hand leaves the second interface is greater than a duration threshold; the user's voice input indicates exiting the gesture adjustment function; and the user's exit gesture is detected.

17. The method according to any one of claims 1-16, characterized in that, When the user is in a first posture, the first surface is the surface of the user's first hand, the first posture includes a lying posture, and the first movement information of the user's hand is the movement information of the user's second hand.

18. A processing apparatus, characterized in that, The processing device is connected to the projection device, and the processing device includes: A processing unit is configured to project a first interface onto a first surface via the projection device. The first surface is located in the cabin of the vehicle, and the first interface is used to indicate the initial state of the controlled object. The acquisition unit is used to acquire the first movement information of the user's hand; The processing unit is also used for: Adjust the state of the controlled object according to the first motion information; The projection device projects a second interface onto the first surface, and the second interface is used to indicate the adjusted state of the controlled object.

19. A computing device, characterized in that, The computing device includes a processor and a memory, the memory storing a program, and the processor executing the program to cause the computing device to perform the method as described in any one of claims 1-17.

20. A vehicle, characterized in that, The vehicle includes a projection device and a processing device as described in claim 18 or a computing device as described in claim 19, the vehicle being used to implement the method as described in any one of claims 1-17.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, the computer program including instructions for performing the method as described in any one of claims 1-17.

22. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor, cause the method as described in any one of claims 1-17 to be implemented.

Images