Screen control method, display device, and vehicle

Abstract

This application provides a screen control method, display device, and vehicle, relating to the field of electronic control technology. The method is applied to a screen controller, where the screen is connected to a screen connection component, and the screen controller is connected to both the screen and the screen connection component. The method includes: acquiring a first instruction; based on the first instruction, sending a second instruction to the screen, the second instruction instructing the screen to play first material; and sending a third instruction to the screen connection component, the third instruction instructing the screen connection component to control the screen to move at least during the playback time of the first material. This solution improves the immersion of the interactive scene and enhances the user experience by controlling the screen movement to adapt to the content being played.

Description

Technical Field

[0001] This application relates to the field of electronic control technology, and in particular to a screen control method, display device, and vehicle. Background Technology

[0002] As the level of automotive intelligence continues to improve, the functions of in-vehicle screens are becoming increasingly diverse, expanding from simple navigation to multiple scenarios such as audio-visual entertainment, vehicle control, and interconnected services.

[0003] However, most mainstream in-vehicle screens are currently fixed in design, unable to be adjusted in angle according to the passenger's position or preference, making it difficult to meet the viewing needs of different users in different riding scenarios. Although some technologies have attempted to achieve limited adjustment of screen orientation, their flexibility and adaptability are still insufficient, making it difficult to match more complex interaction methods and diverse usage scenarios.

[0004] Therefore, a smarter solution is urgently needed to improve the user experience. Summary of the Invention

[0005] This application provides a screen control method, a display device, and a vehicle, which can improve the immersion of interactive scenes and enhance user experience by controlling the movement of the screen to adapt to the content played on the screen.

[0006] To achieve the above objectives, this application adopts the following technical solution: In a first aspect, a screen control method is provided, characterized in that it is applied to a screen controller, wherein the screen is connected to a screen connection component, and the screen controller is respectively connected to the screen and the screen connection component. The method includes: acquiring a first instruction; based on the first instruction, sending a second instruction to the screen, the second instruction being used to instruct the screen to play first material; and sending a third instruction to the screen connection component, the third instruction being used to instruct the screen connection component to control the screen to move at least during the playback time period of the first material.

[0007] Alternatively, the screen controller can be set up separately or integrated into the screen.

[0008] Optionally, the screen connection component may include a screen connector and a screen connector controller. In this case, the screen connector is connected to the screen and the screen connector controller, respectively, and the screen connector controller is connected to the screen controller.

[0009] Optionally, the screen can be the vehicle's central control screen, passenger screen, driver's headrest screen, passenger's headrest screen, screen behind the central control armrest, roof folding screen, etc. This application does not limit the location of the screen in the vehicle.

[0010] Optionally, the first material can be material produced by a manufacturer or a third party, or it can be user-generated material. The type of the first material can be video, music, images, etc., and this application does not limit the type of material.

[0011] Optionally, the motion can be unidirectional or tridirectional (xyz axis).

[0012] In this embodiment, the screen controller acquires a first instruction, sends a second instruction to the screen based on the first instruction to instruct the screen to play first material, and sends a third instruction to the screen connection component to instruct the screen connection component to control the screen to move at least during the playback period of the first material. This movement refers to the screen performing one or more actions such as scaling, panning, tilting left and right, nodding up and down, turning, and shaking. Therefore, by playing the first material while the screen moves, the interactive immersion of the scene can be improved, enhancing the user experience.

[0013] In one possible implementation of the first aspect, the third instruction is used to instruct the screen connection component to control the screen to move at least during the playback time of the first material based on a first motion orchestration signal; wherein the first motion orchestration signal includes one or more motion signals, and different motion signals are used to instruct the screen connection component to control the screen to perform different actions.

[0014] Optionally, the first action arrangement signal may also include information such as the frequency and arrangement order of the action signal.

[0015] In this embodiment of the application, the screen connection component can, under the instruction of the third instruction, control the screen to move at least during the playback time of the first material according to the action signal included in the first action arrangement signal, and execute the arrangement action corresponding to the first material, thereby achieving the effect of playing while the screen moves.

[0016] In one possible implementation of the first aspect, the movement of the screen at least during the playback time of the first material matches a first trajectory corresponding to a target object in the first material, the matching including spatial path consistency and / or time synchronization.

[0017] Optionally, the target object may include one or more objects, which may indicate various types of objects such as people and animals, or interactive behaviors between multiple objects. Further, if the object is a single object, the target object may indicate all or part of that object's content.

[0018] The target objects in the first source material can be extracted and filtered, and the trajectories corresponding to the target objects can also be designed and modified.

[0019] In this embodiment, by controlling the screen movement to match the first trajectory of the target object in the first material, the screen movement and the material content can be adapted, so that the screen movement synchronously follows the changes in the displayed content, thereby enhancing the immersive experience.

[0020] In one possible implementation of the first aspect, after obtaining the first instruction, the method further includes: determining whether a triggering condition is met; if so, sending the second instruction to the screen based on the first instruction, and sending the third instruction to the screen connection component.

[0021] In this embodiment, it can be determined first whether the triggering condition is met. If the triggering condition is met, then the second and third instructions are sent. Therefore, by determining the triggering condition, it is possible to avoid affecting the user's normal driving or other operations.

[0022] In one possible implementation of the first aspect, the triggering conditions include one or more of the following: keyword recognition, first power-on of the day, the current time point being within the effective usage period of the first material, and the target switch included in the target application being turned on; wherein, when the target switch is turned on, it is used to indicate the activation of the target interaction mode, the target interaction mode includes the screen playing the target material, and the screen connection component controlling the screen to move at least during the playback time period of the target material, the first material being included in the target material.

[0023] Keywords can be preset and have a one-to-one correspondence with the materials that the screen controller instructs the screen to play.

[0024] The "first power-on" instruction indicates the first power-on of the screen controller each day, or the first power-on of the vehicle that includes the screen controller each day.

[0025] The effective usage period of the first material can be preset. The current time point can indicate the time when the screen controller receives the first instruction.

[0026] The target application can be the settings application, and the target switch can be a virtual switch within the settings application.

[0027] In one possible implementation of the first aspect, when the screen controller is included in a vehicle, the triggering condition further includes: the vehicle is currently in a non-driving state.

[0028] When the vehicle is not in motion, its gear is P or it is in park.

[0029] In this embodiment of the application, the vehicle being in a non-driving state is used as the trigger condition. When the screen is playing and moving at the same time, it can avoid affecting the user's normal driving or normal performance of other operations.

[0030] In one possible implementation of the first aspect, the method further includes: sending a fourth instruction to the screen, the fourth instruction being used to instruct the screen to play second material; and sending a fifth instruction to the screen connection component, the fifth instruction being used to instruct the screen connection component to control the screen to move to a preset position.

[0031] Optionally, the second material can be material produced by a manufacturer or a third party, or it can be user-generated material. The type of the second material can be video, music, images, etc., and this application does not limit the type of material.

[0032] It should be understood that the second material is also included in the target material.

[0033] In this embodiment, the screen can play the second material and move to a preset position before or after playing the first material and moving, so as to serve as preview content before playing the first material and moving or ending content after playing the second material, thereby enhancing the user experience.

[0034] In one possible implementation of the first aspect, when the screen controller is included in a vehicle, the method further includes: Obtain the seating status, which is any one of the following: only the driver is seated, only the passenger is seated, or both the driver and the passenger are seated. Based on the seated state, the preset position corresponding to the screen is determined.

[0035] It should be understood that different seating positions can be set to correspond to different preset positions on the screen.

[0036] In this embodiment of the application, different preset positions are set for the screen for different seating states. This allows passengers in different seating positions to view the screen content from the best angle, or to facilitate passengers to perform subsequent operations after the current content has finished playing.

[0037] In one possible implementation of the first aspect, the fifth instruction is used to instruct the screen connection component to control the screen to move to the preset position during the playback time of the second material.

[0038] The screen connection component can control the screen to move during the playback period of the second material based on the fifth instruction, and move it to a preset position. Thus, the effect of playing the video while the screen moves to the preset position can be achieved.

[0039] In one possible implementation of the first aspect, after sending a third instruction to the screen connection component, the method further includes: obtaining a sixth instruction; based on the sixth instruction, sending a seventh instruction to the screen, and sending an eighth instruction to the screen connection component; wherein the seventh instruction is used to instruct the screen to pause playback, and the eighth instruction is used to instruct the screen connection component to control the screen to pause movement; or, the seventh instruction is used to instruct the screen to end playback, and the eighth instruction is used to instruct the screen connection component to control the screen to end movement; or, the seventh instruction is used to instruct the screen to switch playback content, and the eighth instruction is used to instruct the screen connection component to control the screen to switch movement.

[0040] In this embodiment, the screen controller obtains a sixth instruction and sends a seventh instruction to the screen based on the sixth instruction to instruct the screen to pause, end, or switch to another interface; and sends an eighth instruction to the screen connection component to instruct the screen connection component to control the screen to pause, end, or switch to another motion. Thus, it is possible to pause, end, or switch to other modes from the mode of playing the first material while moving, thereby meeting other functional needs of the user.

[0041] In one possible implementation of the first aspect, the method further includes: receiving a first data packet, the first data packet including the first material, or the first material and the first motion orchestration signal; generating the first motion orchestration signal based on the first material when the first data packet does not include the first motion orchestration signal; sending the first material to the screen; and sending the first motion orchestration signal to the screen connection component.

[0042] In this embodiment of the application, the first data packet can be sent to the screen controller through software upgrade. The first data packet may include the first material, or may include the first material and the first motion arrangement signal. When the first data packet does not include the first motion arrangement signal, the screen controller can generate the first motion arrangement signal based on the first material. Thus, the actions of the screen connection components can be arranged in combination with the content of the material.

[0043] In one possible implementation of the first aspect, when the first material is a video, generating the motion choreography signal based on the first material includes: determining a first trajectory based on the first material; and generating the first motion choreography signal based on the first trajectory.

[0044] In this embodiment, by generating a first motion choreography signal based on a first trajectory determined from the first material, the screen movement can be controlled to match the first trajectory in the first material, thereby achieving screen movement and material content adaptation, so that the screen movement synchronously follows the changes in the displayed content, thereby enhancing the immersive experience.

[0045] In one possible implementation of the first aspect, determining a first trajectory based on the first material includes: acquiring a keyframe sequence from the first material; acquiring a target object from the keyframe sequence; and determining the first trajectory based on the keyframe sequence and the target object, wherein the first trajectory includes the coordinates of the target object in each keyframe.

[0046] In this embodiment of the application, the calculation of the first trajectory using a keyframe sequence can improve computational efficiency and save resources.

[0047] In one possible implementation of the first aspect, generating the first action choreography signal based on the first trajectory includes: generating a first action sequence based on the first trajectory; and generating the first action choreography signal based on the first action sequence.

[0048] In this embodiment of the application, when the screen connection component controls the screen to perform each action, the timing of each action is consistent with the timing of the timing nodes in the first trajectory.

[0049] In a second aspect, a display device is provided, the display device including a screen controller, a screen, and a screen connection component, the screen being connected to the screen connection component, and the screen controller being connected to both the screen and the screen connection component; the screen controller is configured to execute the screen control method provided in the first aspect or any possible implementation thereof; the screen is configured to play the first material in response to a second instruction; and the screen connection component is configured to control the screen to move at least during the playback time period of the first material in response to the third instruction.

[0050] Thirdly, a vehicle is provided, including a display device as provided in the second aspect.

[0051] Thirdly, a readable storage medium is provided, the readable storage medium including instructions that, when at least one processor of the device executes the instructions, cause the device to perform a screen control method as described in the first aspect or any possible implementation thereof; and / or, a screen control method as described in the first aspect or any possible implementation thereof.

[0052] Fourthly, a computer program product is provided, the computer program product including instructions that, when at least one processor of a device executes the instructions, cause the device to perform the screen control method as described in the first aspect or any possible implementation thereof.

[0053] Fifthly, a chip is provided, comprising: a processor and a memory, the memory being used to store a computer program; the processor being used to run the computer program to enable an electronic device in which the chip is located to implement the screen control method as described in the first aspect or any possible implementation thereof.

[0054] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0055] Figure 1 A schematic diagram of a fixed central control display screen provided for related technologies; Figure 2 A schematic diagram of the structure of a vehicle provided in an embodiment of this application; Figure 3 A schematic diagram of an in-vehicle screen provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a display device provided in an embodiment of this application; Figure 5 A flowchart illustrating an interaction method provided in an embodiment of this application; Figure 6 This is a schematic diagram of a vehicle coordinate system; Figure 7 For the screen Figure 6 The diagram shown is in the vehicle coordinate system. Figure 8 A schematic diagram illustrating the playback and motion time periods provided in the embodiments of this application; Figure 9 A schematic diagram illustrating screen movement in a holiday scene, provided as an embodiment of this application; Figure 10 A schematic diagram illustrating screen movement in a cute pet scene, provided as an embodiment of this application; Figure 11 A schematic diagram illustrating screen motion in a ball impact scenario provided in an embodiment of this application; Figure 12 A flowchart illustrating another interaction method provided in an embodiment of this application; Figure 13 A flowchart illustrating a process for determining whether a triggering condition is met, provided as an embodiment of this application; Figure 14 A flowchart illustrating another interaction method provided in an embodiment of this application; Figure 15 A flowchart illustrating another interaction method provided in an embodiment of this application; Figure 16A flowchart illustrating another interaction method provided in an embodiment of this application; Figure 17 This is a flowchart for generating a first action orchestration signal, provided as an embodiment of this application.

[0056] Marker explanation: 100 - Vehicle; 110 - Display device; 111 - Screen; 112 - Screen controller; 113 - Screen connection component; 120 - Sensing system; 130 - Computing platform. Detailed Implementation

[0057] The following sections will discuss the fabrication and use of various embodiments in detail. However, it should be understood that many applicable inventive concepts provided in this application can be implemented in a variety of specific environments. The specific embodiments discussed are merely illustrative of specific ways of implementing and using this description and technology, and do not limit the scope of this application. Unless otherwise defined, all technical terms used herein have the same meaning as commonly known to one of ordinary skill in the art. Circuits or other components may be described or referred to as "for" performing one or more tasks. In this context, "for" is used to imply a structure by indicating that the circuit / component includes a structure (e.g., a circuit system) that performs one or more tasks during operation. Therefore, even when the specified circuit / component is currently inoperable (e.g., not turned on), it can still be referred to as "for performing that task." Circuits / components used with the term "for" include hardware, such as circuits that perform operations.

[0058] The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, a, b, and c; where a, b, and c can be single or multiple.

[0059] The embodiments of this application use terms such as "first" and "second" to distinguish objects with similar names, functions, or effects. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or order of execution. The term "coupling" is used to indicate an electrical connection, including direct connection via wires or terminals or indirect connection via other devices. Therefore, "coupling" should be considered as a broad type of electronic communication connection.

[0060] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0061] First, let me explain the relevant terms used in this application.

[0062] 1. The vehicle coordinate system refers to a three-dimensional Cartesian coordinate system fixed on the vehicle body. It is a rigid body coordinate system that moves and rotates as the vehicle moves, and is used to describe the position, attitude, and motion state of the vehicle itself and its various components.

[0063] Figure 6 This is a schematic diagram of a vehicle coordinate system.

[0064] like Figure 6 As shown, the origin is usually located at the vehicle's design reference point, which is the intersection of the rear axle center and the vehicle's longitudinal plane of symmetry when the vehicle is stationary. The x-axis represents the vehicle's forward direction, with the positive direction pointing directly forward; the y-axis represents the driver's left direction, with the positive direction pointing to the left side of the vehicle (from the driver's perspective); and the z-axis represents the vertical upward direction, with the positive direction pointing towards the top of the vehicle.

[0065] 2. A trajectory refers to the continuous path left by an object when it moves in space; it is a curve showing how its position changes over time.

[0066] Before introducing the embodiments of this application, the relevant technical background involved in this application will be described below.

[0067] As the level of automotive intelligence continues to improve, the functions of in-vehicle screens are becoming increasingly diverse, expanding from simple navigation to multiple scenarios such as audio-visual entertainment, vehicle control, and connected services. However, most mainstream in-vehicle screens are currently fixed in design, unable to be adjusted according to the position or preference of passengers, making it difficult to meet the viewing needs of all users in different riding scenarios.

[0068] Figure 1 A schematic diagram of a fixed central control display screen provided for related technologies.

[0069] For example, such as Figure 1 As shown, the central control display screen in a vehicle is usually fixed between the driver's and front passenger's seats, facing the rear seats. Because the angle cannot be adjusted, it cannot meet the different viewing angle needs of passengers. For example, passengers sitting in the driver's or front passenger's seat cannot properly view the content displayed on the central control screen.

[0070] Although some technologies have attempted to achieve limited adjustments to screen orientation, such as rotating from landscape to portrait mode, or rotating to the left or right at a preset angle, their flexibility and adaptability are still insufficient, making it difficult to match more complex interaction methods and diverse usage scenarios.

[0071] Therefore, a smarter solution is urgently needed to improve the user experience.

[0072] In view of this, this application provides a screen control method. The screen is connected to a screen connection component, and the screen and the screen connection component are also connected to a screen controller. Based on this structure, the screen controller obtains a first instruction, sends a second instruction to the screen based on the first instruction, and sends a third instruction to the screen connection component. The second instruction instructs the screen to play first material, and the third instruction instructs the screen connection component to control the screen to move at least during the playback time of the first material. This movement refers to the screen performing one or more actions such as scaling, panning, tilting left and right, nodding up and down, turning, and shaking. Therefore, by controlling the screen to move while playing content, adapting the movement to the content being played, the immersive interaction of the scene can be improved, enhancing the user experience.

[0073] Optionally, in holiday or pet-themed scenarios, the screen can play relevant materials such as videos, audio, and images. The screen connection component can control the screen to adapt to the movement of the relevant materials, at least during the time the materials are played on the screen.

[0074] To facilitate understanding, the structure and applicable scenarios of the vehicle will be introduced below.

[0075] The embodiments provided in this application can be applied to various vehicles, including but not limited to: road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or recreational equipment. The vehicle can be defined in a broad sense. For example, a vehicle can be a means of transportation (such as cars, buses, subways, high-speed trains, motorcycles, flying cars, trains, etc.), an industrial vehicle (such as forklifts, trailers, tractors, etc.), a construction vehicle (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, boats, hovercrafts, submarines, airplanes, helicopters, etc. The embodiments in this application do not limit the specific type, form, or function of the vehicle.

[0076] Figure 2 This is a structural schematic diagram of a vehicle 100 provided in an embodiment of this application. Figure 2 As shown, the vehicle 100 may include various subsystems, such as a display device 110, a sensing system 120, and a computing platform 130. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include one or more components. Furthermore, each subsystem and component of the vehicle 100 may be interconnected via wired or wireless means.

[0077] The display device 110 may include one or more screens 111. It should be understood that the screens 111 are used to display images, videos, etc. The screens 111 may be flat screens, curved screens, foldable screens, etc., and this application does not limit the type of screen or the process used.

[0078] Figure 3 This is a schematic diagram of an in-vehicle screen provided in an embodiment of this application.

[0079] like Figure 3 As shown, the screen 111 can be the vehicle's central control screen 1111, passenger side screen 1112, driver's headrest screen 1113, passenger side headrest screen 1114, screen behind the central control armrest 1115, roof folding screen 1116, etc. This application does not limit the position of the screen 111 in the vehicle.

[0080] Taking screen 111 as the central control screen as an example, Figure 4 This is a schematic diagram of the structure of a display device provided in an embodiment of this application.

[0081] like Figure 4 As shown, the display device 110 may further include a screen controller 112 and a screen connection component 113. The screen 111 is connected to the screen connection component 113, and the screen controller 112 is connected to the screen 111 and the screen connection component 113 respectively.

[0082] Optionally, when the display device 110 includes multiple screens 111, the display device 110 should include multiple screen connection components 113, and the screens 111 and screen connection components 113 have a one-to-one correspondence. The display device 110 may include one or more screen controllers 112. Wherein, if the display device 110 includes one screen controller 112, the screen controller 112 is set independently of the screen or integrated into a certain screen (such as a central control screen); if the number of screen controllers 112 included in the display device 110 is the same as the number of screens 111, and each screen controller has a corresponding relationship with a group of screens 112 and screen connection components 113, then each screen controller 112 can be set independently of the screen 111 or integrated into its corresponding screen 111. This application does not limit this.

[0083] In this embodiment, the screen controller 112 is communicatively connected to the screen 111 and the screen connection component 113. Specifically, wired communication can be used, but wireless communication methods such as Bluetooth, NearLink, Near Field Communication (NFC), ZigBee, Radio Frequency Identification (RFID), Ultra Wide Bandwidth (UWB), Wireless Fidelity (Wi-Fi), and other future short-range communication technologies can also be used. This embodiment does not limit the connection method between the screen controller 112 and the screen 111 and the screen connection component 113.

[0084] The screen controller 112 can be used to perform the functions provided in the embodiments of this application. Figure 5 , Figures 11 to 17 The relevant steps in the method described herein will not be repeated here; please refer to the subsequent content for details.

[0085] Optionally, the screen 111 and the screen connection assembly 113 can be fixedly connected or detachably connected. For example, the screen 111 and the screen connection assembly 113 can be connected by threads, snaps, or plugs, or they can be magnetically attached.

[0086] As an example, the screen connection assembly 113 may include a three-bar linkage or a rotating structure. The three-bar linkage comprises three links, each connected to the screen back panel. By extending, retracting, and rotating these three links to varying degrees, the screen's movement can be controlled.

[0087] As another example, the screen connection assembly 113 may further include a screen connector and a screen connector controller. The screen connector is connected to the screen, and the screen connector controller is connected to the screen controller. The screen connector may include a three-bar linkage or a rotating structure. The screen connector controller receives instructions from the screen controller and sends action signals to the screen connector based on the instructions, thereby controlling the screen connector to perform corresponding actions according to the action signals.

[0088] The perception system 120 may include several types of sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a positioning system, which may be a global positioning system (GPS), a BeiDou system, or another positioning system. The perception system 120 may include one or more of the following: an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0089] In this embodiment, the sensing system 120 may further include one or more of an audio sensor, a pressure sensor, a capacitive sensor, an infrared sensor, or a seatbelt buckle sensor.

[0090] Some or all of the functions of vehicle 100 can be controlled by computing platform 130. Computing platform 130 may include processors 131 to 13n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor unit (MPU), graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 130 may also include a memory for storing instructions. Some or all of the processors 131 to 13n can call the instructions in the memory to implement the corresponding functions.

[0091] The computing platform 130 can control the functions of the vehicle 100 based on inputs received from various subsystems, such as the sensing system 120. In some embodiments, the computing platform 130 can be used to provide control over many aspects of the vehicle 100 and its subsystems.

[0092] In this embodiment, the sensing system 120 can detect or receive user operations and send the acquired input data to the processor in the computing platform 130. The processor in the computing platform 130 can then generate instructions (such as a first instruction or a sixth instruction) based on the input data and send the instructions and related status information to the screen controller. The screen controller can be located in the display device 110, integrated into the processor in the computing platform 130, or it can be a processor within the computing platform 130; this application does not limit this. Furthermore, the description of the instructions and status information will be provided later and will not be repeated here.

[0093] In this embodiment, the processor in the computing platform 130 can be connected to the screen controller via Ethernet, CAN / CAN FD bus, LIN bus and A2B bus to achieve data communication and collaborative control.

[0094] Optionally, the above components are just an example. In actual applications, the components in each of the above modules may be added or deleted as needed. Understandably, vehicle 100 may also include more or fewer subsystems, and each subsystem may include multiple components. For example, vehicle may also include computer systems, sensor systems, etc., the specifics of which will not be elaborated in the embodiments of this application. Figure 2 This should not be construed as a limitation on the embodiments of this application. It is understood that each subsystem and component of the vehicle 100 can be interconnected via wired or wireless means.

[0095] In conjunction with the aforementioned vehicle 100, the method provided in this application can be applied to non-driving scenarios, in which the vehicle is in P gear or in a parked state; in other words, the vehicle is powered on but not in a driving state.

[0096] It should be understood that "vehicle is powered on" refers to a broad definition, meaning that the low-voltage electrical system in the vehicle is at least operational, and some components such as controllers (e.g., screen controllers) and screens are working. "Vehicle is not moving" means the vehicle is stationary.

[0097] For example, when a user approaches a vehicle with a physical or smart key and performs an unlocking operation, the screen starts working, but the vehicle is not in motion.

[0098] For example, a user remotely operates the vehicle's screen via an electronic device (such as a mobile phone), but the vehicle is not in motion.

[0099] For example, the screen starts working after the user opens the car door or presses the start button, but the vehicle is not in motion.

[0100] For example, after the user is seated in the driver's seat, presses the brake pedal and then the start button, the screen begins to operate, but the vehicle is not in motion. It should be understood that the above are merely a few examples, and the screen control method provided in this application can also be applied to other situations where the vehicle is not in motion.

[0101] Optionally, when the vehicle is not in motion, the screen controlled by the screen control method provided in this application can be any one or more screens in the vehicle, such as the central control screen, the passenger screen, the headrest screen, the screen behind the central armrest, the roof folding screen, etc., and this application does not limit it.

[0102] In addition, the method provided in this application embodiment can also be applied to vehicle driving scenarios, in which the vehicle is in D or R gear, in other words, the vehicle is in a driving state.

[0103] Optionally, when the vehicle is in motion, in order not to affect driving, the screen controlled by the method provided in this application can be any one or more screens other than the central control screen, such as the passenger screen, headrest screen, screen behind the central armrest, roof folding screen, etc. This application does not limit this.

[0104] Optionally, the central control screen can also be controlled when the vehicle is in motion. However, it can first be determined whether the screen control method provided in this application can be executed based on the preset level corresponding to the content currently displayed on the central control screen.

[0105] It should be understood that the above are only a few examples, and the screen control method provided in this application can also be applied to other situations where the vehicle is in motion.

[0106] Figure 5 This is a flowchart illustrating an interaction method provided in an embodiment of this application.

[0107] like Figure 5 As shown, this method can be applied to the above. Figure 2 The vehicle 100 shown includes a display device 110. The method may include steps S110 to S114, which will be described in detail below.

[0108] S110, the screen controller receives the first instruction.

[0109] It should be understood that the first instruction is used to instruct the screen controller to send a second instruction to the screen and a third instruction to the screen connection component.

[0110] Optionally, the vehicle may receive a first operation, generate a first instruction in response to the first operation, and then send the first instruction to the screen controller. The first operation is used to instruct the activation of the display device or, in other words, to activate the operation of the screen controller.

[0111] For example, the first operation can be a touch operation, a voice operation, a gesture operation, or a biometric operation. The touch operation can include pressing a physical button on the vehicle key or vehicle, bringing the smart key near or attaching it to the vehicle, or opening a vehicle door.

[0112] For example, a user presses the unlock button or remote start button on the vehicle key; in response to the press, the vehicle unlocks and generates a first command.

[0113] For example, if a user binds their vehicle to a mobile application (APP), and the user approaches the vehicle with their phone, the vehicle can unlock itself and generate a first command via Bluetooth sensing; or, the user can place a card key, an NFC-enabled mobile phone, or a smartwatch on the door's sensing area, and the vehicle will unlock and generate a first command.

[0114] For example, after the user opens the car door, the vehicle unlocks and generates the first command; or, after the user opens the car door and presses the start button on the vehicle, the vehicle unlocks and generates the first command.

[0115] For example, before or after a user opens the car door, after the user performs a biometric operation such as fingerprint, facial, or voice recognition, the vehicle generates the first instruction based on the recognition result.

[0116] For example, if a user long-presses a physical switch on the screen, the screen controller corresponding to the current screen can receive a first instruction in response to the long-press operation; or, if a user clicks a virtual switch displayed on a first screen (such as a central control screen), the screen controller corresponding to a second screen (such as a folding screen on the roof) can receive a first instruction in response to the click operation.

[0117] It should be understood that the vehicle can detect or receive the first operation based on the sensors and other devices included in the perception system 120, and send the acquired input data to the processor in the computing platform 130; then, the processor can generate a first instruction based on the input data, and then send the first instruction to the screen controller in the display device 110.

[0118] Optionally, if multiple screen controllers are shared, the first instruction can be sent to the shared screen controller. The first instruction may include the screen and the unique identifier (ID) of the screen connection component that corresponds one-to-one with each screen. If the number of screen controllers is the same as the number of screens, and each group of screen controllers, screens, and screen connection components has a corresponding relationship, then the first instruction needs to be sent to the target screen controller and includes the IDs of the screen and screen connection component that correspond to the target screen controller.

[0119] S111. Based on the first instruction, the screen controller sends a second instruction to the screen, which instructs the screen to play the first material.

[0120] Optionally, the screen controller can send the second instruction to the screen and simultaneously send the third instruction to the screen connection component, or it can send the second instruction to the screen and the third instruction to the screen connection component in sequence. This application does not limit the order of sending.

[0121] The first material can be video, audio, images, etc., and this application does not limit the type of material.

[0122] Optionally, the first material can be stored on the screen, and the second instruction can include information such as the name, ID, and storage address of the first material to instruct the screen to call and play the first material. Alternatively, the first material can also be stored in the screen controller, and the second instruction includes the data packet corresponding to the first material, instructing the screen to play it after receiving the data packet.

[0123] The first material can be material produced by a manufacturer or a third party, or it can be user-generated material. If the first material is user-generated material, it can be received through a screen, other electronic devices connected to the screen, or external interfaces physically connected to the screen, etc. This application does not limit the method of receiving the first material.

[0124] S112. Based on the first instruction, the screen controller sends a third instruction to the screen connection component. The third instruction is used to instruct the screen connection component to control the screen to move at least during the playback time of the first material.

[0125] The movement refers to the screen being able to perform actions such as stretching, panning, tilting left and right, nodding up and down, tilting, and shaking. The movement can include a repetitive movement of a single action, or continuous or intermittent movement of multiple actions. This application does not limit the scope of the movement.

[0126] Figure 7 For the screen Figure 6 The diagram shown is in the vehicle coordinate system.

[0127] In the embodiments of this application, combined with Figure 6 and Figure 7 The vehicle coordinate system shown allows the screen to move back and forth along the x-axis, translate left and right along the y-axis, or move up and down along the z-axis under the control of the screen connection component; it can also rotate around the x-axis, rotate around the y-axis (i.e., nod up and down), or rotate around the z-axis (i.e., shake left and right).

[0128] Based on this, the screen can also perform compound movements in multiple directions simultaneously, such as stretching, translating, and rotating around a certain axis at the same time. In other words, the screen can make arbitrary movements in three-dimensional space under the control of the screen connection components.

[0129] In addition, "the screen moves at least during the playback time of the first material" means that the screen's movement time is at least included within the playback time of the first material.

[0130] Figure 8 This is a schematic diagram of the playback time period and the exercise time period provided in the embodiments of this application.

[0131] like Figure 8 As shown in (a), assuming the playback time of the first material is from T1 to T2, the screen starts moving simultaneously when the first material begins playing, but stops moving before the playback ends; as Figure 8 As shown in (b), the screen does not move when the first piece of material begins to play, but starts moving after a certain period of time, and then stops moving simultaneously at the end of the playback; as... Figure 8 As shown in (c), the screen begins to move simultaneously with the start of the first clip and stops moving simultaneously with the end of the clip. Figure 8 As shown in (d), the screen intermittently underwent three movements during the playback of the first material.

[0132] Optionally, the screen's motion time period may also extend beyond the playback time period of the first material, that is, the motion may occur before playback begins, and / or the motion may stop after playback ends.

[0133] like Figure 8 As shown in (e), the screen moves before playback begins but stops moving before playback ends; as... Figure 8 As shown in (f), the screen does not move when the first piece of material begins playing, but starts moving after a certain period of time, and then stops moving again some time after the playback ends; as... Figure 8 As shown in (g), the screen moves before the first piece of material begins playback and stops moving at the end of playback; alternatively, the screen may start moving at the beginning of playback but stop moving some time after playback ends (not shown in the figure). Figure 8 As shown in (h), the screen moves before the first material is played. During the playback, it moves intermittently in three segments. Alternatively, the screen can move intermittently in three segments during the playback and then stop moving some time after the playback ends (not shown in the figure).

[0134] It should be understood that the above are merely examples, and this application does not limit them.

[0135] Optionally, the third instruction is used to instruct the screen connection component to control the screen to move at least during the playback period of the first material, based on the first motion arrangement signal.

[0136] The first action orchestration signal includes one or more action signals, with different action signals used to instruct the screen connection component to control the screen to perform different actions. For each of the one or more action signals, the first action orchestration signal may also include information such as the frequency and orchestration order of each action signal.

[0137] For example, the first motion orchestration signal includes an motion signal that instructs the screen connection component to control the screen to nod up and down. The first motion orchestration signal may also include the frequency corresponding to the motion signal, such as twice per second.

[0138] For example, the first action arrangement signal includes 5 action signals, which respectively instruct the screen connection component to control the screen to perform 5 different actions. The first action arrangement signal may also include the arrangement order corresponding to the 5 action signals, such as executing them in the order of 1, 3, 1, 2, 1, 4, 5.

[0139] Optionally, the first motion orchestration signal can be stored in the screen connection component, and the third instruction can include information such as the name, ID, and storage address of the first motion orchestration signal, instructing the screen connection component to call the first motion orchestration signal and perform motion to achieve the purpose of controlling screen motion; or, the first motion orchestration signal can also be stored in the screen controller, and the third instruction can include the data packet corresponding to the first motion orchestration signal, and instruct the screen connection component to call it after receiving the data packet.

[0140] The first action arrangement signal can be signal data produced by the manufacturer or a third party, or it can be signal data generated by the screen controller based on the content of the first material. This application does not limit it in this way.

[0141] In this embodiment of the application, the screen connection component can, under the instruction of the third instruction, control the screen to move at least during the playback time of the first material according to the action signal included in the first action arrangement signal, and execute the arrangement action corresponding to the first material, thereby achieving the effect of playing while the screen moves.

[0142] S113, the first source material is played on the screen.

[0143] After receiving the second instruction, the screen responds by starting to play the first material.

[0144] After the first piece of material has finished playing on the screen, the screen can also send a notification message to the screen controller to inform that the first piece of material has finished playing.

[0145] S114, The screen connection component controls the screen to move at least during the playback time of the first material.

[0146] Upon receiving the third instruction, the screen connection component responds to the third instruction by controlling the screen to move for at least the duration of the first material's playback.

[0147] If the screen connection component includes a screen connector and a screen connector controller, the screen connector controller can receive a third instruction, and then, in response to the third instruction, the screen connector controller, through the screen connector, begins to control the screen to move at least during the playback time of the first material.

[0148] Optionally, the movement of the screen during at least the playback time of the first material matches the first trajectory of the target object in the first material, the matching including spatial path consistency and / or time synchronization.

[0149] The target object in the first material may include one or more objects, which may indicate various types of objects such as people and animals, or interactive behaviors between multiple objects; further, if the object is a single object, the target object may indicate all or part of the object's content.

[0150] The target object in the first source material can be extracted and / or filtered. For example, if an object (such as a kitten) is displayed in multiple frames of images included in the first source material, with some images showing partial content (such as the kitten's head) and others showing the full content (such as the entire kitten), then when filtering the target object, the intersection content corresponding to the object in the multiple frames of images can be determined as the target object (such as filtering the kitten's head as the target object).

[0151] The initial trajectory corresponding to the target object can also be designed and modified. For example, after determining the initial trajectory corresponding to the target object, filtering or smoothing can be performed to obtain the first trajectory. This application does not limit the processing.

[0152] The synchronization of screen movement with the first trajectory time means that when the screen connection component controls the screen to perform each action, the time node of each action is consistent with the time node sequence in the first trajectory. For example, if the target object moves from the first time point to the second time point, from right to left, the screen can synchronously perform an action between the first time point and the second time point. However, this action can be any action such as translation, rotation, or translation plus rotation. Of course, the screen can also move from right to left.

[0153] In this embodiment, by controlling the screen movement to match the first trajectory of the target object in the first material, the screen movement and the material content can be adapted, so that the screen movement synchronously follows the changes in the displayed content, thereby enhancing the immersive experience.

[0154] For example, Figure 9 This is a schematic diagram of screen movement in a holiday scene, provided as an embodiment of this application.

[0155] like Figure 9 As shown, assuming the holiday scene is Christmas, the screen can play Christmas-related videos (i.e., the first source material), such as a Christmas tree shaking its branches to shake off the snow. The objects in this video include the Christmas tree and the snow on it. The target object obtained from the video is the interaction between the Christmas tree and the snow, specifically the shaking of the tree branches to shake off the snow. Based on this interaction, the corresponding trajectory can be extracted; this trajectory can also be called the shaking trajectory. Then, based on this shaking trajectory, the screen can be controlled to perform spatial path and / or temporal matching.

[0156] For example, based on this shaking trajectory, the screen connection component can control the screen to rotate around the z-axis and move back and forth along the y-axis at the same time. The spatial path of the screen movement is consistent with the shaking trajectory, and / or the time node of the screen movement controlled by the screen connection component corresponds to the timing of the shaking node, thereby simulating the shaking effect of tree branches.

[0157] For example, Figure 10 This is a schematic diagram of screen movement in a cute pet scene provided in an embodiment of this application.

[0158] like Figure 10 As shown, assuming the main character in a cute pet scene is a kitten, the screen can play multiple video clips related to the kitten (i.e., the first source material), such as the kitten curling up and scrolling from right to left on the screen, and then from left to right. The object in the video is the kitten, and the target object obtained from the video can be simply the kitten's head. Based on the kitten's head, a corresponding trajectory can be extracted; this trajectory can also be called the head movement trajectory. Then, based on this head movement trajectory, the screen can be controlled to perform spatial path and / or temporal matching.

[0159] For example, based on the head movement trajectory, the screen connection component can control the screen to move in the yoz plane and rotate around the x-axis at the same time. The spatial path of the screen movement is consistent with the head movement trajectory, and / or the screen connection component controls the time nodes of the screen movement to correspond to the timing of the head movement nodes, thereby simulating the rolling effect of a kitten.

[0160] For example, Figure 11This is a schematic diagram of screen motion in a ball impact scenario provided in an embodiment of this application.

[0161] like Figure 11 As shown, assuming a ball-collision scenario, the screen can play a video related to the ball collisions (i.e., the first source material), such as multiple balls colliding with each other and rolling back and forth within the screen's range. The objects in this video are multiple balls, and the target object obtained from the video can be the interaction behavior between these balls, i.e., the collisions between them. Based on this interaction behavior, the corresponding trajectory can be extracted; this trajectory can also be called the collision trajectory. Then, based on this collision trajectory, the screen can be controlled to perform spatial path and / or temporal matching.

[0162] For example, based on the impact trajectory, the screen connection component can control the screen to move in three-dimensional space, and the spatial path of the screen movement is consistent with the impact trajectory. And / or, the time node of the screen movement controlled by the screen connection component corresponds to the timing of the impact node, thereby simulating the impact effect of multiple small balls.

[0163] After the screen motion is complete, the screen connection component can also send a notification to the screen controller to inform it that the motion is complete.

[0164] In this embodiment, the screen controller acquires a first instruction, sends a second instruction to the screen based on the first instruction to instruct the screen to play first material, and sends a third instruction to the screen connection component to instruct the screen connection component to control the screen to move at least during the playback period of the first material. This movement refers to the screen performing one or more actions such as scaling, panning, tilting left and right, nodding up and down, turning, and shaking. Therefore, by playing the first material while the screen moves, the interactive immersion of the scene can be improved, enhancing the user experience.

[0165] Figure 12 This is a flowchart illustrating another interaction method provided in an embodiment of this application.

[0166] Optionally, such as Figure 12 As shown, after S110 above, the method may further include step S115.

[0167] S115, The screen controller determines whether the triggering conditions are met.

[0168] When the triggering condition is met, subsequent steps, such as steps S111 and S112, can be executed. When the triggering condition is not met, subsequent steps can be skipped and the process can end; alternatively, step S115 can be repeated until the triggering condition is met.

[0169] Optionally, the triggering condition may include the identification of keywords.

[0170] It should be understood that keywords can be preset and have a one-to-one correspondence with the materials that the screen controller instructs the screen to play.

[0171] For example, if the triggering condition is only when a keyword is identified, after the screen controller receives the first instruction, the screen controller also needs to confirm whether the first instruction includes a keyword, or the screen controller also needs to confirm whether other instructions including a keyword have been received.

[0172] For example, after a user opens the car door, they perform a voice recognition operation. Based on the recognition result, the vehicle generates a first command, which includes keywords. Alternatively, after a user long-presses a physical switch on the screen, the screen controller corresponding to the current screen can obtain the first command in response to the long-press operation; then, the user performs a voice operation, and in response to the voice operation, the screen controller can obtain reference information including keywords.

[0173] If "Merry Christmas" is detected and matches the corresponding keyword in the content, the screen controller can instruct the screen to play Christmas-related content. If "hello Mimi" is detected and matches the corresponding keyword in the content, the screen controller can instruct the screen to play content related to the cute pet Mimi. If "ball collision" is detected and matches the corresponding keyword in the content, the screen controller can instruct the screen to play content related to ball collision.

[0174] Optionally, the triggering condition may include the first power-on of the day.

[0175] It should be understood that "each first power-on" indicates the first power-on of the screen controller each day, or the first power-on of the vehicle including the screen controller each day.

[0176] For example, after a user opens the car door, the vehicle unlocks and generates a first command. After the screen controller receives the first command, it can obtain reference information, including the number of power-ups, from the processor and determine whether the current power-up count is the first of the day. If so, the screen controller is triggered to send the second and third commands; otherwise, the second and third commands will not be triggered.

[0177] Optionally, the triggering condition may include the current time point being within the valid usage period of the first material.

[0178] It should be understood that the effective usage period of the first material can be preset.

[0179] For example, if the first material is related to Christmas, the corresponding valid usage period is from the early morning of December 20th to the early morning of December 26th; if the first material is related to New Year's Day, the corresponding valid usage period is from the early morning of January 1st to the early morning of January 4th.

[0180] After receiving the first instruction, the screen controller can obtain reference information, including the current time, from the processor. The current time indicates the time when the screen controller received the first instruction, and determines whether the current time is within the valid usage period of the first material. If so, it then triggers the sending of the second and third instructions.

[0181] Optionally, the triggering condition may include that the target switch included in the target application is turned on.

[0182] The target application can be a settings application, and the target switch can be a virtual switch within the settings application. In response to a user's click, the target switch can switch from an off state to an on state, or vice versa.

[0183] When the target switch is turned on, it indicates that the target interactive mode is activated; when the target switch is turned off, it indicates that the target interactive mode is deactivated. The target interactive mode includes playing target material on the screen, and the screen connection component controlling the screen to move at least during the playback period of the target material, wherein the first material is included in the target material.

[0184] The screen controller stores the current state of the target switch. After receiving the first instruction, it can obtain the current state of the target switch from itself and determine whether the current state is on. If so, it will then trigger the sending of the second and third instructions.

[0185] Optionally, the triggering condition may also include the vehicle being in a non-driving state.

[0186] It should be understood that when a vehicle is not in motion, the gear is in P (Park) or in the parking position. Examples of when a vehicle is not in motion can be found in the above description of non-driving scenarios, and will not be repeated here.

[0187] After receiving the first instruction, the screen controller can obtain reference information including the vehicle status from the processor, and when it determines that the vehicle status is not driving, it will then trigger the sending of the second and third instructions.

[0188] It should be noted that the triggering conditions described above can be combined in any way. An example of one combination method is described below.

[0189] Figure 13 This is a flowchart illustrating a process for determining whether a triggering condition is met, as provided in an embodiment of this application.

[0190] like Figure 13 As shown, step S115 may include steps S1151 to S1154.

[0191] S1151. Determine whether a keyword has been identified. If a keyword has been identified, continue to step S1152, or determine whether the triggering condition is met; otherwise, the triggering condition is not met.

[0192] If no keywords are recognized when recognizing voice information, the triggering condition is not met. The screen controller can then perform other controls based on the voice information, such as playing music or opening navigation.

[0193] S1152. Determine if this is the first power-on of the day. If it is the first power-on of the day, continue to step S1153, or determine if the triggering condition is met; otherwise, the triggering condition is not met.

[0194] S1153. Determine whether the current time point is within the effective usage period of the first material; if yes, continue to step S1154, or determine whether the triggering condition is met; if not, the triggering condition is not met.

[0195] If the current time is not within the valid usage period of the first material, the triggering condition is not met. The screen controller can respond with a voice message to remind the user that the valid usage period has not expired or has expired. It can also provide keywords for other materials for the user to choose from.

[0196] S1154. Determine whether the target switch included in the target application is turned on; if yes, continue to step S1155, or determine whether the trigger condition is met; if not, the trigger condition is not met.

[0197] If the target switch included in the target application is not turned on, the triggering condition is not met. The screen controller can simply respond with a voice message, such as "Merry Christmas" or "Happy New Year." Alternatively, it can remind the user that the target switch is not turned on and ask the user if they want to turn it on. If the user continues to receive a voice command to turn on the switch, then the target switch can be turned on.

[0198] S1155. Determine whether the vehicle is currently in a non-driving state; if yes, the triggering condition is met; otherwise, the triggering condition is not met.

[0199] If the vehicle is currently in motion, and the triggering conditions are not met, the screen controller can simply respond with voice commands such as "Merry Christmas" or "Happy New Year." Alternatively, it can remind the user that the vehicle is in motion and ask if the user wants to continue playing the first material. If the user receives the instruction to continue playing, it can send the second and third instructions. Alternatively, the screen controller can further determine whether the second and third instructions can be sent based on the preset level corresponding to the content currently displayed on the screen.

[0200] For example, if a navigation map is currently displayed, it can be determined that the map application is set to a higher level, so the second and third commands cannot be sent; if a short video is currently displayed, it can be determined that the short video application is set to a lower level, so the second and third commands can be sent; in addition, the short video content can continue to play after the first material has finished playing.

[0201] It should be understood that the above is only one combination method, and this application does not limit the number or order of the triggering conditions used for combination.

[0202] For example, targeting Figure 9 For the festival scenario shown, steps S1151 to S1155 can be executed; while for... Figure 10 The cute pet scene shown can be executed by simply performing steps S1151, S1153, and S1154; for Figure 11 The ball collision scenario shown can be performed by simply executing steps S1152, S1153, and S1154.

[0203] In this embodiment of the application, before the screen controller sends a second instruction to the screen to instruct the screen to play the first material based on the first instruction, and sends a third instruction to the screen connection component to instruct the screen connection component to control the screen to move at least during the playback time of the first material, it can first determine whether the triggering condition is met. If the triggering condition is met, the second instruction and the third instruction are then sent. Thus, by judging the triggering condition, it is possible to avoid affecting the user's normal driving or normal execution of other operations.

[0204] Figure 14 A flowchart of yet another interaction method provided in an embodiment of this application.

[0205] like Figure 14 As shown, in Figure 5 or Figure 12 Based on this, the method may further include steps S116 to S121.

[0206] If the method includes step S115, then steps S116 and S117 can be executed before step S115, and steps S118 to S121 can be executed after step S115; or, steps S116 to S121 can also be executed after step S115 and before S111; or, steps S116 to S121 can also be executed after step S114, and this application does not limit this.

[0207] The following explanation will take steps S116 to S121 as examples of being executed after step S115 and before step S111, or after step S114.

[0208] S116. Obtain the seating status, which can be any one of the following: only the driver is seated, only the passenger is seated, or both the driver and the passenger are seated.

[0209] The above are just a few examples. The seating arrangements can also include only the rear seats behind the driver's seat, only the rear seats behind the front passenger seat, and both the rear seats behind the driver's seat and the rear seats behind the front passenger seat, depending on the number of seats in the vehicle.

[0210] It should be understood that a vehicle can use a pressure sensor under the seat to determine the seating status by measuring changes in weight; it can also combine one or more of a capacitive sensor, an infrared sensor, a camera, an audio sensor, or a seatbelt buckle sensor to determine the seating status. This application does not limit the method of determining the seating status.

[0211] It should also be understood that after the vehicle determines the seating status through one or more sensors, it can send the seating status to the screen controller or the processor; after receiving the first instruction, the screen controller can obtain reference information including the seating status from the processor.

[0212] S117. Based on the seated state, determine the preset position corresponding to the screen.

[0213] Different seating positions allow you to set different preset positions for the screen.

[0214] For example, when only the driver is seated, the preset position of the screen can be set to the first preset position facing the driver; when only the passenger is seated, the preset position of the screen can be set to the second preset position facing the passenger; when both the driver and passenger are seated, the preset position of the screen can be set to the third preset position that maintains the center orientation.

[0215] For other seating positions, different preset positions can be set separately. Please refer to the above content for details, which will not be repeated here.

[0216] Optionally, the preset position corresponding to the screen can also be preset based on the content being played (such as the second material), or it can be determined based on other methods, such as setting based on information such as the vehicle's pose and speed.

[0217] In this embodiment of the application, different preset positions are set for the screen for different seating states. This allows passengers in different seating positions to view the screen content from the best angle, or to facilitate passengers to perform subsequent operations after the current content has finished playing.

[0218] S118, The screen controller sends a fourth instruction to the screen, which is used to instruct the screen to play the second material.

[0219] Optionally, the screen controller can send the fourth instruction to the screen and simultaneously send the fifth instruction to the screen connection component, or it can send the fourth instruction to the screen and the fifth instruction to the screen connection component sequentially. The order is not limited in this application.

[0220] The second material can be video, audio, images, etc., and this application does not limit the type of material.

[0221] Optionally, the second material can be stored on the screen, and the fourth instruction can include information such as the name, ID, and storage address of the second material to instruct the screen to call up and play the second material. Alternatively, the second material can also be stored in the screen controller, and the fourth instruction includes the data packet corresponding to the second material and instructs the screen to receive the data packet and play it.

[0222] The second material can be material produced by the manufacturer or a third party, or it can be user-defined material. If the second material is user-defined material, it can be received through a screen, other electronic devices connected to the screen, or external interfaces physically connected to the screen, etc. This application does not limit this.

[0223] S119. The screen controller sends a fifth instruction to the screen connection component, which instructs the screen connection component to control the screen to move to a preset position.

[0224] The fifth instruction can be used to instruct the screen connection component to control the screen to move to a preset position based on a preset position signal.

[0225] Different preset position signals are used to instruct the screen connection component to control the screen to move to different preset positions.

[0226] Optionally, the fifth instruction is used to instruct the screen connection component to control the screen to move to a preset position during the playback time of the second material. That is, the time period during which the screen moves to the preset position is included in the playback time of the second material.

[0227] For example, taking steps S116 to S121 as being executed after step S115 and before S111, combined with Figure 9 In the holiday scene shown, before playing the first material, the screen can also play the second material, such as the audio "Merry Christmas, Xiaoyi is giving you a Christmas tree". During the playback period, the screen connection component can also control the screen to move to a preset position (which can also be called the preset initial position). For example, if only the driver sits down, it will move to the first preset position facing the driver.

[0228] For example, taking the execution of steps S116 to S121 after step S114 as an example, combined with Figure 9In the holiday scene shown, after playing the first material, the screen can also play the second material, such as the audio "Isn't the Christmas tree beautiful? See you next time." During the playback period, the screen connection component can also control the screen to move to a preset position (which can also be called a preset end position), such as a third preset position that maintains the center orientation.

[0229] In this embodiment of the application, the screen connection component can control the screen to move during the playback time of the second material based on the fifth instruction, and move to a preset position. Thus, the effect of playing the video while the screen moves to the preset position can be achieved.

[0230] Alternatively, the screen can be moved to a preset position at the start or end of the playback of the second material. In this case, the time period during which the screen moves to the preset position is outside the playback time period of the second material, and this application does not limit this.

[0231] In addition, the preset position may also be consistent with the current position of the screen. That is, the fifth instruction is equivalent to instructing the screen connection component to control the screen to keep its position unchanged.

[0232] S120, the screen plays the second material.

[0233] After receiving the fourth instruction, the screen responds by starting to play the second material.

[0234] After the second clip finishes playing, the screen can send a notification to the screen controller to indicate that the second clip has finished playing. Then, the screen controller can issue a second command to the screen to instruct it to play the first clip.

[0235] S121, The screen connection component controls the screen to move to a preset position.

[0236] After receiving the fifth instruction, the screen connection component responds to the fifth instruction by controlling the screen to move to the preset position during the playback time of the second material.

[0237] If the screen connection component includes a screen connector and a screen connector controller, the screen connector controller can receive a fifth instruction. Then, in response to the fifth instruction, the screen connector controller, through the screen connector, begins to control the screen to move to a preset position during the playback time of the second material.

[0238] After the screen moves to the preset position, the screen connection component can also send a notification to the screen controller to indicate that the movement is complete. Then, the screen controller can send a third command to the screen connection component to instruct the screen to move.

[0239] It should be understood that before moving in response to the third instruction, the screen connection component has already controlled the screen to move to the preset position. Therefore, the screen connection component is equivalent to controlling the screen to start moving from the preset position.

[0240] In this embodiment, the screen controller, by acquiring the seating state, can determine the corresponding preset position of the screen based on the seating state. Then, it sends a fourth instruction to the screen connection component to instruct the screen to play the second material, and a fifth instruction to the screen connection component to instruct the screen connection component to control the screen to move to the preset position. Thus, the screen can play the second material and move to the preset position before or after playing the first material and moving, serving as preview content before the first material is played or ending content afterward, enhancing the user experience.

[0241] Figure 15 This is a flowchart illustrating another interaction method provided in an embodiment of this application.

[0242] like Figure 15 As shown, in Figure 5 , Figure 12 or Figure 14 Based on this, the method may further include steps S210 to S214.

[0243] If the method includes steps S116 to S121, steps S210 to S214 can be executed during the execution of steps S120 and S121; or steps S210 to S214 can be executed during the execution of steps S113 and S114.

[0244] The following explanation will take the execution of steps S210 to S214 during the execution of steps S113 and S114 as an example.

[0245] S210, the screen controller obtains the sixth instruction.

[0246] It should be understood that the sixth instruction is used to instruct the screen control to send the seventh instruction to the screen and the eighth instruction to the screen connection component.

[0247] Optionally, the vehicle can receive a second operation, generate a sixth instruction in response to the second operation, and then send the sixth instruction to the screen controller. The second operation is used to indicate a pause, switch, or close operation.

[0248] For example, the second operation can be a touch operation, a voice operation, a gesture operation, or a biometric operation. Touch operations can include pressing a physical button on the vehicle key or vehicle, or shifting vehicle gears.

[0249] For example, when a user performs a voice command, such as "Hey Celia, open the navigation," the vehicle generates a sixth command in response to the voice command.

[0250] For example, when a user shifts into D or R gear, the vehicle can generate a sixth command in response to the gear shift.

[0251] It should be understood that the vehicle can detect or receive the second operation based on the sensors and other devices included in the perception system 120, and send the acquired input data to the processor in the computing platform 130; then, the processor can generate a sixth instruction based on the input data, and then send the sixth instruction to the screen controller in the display device 110.

[0252] Optionally, if multiple screen controllers are shared, the sixth instruction can be sent to the shared screen controller. Furthermore, the sixth instruction may include the IDs of the screens and the screen connection components that have a one-to-one correspondence with each screen. If the number of screen controllers and screens is the same, and each group of screen controllers, screens, and screen connection components has a corresponding relationship, then the sixth instruction needs to be sent to the target screen controller and includes the IDs of the screens and screen connection components that correspond to the target screen controller.

[0253] S211. Based on the sixth instruction, the screen controller sends a seventh instruction to the screen. The seventh instruction is used to instruct the screen to pause playback, end playback, or switch playback content.

[0254] It should be understood that when the screen controller instructs the screen to pause playback, it means that the screen pauses the playback of the first material; when the screen controller instructs the screen to stop playback, it means that the screen stops the playback of the first material; when the screen controller instructs the screen to switch playback content, it means that the screen switches from the playback interface of the first material to another interface.

[0255] S212. Based on the sixth instruction, the screen controller sends an eighth instruction to the screen connection component. The eighth instruction is used to instruct the screen connection component to control the screen to pause motion, end motion, or switch motion.

[0256] The seventh and eighth instructions correspond to each other, both pausing, both ending, or both switching.

[0257] It should be understood that the screen connection component controlling the screen to pause motion indicates that the screen is paused from motion based on the first action programming signal; the screen connection component controlling the screen to end motion indicates that the screen is ended from motion based on the first action programming signal; the screen connection component controlling the screen to switch motion indicates that the screen is switched from the current motion to another motion.

[0258] S213. If the screen is currently playing the first material, then in response to the seventh instruction, the screen can pause the playback of the first material, stop the playback of the first material, or switch to another interface.

[0259] S214. If the current screen connection component is controlling the screen to move based on the first action orchestration signal, then in response to the eighth instruction, the screen can pause the movement, end the movement, or switch to another movement.

[0260] In this embodiment, the screen controller obtains a sixth instruction and sends a seventh instruction to the screen based on the sixth instruction to instruct the screen to pause, end, or switch to another interface; and sends an eighth instruction to the screen connection component to instruct the screen connection component to control the screen to pause, end, or switch to another motion. Thus, it is possible to pause, end, or switch to other modes from the mode of playing the first material while moving, thereby meeting other functional needs of the user.

[0261] It should also be understood that if steps S210 to S214 are executed after step S114, the seventh instruction can instruct the screen to pause playback of the second material, stop playback of the second material, or switch to another interface; the eighth instruction can be used to instruct the screen connection component to control the screen to pause, stop, or switch to another position. Controlling the screen to pause indicates that the movement performed by pausing the screen is paused; controlling the screen to stop indicates that the movement performed by stopping the screen is stopped; controlling the screen to switch to another position indicates that the screen is switched from its current position to another position.

[0262] In addition, when the current time reaches the end of the effective usage period of the first material, the screen controller can also obtain the sixth instruction, and then, based on the sixth instruction, send the seventh instruction to the screen to instruct the screen to stop playing the first material or the second material; and send the eighth instruction to the screen connection component to instruct the screen connection component to control the screen to stop moving or shifting.

[0263] Figure 16 This is a flowchart illustrating another interaction method provided in an embodiment of this application.

[0264] like Figure 16 As shown, in Figure 5 , Figure 12 , Figure 14 or Figure 15 Based on this, the method may also include steps S01 to S04 before step S110, which will be described in detail below.

[0265] S01. The screen controller receives a first data packet, which includes a first material, or the first material and a first motion arrangement signal.

[0266] The screen controller can connect to the cloud server via the network and request OTA (Over-the-Air Technology) upgrades from the cloud server; after receiving the request, the cloud server can send the first data packet to the screen controller.

[0267] The first data packet may include only the first material, or it may include the first material and a first motion arrangement signal, wherein the first motion arrangement signal may be signal data produced by a manufacturer or a third party and has only a playback relationship with the first material; or the first motion arrangement signal may also be signal data generated based on the content of the first material and has an association relationship with the content of the first material.

[0268] In addition, the first data package may also include relevant information such as the keywords corresponding to the first material and the available time period corresponding to the first material.

[0269] Optionally, the first data packet may also include the second material, or the second material and a preset position corresponding to the screen.

[0270] S02. When the first data packet does not include the first motion arrangement signal, generate the first motion arrangement signal based on the first material.

[0271] It should be understood that when the first data packet includes the first action arrangement signal, the subsequent step S03 can be executed directly.

[0272] For example, Figure 17 This is a flowchart for generating a first action orchestration signal, provided as an embodiment of this application.

[0273] like Figure 17 As shown, step S02 above may include steps S021 to S026.

[0274] S021. Obtain the keyframe sequence from the first source material.

[0275] When the first source material is video, a keyframe sequence can be obtained from the video frames indicated by the first source material. Typically, the number of keyframes is less than the number of video frames; therefore, the number of frames in the keyframe sequence is less than the number of frames in the first source material.

[0276] S022. Obtain the target object in the keyframe sequence.

[0277] The target object in a keyframe sequence can include one or more objects, which can indicate various types of objects such as people and animals, or interactive behaviors between multiple objects; further, if the object is a single object, the target object can indicate the global or local content of that object.

[0278] For example, if the first material includes an object, such as a kitten, then the target object could indicate the kitten's head.

[0279] S023. Based on the keyframe sequence and the target object, determine the first trajectory, which includes the coordinates of the target object in each keyframe.

[0280] In this embodiment of the application, the calculation of the first trajectory using a keyframe sequence can improve computational efficiency and save resources.

[0281] With sufficient computing power, it is also possible to obtain the target object in the first material from all frame sequences of the first material, and determine the first trajectory based on the entire frame sequence and the target object. The first trajectory includes the coordinates of the target object in each frame.

[0282] S024. Based on the first trajectory, generate the first action sequence.

[0283] Optionally, three-dimensional motion data along the x, y, and z axes can be extracted based on the first trajectory; then, a first action sequence can be generated based on the extracted three-dimensional motion data along the x, y, and z axes. The first action sequence may include multiple actions in a sequential order.

[0284] S025. Generate the first action arrangement signal based on the first action sequence.

[0285] The first action sequence can include actions such as stretching, translating, shaking the head left and right, nodding up and down, tilting the head, and shaking. Based on each action included in the first action sequence, a corresponding action signal can be generated. Then, after arranging them in sequence, a first action arrangement signal can be formed.

[0286] S03, The screen controller sends the first material to the screen.

[0287] S04. The screen controller sends the first action arrangement signal to the screen connection component.

[0288] The first motion arrangement signal includes one or more motion signals. Different motion signals are used to instruct the screen connection component to control the screen to perform different actions, which is equivalent to controlling the screen to move. Furthermore, the movement is consistent with the first trajectory corresponding to the target object in the first material in terms of spatial path and / or synchronized in terms of time.

[0289] In this embodiment of the application, the first data can be sent to the screen controller through software upgrade. The first data packet may include the first material, or may include the first material and the first motion orchestration signal. When the first data packet does not include the first motion orchestration signal, the screen controller can generate the first motion orchestration signal based on the first material. Thus, the actions of the screen connection components can be orchestrated in combination with the content of the material.

[0290] It should be understood that Figures 1 to 17 The structural diagrams, flowcharts, or scenario diagrams shown are for illustrative purposes only and are not intended to limit the embodiments of this application to the examples illustrated. In fact, those skilled in the art can interpret them based on... Figures 1 to 17 The examples in the examples can be equivalently transformed to obtain more implementation methods. In the above embodiments, the steps can be executed in different orders as presented in each embodiment, and it is not necessary to execute all the operations in the embodiments of this application. Moreover, the order of the steps does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0291] This application also provides a chip, which includes a processor and a memory, wherein the memory stores a computer program; and the processor runs the computer program to enable the electronic device in which the chip is located to perform the above-described functions. Figure 5 , Figures 12 to 17 The steps performed by the screen controller in the method described in the illustrated embodiment.

[0292] Optionally, the chip further includes a memory connected to the processor via a circuit or wire, the processor being used to read and execute computer programs stored in the memory. Further optionally, the chip includes a communication interface to which the processor is connected. The communication interface is used to receive data and / or information that needs to be processed, the processor obtaining the data and / or information from the communication interface and processing the data and / or information. The communication interface can be an input / output interface.

[0293] The memory can be read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices. Alternatively, it can be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0294] This application also provides a chip system, which includes a processor and a memory. The memory stores a computer program, and the processor runs the computer program to enable the electronic device in which the chip system resides to perform the above-described functions. Figure 5 , Figures 12 to 17 The steps performed by the screen controller or display device in the method described in the illustrated embodiment.

[0295] This application also provides a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform... Figure 5 , Figures 12 to 17 The steps performed by the screen controller or display device in the method described in the illustrated embodiment.

[0296] This application also provides a computer program product containing instructions that, when run on a computer or any at least one processor, cause the computer to execute... Figure 5 , Figures 12 to 17 The steps performed by the screen controller or display device in the method described in the illustrated embodiment.

[0297] The chips, chip systems, computer storage media, or computer program products provided in this application embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0298] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0299] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0300] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0301] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A screen control method, characterized in that, The method, applied to a screen controller, wherein the screen is connected to a screen connection component, and the screen controller is respectively connected to the screen and the screen connection component, includes: Obtain the first instruction; Based on the first instruction, a second instruction is sent to the screen, the second instruction being used to instruct the screen to play the first material; Furthermore, a third instruction is sent to the screen connection component, the third instruction being used to instruct the screen connection component to control the screen to move at least during the playback time of the first material.

2. The method as described in claim 1, characterized in that, The third instruction is used to instruct the screen connection component to control the screen to move at least during the playback time of the first material based on the first motion arrangement signal; The first action orchestration signal includes one or more action signals, and different action signals are used to instruct the screen connection component to control the screen to perform different actions.

3. The method as described in claim 1 or 2, characterized in that, The movement of the screen at least during the playback time of the first material matches the first trajectory corresponding to the target object in the first material, and the matching includes spatial path consistency and / or time synchronization.

4. The method according to any one of claims 1 to 3, characterized in that, After obtaining the first instruction, the method further includes: Determine if the triggering conditions are met; If so, then based on the first instruction, the second instruction is sent to the screen, and the third instruction is sent to the screen connection component.

5. The method as described in claim 4, characterized in that, The triggering conditions include one or more of the following: Keywords identified, first power-on of the day, current time is within the effective usage period of the first material, and the target switch included in the target application has been turned on; When the target switch is turned on, it is used to indicate the activation of the target interaction mode. The target interaction mode includes the screen playing target material, and the screen connection component controlling the screen to move at least during the playback time of the target material. The first material is included in the target material.

6. The method as described in claim 4 or 5, characterized in that, When the screen controller is included in a vehicle, the triggering condition further includes: the vehicle is currently in a non-driving state.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Send a fourth instruction to the screen, the fourth instruction being used to instruct the screen to play the second material; Furthermore, a fifth instruction is sent to the screen connection component, which instructs the screen connection component to control the screen to move to a preset position.

8. The method as described in claim 7, characterized in that, When the screen controller is included in a vehicle, the method further includes: Obtain the seating status, which is any one of the following: only the driver is seated, only the passenger is seated, or both the driver and the passenger are seated. Based on the seated state, the preset position corresponding to the screen is determined.

9. The method as described in claim 7 or 8, characterized in that, The fifth instruction is used to instruct the screen connection component to control the screen to move to the preset position during the playback time of the second material.

10. The method according to any one of claims 1 to 9, characterized in that, After sending a third instruction to the screen connection component, the method further includes: Obtain the sixth instruction; Based on the sixth instruction, a seventh instruction is sent to the screen, and an eighth instruction is sent to the screen connection component; Wherein, the seventh instruction is used to instruct the screen to pause playback, and the eighth instruction is used to instruct the screen connection component to control the screen to pause movement; or, the seventh instruction is used to instruct the screen to end playback, and the eighth instruction is used to instruct the screen connection component to control the screen to end movement; or, the seventh instruction is used to instruct the screen to switch playback content, and the eighth instruction is used to instruct the screen connection component to control the screen to switch movement.

11. The method according to any one of claims 2 to 10, characterized in that, The method further includes: Receive a first data packet, the first data packet including the first material, or the first material and the first motion orchestration signal; When the first data packet does not include the first motion orchestration signal, the first motion orchestration signal is generated based on the first material. Send the first material to the screen; The first action orchestration signal is sent to the screen connection component.

12. The method as described in claim 11, characterized in that, When the first material is video, the motion choreography signal is generated based on the first material, including: Based on the first material, determine the first trajectory; Based on the first trajectory, the first action choreography signal is generated.

13. The method as described in claim 12, characterized in that, Based on the first material, the first trajectory is determined, including: Obtain the keyframe sequence from the first source material; Obtain the target object in the keyframe sequence; Based on the keyframe sequence and the target object, the first trajectory is determined, and the first trajectory includes the coordinates of the target object in each keyframe.

14. The method as described in claim 12 or 13, characterized in that, Based on the first trajectory, generating the first action arrangement signal includes: generating a first action sequence based on the first trajectory; Based on the first action sequence, the first action arrangement signal is generated.

15. A display device, characterized in that, The display device includes a screen controller, a screen, and a screen connection component. The screen is connected to the screen connection component, and the screen controller is connected to the screen and the screen connection component, respectively. The screen controller is used to perform the screen control method as described in any one of claims 1 to 14; The screen is used to play the first material in response to the second instruction; The screen connection component is used to respond to the third instruction to control the screen to move at least during the playback time of the first material.

16. A vehicle, characterized in that, The vehicle includes the display device as described in claim 15.

17. A readable storage medium, characterized in that, The readable storage medium includes instructions that, when executed by at least one processor of the device, cause the device to perform the method as described in any one of claims 1 to 14.

18. A computer program product, characterized in that, The computer program product includes instructions that, when executed by at least one processor of the device, cause the device to perform the method as described in any one of claims 1 to 14.

19. A chip, characterized in that, The chip includes a processor for performing the steps of the method according to any one of claims 1 to 14.

Images