Vehicle entertainment control method and device, vehicle and storage medium

Abstract

The application relates to a vehicle-mounted entertainment control method and device, a vehicle and a storage medium. The method comprises the following steps: acquiring vehicle state data and environment perception data; in response to at least one user interaction instruction, determining a current working scene of the vehicle based on the vehicle state data, the environment perception data and the at least one user interaction instruction; generating a cooperative control instruction according to the current working scene, and performing linkage adjustment on display parameters of a projection device based on the cooperative control instruction, or arbitrating the execution sequence of the at least one user interaction instruction, or outputting safety warning information and entertainment content after fusion rendering. Therefore, the scene is identified through multi-source data fusion, the cooperative control instruction is generated, linkage adjustment of projection, dynamic arbitration of instructions and safety-entertainment fusion rendering are realized, and the problem that a vehicle-mounted entertainment system cannot simultaneously consider the stability of large-size projection picture quality, the safety of multi-person interaction and the continuity of entertainment experience under a dynamic driving environment is solved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to an in-vehicle entertainment control method, device, vehicle, and storage medium. Background Technology

[0002] With the development of vehicle intelligence, in-cabin entertainment systems are evolving towards larger sizes and more immersive experiences. Solutions that can accommodate retractable screens and in-vehicle projectors are gradually emerging, providing a giant-screen viewing experience in limited in-vehicle space.

[0003] Among related technologies, there are already methods that attempt to apply projection display technology to in-vehicle scenarios, such as windshield projection display and exterior projection lighting. Some methods also involve adaptive screen brightness adjustment or multimodal interactive control.

[0004] However, when the above methods are applied to large-screen entertainment projection in vehicles, the in-vehicle entertainment system has problems in maintaining the stability of large-screen projection image quality, the safety of multi-person interaction, and the continuity of entertainment experience under dynamic driving conditions, which urgently need to be solved. Summary of the Invention

[0005] This application provides an in-vehicle entertainment control method, device, vehicle, and storage medium to solve the problem that in-vehicle entertainment systems cannot simultaneously ensure the stability of large-size projection image quality, the safety of multi-user interaction, and the continuity of entertainment experience in dynamic driving environments, thereby achieving intelligent coordination and dynamic balance between driving safety and immersive viewing experience.

[0006] To achieve the above objectives, the first aspect of this application proposes an in-vehicle entertainment control method, comprising the following steps: Acquire vehicle status data and environmental perception data; In response to at least one user interaction command, the current working scenario of the vehicle is determined based on the vehicle status data, the environmental perception data, and the at least one user interaction command. Based on the current working scenario, generate collaborative control instructions, and based on the collaborative control instructions, adjust the display parameters of the projection device in a coordinated manner, or arbitrate the execution order of the at least one user interaction instruction, or merge and render safety warning information with entertainment content and output it to a retractable screen.

[0007] According to one embodiment of this application, the step of adjusting the display parameters of the projection device in conjunction with the projection device includes: Vibration energy characteristic values ​​are determined based on the vehicle state data, and illumination gradient characteristic values ​​are determined based on the environmental perception data. Based on the vibration energy characteristic value and the illumination gradient characteristic value, the fusion control parameters are calculated; In response to the fusion control parameter being greater than a preset threshold, an anti-shake command and a brightness adjustment command are generated, and the projection device is controlled to perform a linkage response based on the anti-shake command and the brightness adjustment command.

[0008] According to one embodiment of this application, the vibration energy characteristic value is:

[0009] in, The characteristic value of the vibration energy, This represents the number of sampling points within the sampling window. The sampling point number, For the first X-axis acceleration values ​​at each sampling point For the first Y-axis acceleration values ​​at each sampling point For the first The squared value of the X-axis acceleration at each sampling point For the first The squared value of the Y-axis acceleration at each sampling point; The characteristic value of the illumination gradient is:

[0010] in, The illumination gradient feature value is... For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For time intervals.

[0011] According to one embodiment of this application, arbitrating the execution order of the at least one user interaction instruction includes: Based on the vehicle status data, determine the driver status parameters and vehicle speed parameters; Calculate the safety risk coefficient based on the driver status parameters and the vehicle speed parameters; In response to the safety risk coefficient being greater than a preset risk threshold, a driver safety instruction is determined from the at least one user interaction instruction, and the priority of the driver safety instruction is set to the highest. The current entertainment task is interrupted to prioritize the execution of the driver safety instruction.

[0012] According to one embodiment of this application, the step of fusing and rendering safety warning information with entertainment content and then outputting it to a retractable screen includes: Based on the collaborative control command, the safety warning information and the entertainment content are combined in a preset display control layer to obtain a composite image; The fused image is projected onto the retractable screen via the projection device.

[0013] According to the in-vehicle entertainment control method proposed in this application, the method acquires vehicle status data and environmental perception data; responds to at least one user interaction command, determines the current working scenario of the vehicle based on the vehicle status data, environmental perception data, and at least one user interaction command; generates a collaborative control command based on the current working scenario, and adjusts the display parameters of the projection device in a coordinated manner based on the collaborative control command, or arbitrates the execution order of at least one user interaction command, or merges and renders safety warning information with entertainment content and outputs it to a retractable screen. Thus, by identifying the scenario through multi-source data fusion and generating collaborative control commands, the method achieves projection linkage adjustment, dynamic command arbitration, and safe entertainment fusion rendering, solving the problem that in-vehicle entertainment systems cannot simultaneously maintain the stability of large-size projection image quality, the safety of multi-user interaction, and the continuity of the entertainment experience in dynamic driving environments. This achieves intelligent coordination and dynamic balance between driving safety and an immersive viewing experience.

[0014] To achieve the above objectives, a second aspect of this application provides an in-vehicle entertainment control device, comprising: The acquisition module is used to acquire vehicle status data and environmental perception data; A determination module is used to determine the current working scenario of the vehicle based on the vehicle status data, the environmental perception data, and the at least one user interaction command in response to at least one user interaction command. The control module is used to generate collaborative control instructions based on the current working scenario, and based on the collaborative control instructions, to adjust the display parameters of the projection device in a coordinated manner, or to arbitrate the execution order of the at least one user interaction instruction, or to merge and render safety warning information with entertainment content and then output it to the retractable screen.

[0015] According to one embodiment of this application, the control module for adjusting the display parameters of the projection device in conjunction with the control module is specifically used for: Vibration energy characteristic values ​​are determined based on the vehicle state data, and illumination gradient characteristic values ​​are determined based on the environmental perception data. Based on the vibration energy characteristic value and the illumination gradient characteristic value, the fusion control parameters are calculated; In response to the fusion control parameter being greater than a preset threshold, an anti-shake command and a brightness adjustment command are generated, and the projection device is controlled to perform a linkage response based on the anti-shake command and the brightness adjustment command.

[0016] According to one embodiment of this application, the vibration energy characteristic value is:

[0017] in, The characteristic value of the vibration energy, This represents the number of sampling points within the sampling window. The sampling point number, For the first X-axis acceleration values ​​at each sampling point For the first Y-axis acceleration values ​​at each sampling point For the first The squared value of the X-axis acceleration at each sampling point For the first The squared value of the Y-axis acceleration at each sampling point; The characteristic value of the illumination gradient is:

[0018] in, The illumination gradient feature value is... For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For time intervals.

[0019] According to one embodiment of this application, the control module, in arbitrating the execution order of the at least one user interaction instruction, is specifically used for: Based on the vehicle status data, determine the driver status parameters and vehicle speed parameters; Calculate the safety risk coefficient based on the driver status parameters and the vehicle speed parameters; In response to the safety risk coefficient being greater than a preset risk threshold, a driver safety instruction is determined from the at least one user interaction instruction, and the priority of the driver safety instruction is set to the highest. The current entertainment task is interrupted to prioritize the execution of the driver safety instruction.

[0020] According to one embodiment of this application, the control module, which integrates and renders safety warning information with entertainment content and then outputs it to a retractable screen, is specifically used for: Based on the collaborative control command, the safety warning information and the entertainment content are combined in a preset display control layer to obtain a composite image; The fused image is projected onto the retractable screen via the projection device.

[0021] According to the vehicle entertainment control device proposed in this application, the device acquires vehicle status data and environmental perception data; responds to at least one user interaction command, determines the current working scenario of the vehicle based on the vehicle status data, environmental perception data, and at least one user interaction command; generates collaborative control commands based on the current working scenario, and adjusts the display parameters of the projection device in a coordinated manner based on the collaborative control commands, or arbitrates the execution order of at least one user interaction command, or merges and renders safety warning information with entertainment content and outputs it to a retractable screen. Thus, by using multi-source data fusion to identify the scenario and generate collaborative control commands, the device achieves projection linkage adjustment, dynamic command arbitration, and safe entertainment fusion rendering, solving the problem that vehicle entertainment systems cannot simultaneously maintain the stability of large-size projection image quality, the safety of multi-user interaction, and the continuity of the entertainment experience in dynamic driving environments. This achieves intelligent coordination and dynamic balance between driving safety and an immersive viewing experience.

[0022] To achieve the above objectives, a third aspect of this application provides a vehicle comprising: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the in-vehicle entertainment control method as described in the above embodiments.

[0023] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the vehicle in-vehicle entertainment control method as described in the above embodiments.

[0024] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the in-vehicle entertainment control method as described in the above embodiments.

[0025] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0026] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an in-vehicle entertainment control method provided according to an embodiment of this application; Figure 2 This is a schematic diagram of the architecture of an in-vehicle entertainment control system according to an embodiment of this application; Figure 3 This is a schematic diagram of a multi-sensor data fusion and image quality optimization process according to an embodiment of this application; Figure 4This is a schematic diagram of a dynamic priority arbitration process for user interaction commands according to an embodiment of this application; Figure 5 This is a block diagram of an in-vehicle entertainment control device provided according to an embodiment of this application; Figure 6 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0028] Among related technologies, three core shortcomings of in-vehicle entertainment systems have been exposed after the introduction of large-size screens: (1) Image quality degradation caused by the failure of multi-sensor data coordination. The vibration compensation, ambient light adaptation and screen deformation monitoring modules of the in-vehicle entertainment system are independent of each other and adopt a simple threshold triggering mechanism. As a result, in the combined harsh environment of vehicle bumps and strong light, the anti-shake algorithm and brightness adjustment strategy cannot work together. The consequence is that although the image shake can be suppressed, the image contrast and color saturation drop sharply, making it difficult to meet the stringent requirements of high-quality display in dynamic driving environment.

[0029] (2) System response chaos and safety risks caused by multimodal input command conflicts. The static priority arbitration mechanism cannot perceive the vehicle status and the role of the passengers. When a large screen attracts multiple passengers to perform voice, gesture and touch operations at the same time, the passengers' entertainment commands may block the driver's key safety commands, creating a driving safety hazard.

[0030] (3) Issues with user experience and performance caused by the disconnect between security warning information and entertainment content display. The relevant technology uses a pop-up interruption warning method, which brutally destroys the immersive entertainment experience and may even cause users to actively disable security functions. The root cause lies in the decoupling of the UI (User Interface) rendering layer and the security control layer in the architecture, which fails to achieve pixel-level deep and seamless integration of warning information and entertainment content.

[0031] Based on the above problems, this application proposes an in-vehicle entertainment control method to systematically solve the coordination problems of image quality optimization, command arbitration and safety warning.

[0032] The following description, with reference to the accompanying drawings, outlines an in-vehicle entertainment control method, apparatus, vehicle, and storage medium according to embodiments of this application.

[0033] Figure 1This is a flowchart of an embodiment of an in-vehicle entertainment control method according to this application.

[0034] Before introducing the in-vehicle entertainment control method proposed in the embodiments of this application, the in-vehicle entertainment control system involved in the in-vehicle entertainment control method will be introduced first.

[0035] like Figure 2 As shown, this in-vehicle entertainment control system is centered on the vehicle's main control chip, constructing a four-layer collaborative architecture from bottom to top: sensor layer, data processing layer, function execution layer, and display control layer. The sensor layer integrates vibration sensors (such as IMUs), ambient light sensors (such as RGB-IR sensors), screen deformation sensors (such as capacitive deformation detectors), and multimodal input devices (such as microphones, gesture cameras, and touchscreens) to comprehensively collect vehicle status, environmental information, and user interaction commands. The data processing layer deploys a real-time data fusion engine, a dynamic command arbitration module, and a safety-entertainment fusion rendering unit, responsible for unified processing of multi-source data uploaded from the sensor layer and intelligent scheduling of commands and display content. The function execution layer, in conjunction with the anti-shake driver, brightness adjustment module, and command execution unit, rapidly converts the control commands output from the data processing layer into specific actions. The display control layer integrates the screen driver, UI renderer, and security control layer, supporting pixel-level content fusion display while ensuring that the security control layer has independent operating permissions to ensure the reliability of security-related operations.

[0036] In practical applications, after the in-vehicle entertainment control system is powered on, the main control chip starts up and loads a hybrid operating system based on a hypervisor. Android Automotive is used for entertainment functions, while QNX (a real-time operating system) is used for safety-critical functions. At the sensor layer, the IMU installed in the center of the chassis begins detecting the vehicle's three-axis acceleration and angular velocity, with data transmitted via the CAN FD (Controller Area Network Flexible Data-Rate) bus. An RGB-IR ambient light sensor on the upper edge of the windshield collects the ambient spectral intensity, with data transmitted via the bus. A ToF (Time of Flight) distance sensor integrated into the projector's optical module initiates 8×8 multi-area ranging to monitor the screen's planar deformation in real time, with data transmitted via the bus. A linear array of microphones inside the vehicle completes beamforming calibration for voice acquisition and sound source localization. A 3D depth camera inside the vehicle activates skeletal tracking mode, covering the entire cabin. In terms of execution and display, the electric roll mechanism hidden in the rear lining of the roof completes its self-check and is ready to unfold; the miniature laser projection unit in the rear center armrest box is activated, and the optical output power is adjusted to the initial lumen value.

[0037] Next, the in-vehicle entertainment control method proposed in the embodiments of this application will be described in detail.

[0038] For example, such as Figure 1 As shown, the in-vehicle entertainment control method includes the following steps: In step S101, vehicle status data and environmental perception data are acquired.

[0039] It is understood that, in this embodiment of the application, vehicle status data refers to information reflecting the vehicle's own operating status, including but not limited to vehicle speed (and acceleration), gear position, parking status, light status, steering wheel angle, vehicle posture (and angular velocity (i.e., the amount of change in an object's angle per unit time, reflecting the vehicle's rotational state)). Environmental perception data refers to information reflecting the environment in which the vehicle is located, including but not limited to ambient light intensity, rainfall, temperature, the position and posture of occupants inside the vehicle, and the distance values ​​of various areas between the projection device and the screen plane.

[0040] Specifically, the system can collect vehicle status data and environmental perception data in real time through the sensor layer. For example, it can use an IMU to collect the vehicle's acceleration and angular velocity data in three-dimensional space, a light sensor to collect ambient light intensity, and a screen deformation sensor to collect distance values ​​between the projection device and the screen plane in various areas. Among these, acceleration and angular velocity data reflect the vehicle's vibration and attitude changes; ambient light intensity reflects the lighting conditions inside and outside the vehicle; and the distance values ​​between the projection device and the screen plane in various areas reflect the screen's deformation state. These data collectively constitute the basic information input for the system to perceive the current driving environment and vehicle status.

[0041] In step S102, in response to at least one user interaction command, the current working scenario of the vehicle is determined based on vehicle status data, environmental perception data, and at least one user interaction command.

[0042] It is understood that, in the embodiments of this application, user interaction commands refer to operation requests issued by users (drivers and passengers in the vehicle) to the system through voice, gestures, or touch, such as "play a movie," "increase the volume," or "navigate to a gas station." The current working scenario refers to the current operating situation of the vehicle, such as a driving movie-watching scenario, a multi-command concurrent scenario, or a scenario where safety information needs to be presented simultaneously.

[0043] Specifically, in response to at least one received user interaction command, the system analyzes vehicle status data to determine whether the vehicle is in a driving or parked state, identifies the lighting conditions and occupant distribution inside the vehicle through environmental perception data, and combines the type and source of the user interaction command to accurately identify whether the current work scenario requires starting movie entertainment, processing multiple concurrent commands, or simultaneously presenting safety warning information, thus providing a decision-making basis for subsequent collaborative control.

[0044] In step S103, a collaborative control instruction is generated according to the current working scenario, and based on the collaborative control instruction, the display parameters of the projection device are adjusted in a coordinated manner, or the execution order of at least one user interaction instruction is arbitrated, or the safety warning information and entertainment content are merged and rendered and then output to the retractable screen.

[0045] It is understood that, in this embodiment, the collaborative control command is a comprehensive control command used to simultaneously or selectively coordinate multiple execution modules to respond in a coordinated manner. Display parameters refer to the operating parameters of the projection device, including but not limited to anti-shake strength, brightness, contrast, keystone correction, etc. A retractable screen is a projection display carrier installed in the roof lining of a vehicle and capable of being unfolded and retracted via an electric roller mechanism.

[0046] In other words, the system can generate a collaborative control command matching the current working scenario identified in step S102, and execute corresponding control operations based on the command. Specifically, in the case of a driving movie-watching scenario, the system can adjust the display parameters of the projection device in a coordinated manner, so that parameters such as image stabilization and brightness respond in tandem; in the case of a multi-command concurrent scenario, the system can arbitrate the execution order of multiple received user interaction commands to determine the order of processing each command; in the case of a scenario where safety information needs to be presented synchronously, the system can merge and render safety warning information with entertainment content, and output the composite image to a retractable screen for display. Thus, the system can adaptively execute corresponding collaborative control strategies according to the needs of different scenarios.

[0047] As one possible implementation, in some embodiments, the display parameters of the projection device are adjusted in a coordinated manner, including: determining vibration energy characteristic values ​​based on vehicle state data and determining illumination gradient characteristic values ​​based on environmental perception data; calculating fusion control parameters based on the vibration energy characteristic values ​​and illumination gradient characteristic values; generating anti-shake commands and brightness adjustment commands in response to the fusion control parameters being greater than a preset threshold, and controlling the projection device to perform a coordinated response based on the anti-shake commands and brightness adjustment commands.

[0048] It is understood that, in this embodiment, the vibration energy characteristic value is used to characterize the severity of the vehicle's current vibration. The illumination gradient characteristic value is used to characterize the severity of changes in ambient light. The fusion control parameter refers to the parameter used to determine whether to trigger linkage adjustment. The preset threshold is a pre-set parameter critical value used to determine whether the current vibration and illumination changes have reached a level requiring linkage adjustment. The anti-shake command is the control command used to activate or enhance the image stabilization function of the projection device. The brightness adjustment command is the control command used to adjust the output light intensity of the projection device.

[0049] Specifically, such as Figure 3 As shown, when the user issues the voice command "play movie," the system enters audio-visual mode. The retractable screen unfolds from the roof to the predetermined position. Simultaneously, the real-time data fusion engine in the data processing layer begins operation (i.e., it adjusts the display parameters of the projection device accordingly). The system can first calculate the vibration energy characteristic value based on vehicle status data. This allows for the quantification of the current vehicle vibration intensity; simultaneously, it calculates the illumination gradient characteristic value based on environmental perception data. This allows for the quantification of the severity of changes in ambient light; furthermore, it enables the calculation of screen deformation characteristic values ​​based on the distance values ​​between the projection device and various areas of the screen plane. This quantifies the deformation state of the curtain. Subsequently, the system inputs these two feature values ​​into a pre-defined correlation matrix model (such as...). A comprehensive calculation is performed to obtain the fusion control parameters. When the fusion control parameter is greater than the preset threshold (e.g., when a vehicle enters a tunnel), (Sudden drop) while encountering road surface joints (i.e. The sudden increase indicates that the current vehicle vibration and lighting changes have reached a level that requires coordinated processing. The data fusion engine then outputs coordinated control commands to the function execution layer. On the one hand, it sends anti-shake commands to the anti-shake driver of the projection device to activate the three-level Kalman filter anti-shake; on the other hand, it sends brightness adjustment commands to the brightness adjustment module to smoothly increase the output power from 800 lumens to 1600 lumens within 0.5 seconds, thereby ensuring that users will not perceive image jitter and insufficient brightness when watching in the tunnel.

[0050] For example, in the process of coordinated control based on fused control parameters, such as Figure 3 As shown, if the fusion control parameters If the value is ≤0.4, it can be determined that the environment is good (i.e., the vehicle is driving smoothly, the lighting conditions are suitable, and the screen deformation is small), and the system is executing the standard mode (i.e., only maintaining the basic working state of the projection device, without additional anti-shake enhancement or brightness increase, in order to reduce energy consumption and ensure basic projection image quality); if 0.4 < If the value is ≤0.7, it can be determined as a moderate disturbance, and the system will implement a strategy of fine-tuning brightness combined with intermediate-level anti-shake (such as a second-level Kalman filter); if If the value is greater than 0.7, it can be determined that the system is operating under strong vibration and enhanced lighting conditions (such as a bumpy road surface while entering a tunnel). The system will then implement an enhancement strategy that includes brightness boosting, advanced image stabilization (such as a three-level Kalman filter), and cool color temperature correction. Finally, the system can output corresponding coordinated control commands based on the above decisions to drive the projection device to perform a coordinated response.

[0051] Among them, the characteristic value of curtain deformation It can be calculated in the following way:

[0052]

[0053] in, This refers to the total number of measurement areas (i.e., the ToF sensor divides the screen into multiple measurement areas, such as 8×8, totaling 64 areas). For the region number, For the first Measured distance values ​​for each area This represents the average distance across all regions.

[0054] Optionally, in some embodiments, the vibration energy characteristic value is:

[0055] in, The characteristic value of vibrational energy. This represents the number of sampling points within the sampling window, which can be 100ms. The sampling point number, For the first X-axis acceleration values ​​at each sampling point For the first Y-axis acceleration values ​​at each sampling point For the first The squared value of the X-axis acceleration at each sampling point For the first The squared value of the Y-axis acceleration at each sampling point; The characteristic value of the illumination gradient is:

[0056] in, The characteristic value of the illumination gradient, For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For time intervals.

[0057] As one possible implementation, in some embodiments, the execution order of at least one user interaction instruction is arbitrated, including: determining driver state parameters and vehicle speed parameters based on vehicle state data; calculating a safety risk coefficient based on the driver state parameters and vehicle speed parameters; and, in response to the safety risk coefficient being greater than a preset risk threshold, determining a driver safety instruction from at least one user interaction instruction, setting the priority of the driver safety instruction to the highest, and interrupting the current entertainment task to prioritize the execution of the driver safety instruction.

[0058] It is understood that, in this embodiment, driver state parameters refer to information characterizing the driver's current driving state, such as the driver's gaze direction and hand position. The safety risk coefficient is used to assess the safety risk level of the current driving environment. The preset risk threshold is a pre-set risk coefficient threshold value used to determine whether the current risk has reached a level requiring priority handling of driver safety instructions. Driver safety instructions refer to user interaction instructions originating from the driver and involving driving safety or vehicle control.

[0059] Specifically, during video playback, the system simultaneously detected two user interaction commands: one was a "volume up" gesture made by a rear passenger, and the other was a voice command from the driver to "navigate to the gas station" (the sound source was confirmed to originate from the driver's seat). At this point, it is necessary to arbitrate the execution order of these multiple user interaction commands. For example... Figure 4 As shown, when arbitrating the execution order of multiple user interaction commands, the system can first receive multimodal inputs such as voice, gestures, and touch, while simultaneously acquiring real-time environmental parameters, including vehicle speed, driver's line of sight, and hand position. Based on these parameters, the system quantitatively calculates the current safety risk coefficient. When the safety risk coefficient exceeds a preset risk threshold (e.g., 0.6), and the system identifies driver commands containing safety keywords (e.g., "brake," "danger") or involving navigation operations from user interaction instructions, the system sets the priority of the driver command to the highest level, immediately interrupts the current entertainment task (e.g., pauses the playback of the in-vehicle entertainment screen) to prioritize the execution of the command, such as planning a route to a gas station. For passenger commands or other non-safety commands, the system dynamically weights them according to their command type and waiting time, and stores them in a queue to be executed in sequence, thereby achieving dynamic arbitration of multiple commands based on real-time risk.

[0060] Furthermore, if the safety risk coefficient exceeds a preset risk threshold (e.g., 0.6) and multiple driver safety commands are identified simultaneously, the system can first sort all driver safety commands by their urgency level. The urgency level can be determined based on the type of safety keywords contained in the command, the current safety risk coefficient, and the vehicle's real-time status. For example, the "brake" command has a higher urgency level than the "navigate to gas station" command. Then, commands are executed sequentially from highest to lowest urgency level: the highest urgency level command immediately interrupts the current task and is executed; after execution, the system automatically executes the next highest urgency level driver safety commands in sequence until all driver safety commands have been processed. After all driver safety commands have been executed, the system resumes the previously interrupted entertainment task or begins processing other non-safety commands in the queue.

[0061] Furthermore, if the safety risk coefficient is less than or equal to the preset risk threshold, it indicates that the current driving environment is relatively safe (e.g., the vehicle is traveling at low speed or parked). In this case, when both driver safety instructions and other non-safety instructions exist, the system handles the situation according to the following rules: Driver safety instructions still have high priority, but instead of immediate interruption and preemption, a non-interrupted delayed execution strategy is adopted. That is, the system can mark the driver safety instruction as a "high-priority instruction to be executed," and execute it immediately after the current entertainment task reaches a natural breakpoint (e.g., the end of a movie chapter or the end of a song), or after a very short delay (e.g., within 1-2 seconds), to avoid abruptly interrupting the entertainment experience. Other non-safety instructions (including driver entertainment instructions and passenger entertainment instructions) are uniformly stored in the execution queue and scheduled for execution in a first-in, first-out order. After the driver safety instruction is executed, the system automatically resumes the entertainment task or continues scheduling non-safety instructions in the queue.

[0062] For example, when the vehicle is traveling at low speed, the driver issues a safety instruction to "navigate to a gas station," while a passenger issues an entertainment instruction to "increase the volume." Because the current safety risk is low, the system will not immediately interrupt the movie playing. Instead, it will execute the navigation instruction after the current movie segment ends, resume movie playback after the navigation instruction is completed, and then process the "increase the volume" instruction. This ensures timely response to safety instructions while avoiding excessive interruption to the entertainment experience.

[0063] In high-risk scenarios, when a driver's command involves navigation, the system can immediately grant the command the highest priority and then execute the following actions: pause the in-vehicle entertainment screen playback; execute the navigation command and plan a route to the gas station; reply via voice, "Route to the gas station 2 kilometers away has been planned for you"; and automatically resume entertainment screen playback after the navigation prompt ends. During this process, the "volume up" gesture command from rear passengers is temporarily stored in the execution queue and will be automatically executed after the navigation command is completed and entertainment playback resumes.

[0064] Therefore, through a dynamic arbitration engine based on real-time environmental perception, the driver's commands can be accurately identified and prioritized, fundamentally eliminating the safety risks caused by the operation of the entertainment system.

[0065] As one possible approach, in some embodiments, the safety warning information and entertainment content are rendered together and then output to a retractable screen, including: based on collaborative control instructions, combining the safety warning information and entertainment content into layers on a preset display control layer to obtain a combined image; and projecting the combined image onto the retractable screen through a projection device.

[0066] It is understood that, in this embodiment, safety warning information refers to prompts related to driving safety, such as navigation turn prompts, low fuel reminders, and collision warnings. Entertainment content refers to media content for passengers to watch or enjoy, such as movies, videos, and game screens. The preset display control layer is the functional layer in the system responsible for image composition and output (e.g., Figure 2 The display control layer shown is used to receive and process the image data to be displayed.

[0067] Specifically, when outputting the blended safety warning information and entertainment content to a retractable screen, the system performs the following operations on the preset display control layer based on collaborative control commands: First, the system can maintain the normal playback of the entertainment content, rendering it as the underlying image. Simultaneously, the system generates safety warning information (such as navigation turn arrows, low fuel icons, and road condition warnings) as an independent layer. This layer does not cover or replace the entertainment content but exists as an overlay layer. Second, the system can semi-transparent or edge-position the safety warning information layer, overlaying it onto specific areas of the entertainment content image (such as the upper left corner, bottom edge, or side of the image) to avoid obstructing the core viewing area. For safety warning information with a high degree of urgency (such as collision warnings), the system can highlight it with a bright color or flashing effect, but still present it in a non-pop-up, non-interrupted manner. Then, the system can perform real-time pixel-level compositing of the entertainment content layer and the safety warning information layer on the display control layer, forming a blended image that simultaneously contains both types of information. During the compositing process, the system dynamically adjusts the transparency, size, and position of the overlay layer based on the urgency level of the safety warning information to balance the prominence of the safety alerts with the immersive entertainment experience. Finally, the system projects the composited image onto a retractable screen using a projection device. While watching, users can simultaneously receive entertainment content and safety warning information, perceiving driving safety alerts without pausing or interrupting playback, thus maintaining the continuity of the entertainment experience while ensuring driving safety.

[0068] Thus, by utilizing hardware-level rendering, a seamless integration of security and entertainment is achieved. The entire system responds quickly and operates stably, providing users with an intelligent in-vehicle entertainment experience.

[0069] According to the in-vehicle entertainment control method proposed in this application, the method acquires vehicle status data and environmental perception data; responds to at least one user interaction command, determines the current working scenario of the vehicle based on the vehicle status data, environmental perception data, and at least one user interaction command; generates a collaborative control command based on the current working scenario, and adjusts the display parameters of the projection device in a coordinated manner based on the collaborative control command, or arbitrates the execution order of at least one user interaction command, or merges and renders safety warning information with entertainment content and outputs it to a retractable screen. Thus, by identifying the scenario through multi-source data fusion and generating collaborative control commands, the method achieves projection linkage adjustment, dynamic command arbitration, and safe entertainment fusion rendering, solving the problem that in-vehicle entertainment systems cannot simultaneously maintain the stability of large-size projection image quality, the safety of multi-user interaction, and the continuity of the entertainment experience in dynamic driving environments. This achieves intelligent coordination and dynamic balance between driving safety and an immersive viewing experience.

[0070] Next, the in-vehicle entertainment control device proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0071] Figure 5 This is a block diagram of an in-vehicle entertainment control device according to an embodiment of this application.

[0072] like Figure 5 As shown, the in-vehicle entertainment control device 10 includes: an acquisition module 100, a determination module 200, and a control module 300.

[0073] Among them, the acquisition module 100 is used to acquire vehicle status data and environmental perception data; The determination module 200 is used to determine the current working scenario of the vehicle based on vehicle status data, environmental perception data and at least one user interaction command in response to at least one user interaction command. The control module 300 is used to generate collaborative control instructions according to the current working scenario, and based on the collaborative control instructions, to adjust the display parameters of the projection device in a coordinated manner, or to arbitrate the execution order of at least one user interaction instruction, or to merge and render safety warning information with entertainment content and output it to the retractable screen.

[0074] Optionally, in some embodiments, the display parameters of the projection device are adjusted in a coordinated manner, and the control module 300 is specifically used for: Vibration energy characteristic values ​​are determined based on vehicle status data, and illumination gradient characteristic values ​​are determined based on environmental perception data. Based on the vibration energy characteristic value and the illumination gradient characteristic value, the fusion control parameters are calculated; In response to the fusion control parameters exceeding a preset threshold, anti-shake commands and brightness adjustment commands are generated, and the projection device is controlled to perform a linkage response based on the anti-shake commands and brightness adjustment commands.

[0075] Optionally, in some embodiments, the vibration energy characteristic value is:

[0076] in, The characteristic value of vibrational energy. This represents the number of sampling points within the sampling window. The sampling point number, For the first X-axis acceleration values ​​at each sampling point For the first Y-axis acceleration values ​​at each sampling point For the first The squared value of the X-axis acceleration at each sampling point For the first The squared value of the Y-axis acceleration at each sampling point; The characteristic value of the illumination gradient is:

[0077] in, The characteristic value of the illumination gradient, For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For time intervals.

[0078] Optionally, in some embodiments, the execution order of at least one user interaction instruction is arbitrated, and the control module 300 is specifically used for: Based on vehicle status data, determine driver status parameters and vehicle speed parameters; Calculate the safety risk coefficient based on driver status parameters and vehicle speed parameters; In response to a safety risk coefficient exceeding a preset risk threshold, a driver safety instruction is determined from at least one user interaction instruction, and the driver safety instruction is set to the highest priority. The current entertainment task is then interrupted to prioritize the execution of the driver safety instruction.

[0079] Optionally, in some embodiments, the safety warning information is merged and rendered with entertainment content before being output to a retractable screen. The control module 300 is specifically used for: Based on collaborative control commands, safety warning information and entertainment content are combined in a preset display control layer to obtain a composite image. The merged image is projected onto a retractable screen using a projection device.

[0080] It should be noted that the foregoing explanation of the embodiment of the in-vehicle entertainment control method also applies to the in-vehicle entertainment control device of this embodiment, and will not be repeated here.

[0081] According to the vehicle entertainment control device proposed in this application, the device acquires vehicle status data and environmental perception data; responds to at least one user interaction command, determines the current working scenario of the vehicle based on the vehicle status data, environmental perception data, and at least one user interaction command; generates collaborative control commands based on the current working scenario, and adjusts the display parameters of the projection device in a coordinated manner based on the collaborative control commands, or arbitrates the execution order of at least one user interaction command, or merges and renders safety warning information with entertainment content and outputs it to a retractable screen. Thus, by using multi-source data fusion to identify the scenario and generate collaborative control commands, the device achieves projection linkage adjustment, dynamic command arbitration, and safe entertainment fusion rendering, solving the problem that vehicle entertainment systems cannot simultaneously maintain the stability of large-size projection image quality, the safety of multi-user interaction, and the continuity of the entertainment experience in dynamic driving environments. This achieves intelligent coordination and dynamic balance between driving safety and an immersive viewing experience.

[0082] Figure 6 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include: The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0083] When the processor 602 executes the program, it implements the in-vehicle entertainment control method provided in the above embodiments.

[0084] Furthermore, the vehicle also includes: Communication interface 603 is used for communication between memory 601 and processor 602.

[0085] The memory 601 is used to store computer programs that can run on the processor 602.

[0086] The memory 601 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0087] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0088] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0089] The processor 602 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0090] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described in-vehicle entertainment control method.

[0091] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described in-vehicle entertainment control method.

[0092] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0093] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0094] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for controlling in-vehicle entertainment, characterized in that, Includes the following steps: Acquire vehicle status data and environmental perception data; In response to at least one user interaction command, the current working scenario of the vehicle is determined based on the vehicle status data, the environmental perception data, and the at least one user interaction command. Based on the current working scenario, generate collaborative control instructions, and based on the collaborative control instructions, adjust the display parameters of the projection device in a coordinated manner, or arbitrate the execution order of the at least one user interaction instruction, or merge and render safety warning information with entertainment content and output it to a retractable screen.

2. The method according to claim 1, characterized in that, The linkage adjustment of the display parameters of the projection device includes: Vibration energy characteristic values ​​are determined based on the vehicle state data, and illumination gradient characteristic values ​​are determined based on the environmental perception data. Based on the vibration energy characteristic value and the illumination gradient characteristic value, the fusion control parameters are calculated; In response to the fusion control parameter being greater than a preset threshold, an anti-shake command and a brightness adjustment command are generated, and the projection device is controlled to perform a linkage response based on the anti-shake command and the brightness adjustment command.

3. The method according to claim 2, characterized in that, The characteristic value of the vibration energy is: in, The characteristic value of the vibration energy, This represents the number of sampling points within the sampling window. The sampling point number, For the first X-axis acceleration values ​​at each sampling point For the first Y-axis acceleration values ​​at each sampling point For the first The squared value of the X-axis acceleration at each sampling point For the first The squared value of the Y-axis acceleration at each sampling point; The characteristic value of the illumination gradient is: in, The illumination gradient feature value is... For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For at any time The ambient light illuminance value is obtained by measuring the ambient light sensor. For time intervals.

4. The method according to claim 1, characterized in that, The arbitration of the execution order of the at least one user interaction instruction includes: Based on the vehicle status data, determine the driver status parameters and vehicle speed parameters; Calculate the safety risk coefficient based on the driver status parameters and the vehicle speed parameters; In response to the safety risk coefficient being greater than a preset risk threshold, a driver safety instruction is determined from the at least one user interaction instruction, and the priority of the driver safety instruction is set to the highest. The current entertainment task is interrupted to prioritize the execution of the driver safety instruction.

5. The method according to claim 1, characterized in that, The process of fusing and rendering safety warning information with entertainment content and then outputting it to a retractable screen includes: Based on the collaborative control command, the safety warning information and the entertainment content are combined in a preset display control layer to obtain a composite image; The fused image is projected onto the retractable screen via the projection device.

6. A vehicle entertainment control device, characterized in that, Includes the following steps: The acquisition module is used to acquire vehicle status data and environmental perception data; A determination module is used to determine the current working scenario of the vehicle based on the vehicle status data, the environmental perception data, and the at least one user interaction command in response to at least one user interaction command. The control module is used to generate collaborative control instructions based on the current working scenario, and based on the collaborative control instructions, to adjust the display parameters of the projection device in a coordinated manner, or to arbitrate the execution order of the at least one user interaction instruction, or to merge and render safety warning information with entertainment content and then output it to the retractable screen.

7. The apparatus according to claim 6, characterized in that, The control module, which adjusts the display parameters of the projection device in a coordinated manner, is specifically used for: Vibration energy characteristic values ​​are determined based on the vehicle state data, and illumination gradient characteristic values ​​are determined based on the environmental perception data. Based on the vibration energy characteristic value and the illumination gradient characteristic value, the fusion control parameters are calculated; In response to the fusion control parameter being greater than a preset threshold, an anti-shake command and a brightness adjustment command are generated, and the projection device is controlled to perform a linkage response based on the anti-shake command and the brightness adjustment command.

8. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the in-vehicle entertainment control method as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the in-vehicle entertainment control method as described in any one of claims 1-5.

10. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, is used to implement the in-vehicle entertainment control method according to any one of claims 1-5.

Images