A vehicle control method and a vehicle

Abstract

The application discloses a vehicle control method and a vehicle. The vehicle comprises a plurality of vehicle screens. The method comprises the following steps: acquiring spatial pointing information and gesture operation information of a plurality of passengers for the same vehicle screen, and acquiring passenger seat information corresponding to each passenger; determining a screen interaction weight value of each passenger according to the spatial pointing information, the gesture operation information and the passenger seat information of each passenger; determining operation right attribution information of the vehicle screen according to the spatial pointing information, the passenger seat information and the screen interaction weight value of each passenger, and then determining an operation sequence of each passenger to the vehicle screen; generating corresponding vehicle control instructions through the vehicle screen according to the spatial pointing information and the gesture operation information of each passenger in the operation sequence; and controlling the vehicle according to the vehicle control instructions. The application can determine the operation sequence of each passenger to the vehicle screen, and then realize stable control of the vehicle by each passenger, and ensure the response speed of vehicle control and user experience.

Description

Technical Field

[0001] This application belongs to the field of vehicle control, specifically relating to a vehicle control method and a vehicle. Background Technology

[0002] Currently, gesture-based interactive technologies are being gradually applied in smart cockpits. By detecting the user's gaze and finger direction, the system can determine the virtual button selected by the user on the in-vehicle screen, and then combine this with gestures to achieve a "what you see is what you point" interactive experience in the smart cockpit.

[0003] However, in actual use, existing technologies do not take into account real-world scenarios where multiple occupants, such as the driver, front passenger, and rear passengers, simultaneously issue remote interaction commands to the same in-vehicle screen. In such scenarios, it is difficult to determine the order of operation of each occupant on the in-vehicle screen, which in turn makes it difficult to achieve stable control of the vehicle by each occupant, affecting the response speed of vehicle operation and user experience. Summary of the Invention

[0004] The purpose of this application is to provide a vehicle control method and a vehicle. This solves the problem of determining the order in which multiple occupants interact with the same in-vehicle screen when multiple occupants simultaneously issue remote control commands.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a vehicle control method, wherein the vehicle includes a plurality of in-vehicle screens, the method comprising: Acquire spatial pointing information and gesture operation information sent by several occupants to the same vehicle screen, and acquire the occupant seat information corresponding to each occupant; For any of the occupants, a pointing duration feature value is determined based on the spatial pointing information, a gesture movement speed feature value is determined based on the gesture operation information, and an occupant role feature value is determined based on the occupant seating information. The pointing duration feature value, the gesture movement speed feature value, and the occupant role feature value are then weighted and summed to obtain the occupant's screen interaction weight value. Based on the spatial orientation information and seating information of each occupant, determine the type of interaction conflict between the occupants; The operation rights attribution information of the in-vehicle screen is determined based on the interaction conflict type and the screen interaction weight value of each occupant. The order in which each passenger operates the in-vehicle screen is determined based on the information regarding the ownership of operating rights. According to the operation sequence, corresponding vehicle control commands are generated through the vehicle screen based on the spatial orientation information and gesture operation information of each passenger. The vehicle is controlled according to the vehicle control command.

[0006] Optionally, the same vehicle screen can be used as the first vehicle screen, and the other vehicle screens among the plurality of vehicle screens other than the first vehicle screen can be used as the second vehicle screen. The step of determining the type of interaction conflict between occupants based on their spatial orientation information and seating information includes: Obtain the number of passengers; If no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is equal to a preset first number, then the interaction conflict type between the occupants is determined to be the first conflict type. If no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is greater than the first number, then the interaction conflict type between the occupants is determined to be the second conflict type. If it is detected that the spatial orientation information of any occupant has been transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the third conflict type. If it is detected that the spatial orientation information of multiple occupants has been transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the fourth conflict type.

[0007] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the first conflict type, then obtain the weight peak value and weight valley value from the screen interaction weight values ​​corresponding to each passenger. If the difference between the weight peak value and the weight valley value is greater than a preset weight difference threshold, then first operation right attribution information is generated; the first operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the weight peak value.

[0008] Optionally, the method further includes: If the difference between the weight peak value and the weight valley value is less than or equal to the weight difference threshold, then the spatial pointing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger. The spatial distance between the spatial pointing points of each passenger is determined based on the spatial pointing point of each passenger on the first vehicle screen. Based on the spatial distance and the screen interaction weight value of each occupant, the interaction area of ​​each occupant in the first vehicle screen is determined; A second operation right attribution information is generated based on the interaction area of ​​each passenger on the first vehicle screen; the second operation right attribution information is used to indicate that each passenger whose interaction conflict type is the first conflict type has operation rights in their respective corresponding interaction area.

[0009] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the second conflict type, then the spatial pointing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger. The spatial distance between the spatial pointing points of each passenger is determined based on the spatial pointing point of each passenger on the first vehicle screen. Based on the spatial distance and the screen interaction weight value of each occupant, the interaction area of ​​each occupant in the first vehicle screen is determined; A third operation right attribution information is generated based on the interaction area of ​​each occupant on the first vehicle screen; the third operation right attribution information is used to indicate that each occupant whose interaction conflict type is the second conflict type has operation rights in their respective corresponding interaction areas.

[0010] Optionally, the step of determining the interaction area of ​​each occupant in the first vehicle screen based on the spatial distance and the screen interaction weight value of each occupant includes: If the spatial distance between the spatial pointing points of each occupant is greater than a preset spatial distance threshold, then the first vehicle screen is divided into non-overlapping interactive areas centered on the spatial pointing points of each occupant, which serve as the interactive areas for each occupant on the first vehicle screen. If the spatial distance between the spatial pointing landing points of at least two occupants is less than or equal to the spatial distance threshold, then the screen division ratio is determined according to the screen interaction weight value of each occupant, and the first vehicle screen is divided into multiple interaction areas according to the screen division ratio, which serve as the interaction areas for each occupant on the first vehicle screen.

[0011] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the third conflict type, then the spatial orientation transfer time of the occupant from the second vehicle screen to the first vehicle screen is determined according to the spatial orientation information. The occupant's screen interaction weight value is adjusted according to the spatial direction transfer duration to obtain the cross-screen interaction weight value; The cross-screen interaction weight value is compared with the screen interaction weight value of other occupants whose spatial orientation information is the first vehicle screen. The cross-screen interaction weight value or screen interaction weight value that is greater than the other interaction weight value is taken as the first interaction weight value. A fourth operation right attribution information is generated based on the first interaction weight value; the fourth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the first interaction weight value.

[0012] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the fourth conflict type, then the sum of the screen interaction weight values ​​of multiple occupants whose spatial direction information is transferred from the second vehicle screen to the first vehicle screen is determined based on the spatial direction information, and is used as the team interaction weight value. The team interaction weight value is compared with the screen interaction weight value of other occupants whose spatial orientation information is the first vehicle screen. The team interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​is taken as the second interaction weight value. A fifth operation right attribution information is generated based on the second interaction weight value; the fifth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the second interaction weight value.

[0013] Optionally, the method further includes: If the second interaction weight value is the team interaction weight value, then the operation time ratio of each passenger is determined according to the ratio of the individual screen interaction weight value of each passenger to the team interaction weight value. According to the screen interaction weight values ​​from high to low, within the preset operation cycle, the operation time periods are allocated to the multiple passengers in sequence according to the operation duration ratio. A sixth operation right attribution information is generated based on the operation time period of each passenger; the sixth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the corresponding passenger for each operation time period within the operation cycle.

[0014] Optionally, after generating operation right attribution information based on the interaction area of ​​each occupant on the first vehicle screen, the method further includes: The operation type is determined based on the gesture operation information; the operation type includes screen area operation and general cockpit operation. If the operation type is the screen area operation, then the screen area operation is responded to in the interaction area corresponding to the occupant who issued the gesture operation information. If the operation type is a general cockpit operation, then the general cockpit operation of the occupant with the highest screen interaction weight value is responded to, and the general cockpit operations of the remaining occupants are temporarily stored.

[0015] Optionally, the method further includes: If it is detected that any occupant's spatial pointing information has been moved out of the first vehicle screen for a duration exceeding a first duration threshold, or if no gesture operation information is issued within a second duration threshold, then the occupant's operation rights on the first vehicle screen are released, and the remaining occupants are allowed to redistribute the interaction area of ​​the first vehicle screen.

[0016] Secondly, embodiments of this application provide a vehicle including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method described above.

[0017] The embodiments of this application have the following advantages: This application acquires spatial directional information and gesture operation information from several occupants targeting the same in-vehicle screen, and also obtains the corresponding seat information for each occupant. By accurately identifying the spatial directional information and gesture operation information from several occupants targeting the same in-vehicle screen, the acquisition of occupant seat information provides a basis for subsequently distinguishing the priority of roles such as driver, front passenger, and rear passenger. For any given occupant, this application determines a directional duration feature value based on spatial directional information, a gesture speed feature value based on gesture operation information, and an occupant role feature value based on seat information. Furthermore, it performs a weighted sum of the directional duration feature value, gesture speed feature value, and occupant role feature value to obtain the occupant's screen interaction weight value. This comprehensive multi-dimensional information determination of screen interaction weight values ​​better reflects actual in-vehicle scenarios. This application determines the type of interaction conflict between occupants based on their spatial orientation and seating information. It then determines the ownership of operation rights for the in-vehicle screen based on the conflict type and each occupant's screen interaction weight value. Based on this ownership, it determines the order in which each occupant operates on the in-vehicle screen. Following this order, it generates corresponding vehicle control commands through the in-vehicle screen based on each occupant's spatial orientation and gesture operation information. By controlling the vehicle according to these commands, the application accurately determines the order in which each occupant operates on the in-vehicle screen, thereby achieving stable vehicle control and ensuring fast response times and a positive user experience. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0019] Figure 1This is a flowchart illustrating the steps of a vehicle control method according to an embodiment of this application; Figure 2 This is a logical schematic diagram of a vehicle control method provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to enable readers to better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and updates based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0021] Existing air-to-air interaction technologies based on eye tracking and gesture recognition generally only support single-person interaction scenarios, assuming that only one user can interact with the vehicle system at any given time, without considering the real multi-passenger environment where the driver, front passenger, and rear passengers can issue operation commands simultaneously.

[0022] Secondly, existing technologies address conflicts between modalities, such as prioritizing responses between gesture and voice commands, rather than conflicts between individuals. Conflicts between individuals refer to the situation where multiple occupants simultaneously want to control the same in-vehicle screen, and the vehicle must prioritize which occupant's command. This is entirely different from modal conflicts.

[0023] Furthermore, existing technologies lack arbitration mechanisms for multi-person interaction scenarios. They do not disclose a weighting calculation method that comprehensively weights factors such as gaze duration, gesture speed, and passenger role priority, nor do they disclose arbitration processing methods suitable for simultaneous multi-person interaction, such as dynamic partitioning, weighted ratio partitioning, and time slice rotation.

[0024] Furthermore, existing technologies for gesture recognition remain at the semantic level, only able to determine the type of gesture. They fail to use the speed and trajectory changes of gestures to quantify the urgency of the occupant's actions, making it impossible to distinguish between calm and emergency operations and difficult to make reasonable priorities among multiple people's instructions.

[0025] Finally, none of the existing technologies address scenarios where three or more passengers simultaneously engage in eye contact and gesture interaction, and none provide a detection mechanism or arbitration solution for conflicts involving three or more passengers.

[0026] Therefore, this application provides a vehicle control method and a vehicle that can determine the order of operation of each passenger on the vehicle screen when accurately identifying spatial pointing information and gesture operation information sent by several passengers to the same vehicle screen, thereby enabling stable control of the vehicle by each passenger and ensuring the response speed of vehicle operation and user experience.

[0027] Reference Figure 1 The diagram shows a flowchart of the steps of a vehicle control method provided in an embodiment of this application.

[0028] In this embodiment, the vehicle includes several in-vehicle screens. Specifically, these in-vehicle screens include, but are not limited to: a central control screen, a passenger entertainment screen, a rear entertainment screen, a panoramic sunroof screen, headrest screens, an AR-HUD (Augmented Reality Head-Up Display) virtual projection interface, and interactive projection areas.

[0029] Vehicle control methods may specifically include the following steps: Step 101: Obtain spatial direction information and gesture operation information sent by several occupants to the same in-vehicle screen, and obtain the occupant seat information corresponding to each occupant.

[0030] In this embodiment of the application, if it is detected that several occupants send spatial pointing information and gesture operation information to the same vehicle screen, these spatial pointing information and gesture operation information can be obtained respectively, and the occupant seat information corresponding to each occupant can be obtained.

[0031] Spatial orientation information refers to the spatial orientation signal of an occupant pointing to a target area. This information can include one or more of the following: eye gaze direction, head orientation, and hand pointing; it can also be a fusion of signals from these sources. Spatial orientation information is used to determine which screen the occupant is focusing on and the specific location within that screen.

[0032] Gesture control information refers to the operation commands issued by the occupant through hand movements, which may include information such as the three-dimensional spatial motion trajectory and speed of key hand points. Gesture control information is used to identify the operation that the occupant wants to perform.

[0033] Occupant seating information refers to the occupant's seat position within the vehicle cabin, which may include the driver's seat, front passenger seat, and rear seats. Occupant seating information is used to determine the occupant's role priority.

[0034] In practice, in-vehicle cameras can be used to collect spatial pointing information in real time, such as the direction of each occupant's gaze, head orientation, or hand pointing, to determine which in-vehicle screen each occupant's spatial pointing point lands on. When it is detected that at least two occupants' spatial pointing points are simultaneously pointing to the same in-vehicle screen, and their respective hand gestures overlap within a time window, it is determined that several occupants have issued spatial pointing and hand gesture information to the same in-vehicle screen. The time window can be set according to actual needs, for example, it can be set to 1 second.

[0035] When valid spatial pointing information cannot be obtained, such as when the occupant's spatial pointing is obstructed, the occupant is not looking at the screen interface, or the sensor malfunctions, the system automatically reverts to a pure gesture arbitration mode. In pure gesture arbitration mode, the pointing duration feature value is set to zero, and the screen interaction weight value is calculated only based on the gesture movement speed feature value and the occupant role feature value. The operation right is determined by exclusive authorization or weighted time slice rotation.

[0036] Step 102: For any occupant, determine the pointing duration feature value based on spatial pointing information, determine the gesture movement speed feature value based on gesture operation information, determine the occupant role feature value based on occupant seating information, and perform a weighted sum of the pointing duration feature value, gesture movement speed feature value, and occupant role feature value to obtain the occupant's screen interaction weight value.

[0037] In this embodiment, after obtaining spatial pointing information and gesture operation information from multiple occupants targeting the same in-vehicle screen, the interaction intent of each occupant can be quantitatively evaluated to determine the screen interaction weight value of each occupant. The screen interaction weight value can represent the occupant's overall competitiveness in the current interaction conflict; the higher the screen interaction weight value, the stronger the occupant's overall competitiveness in the current interaction conflict.

[0038] In this embodiment of the application, for each occupant participating in the interaction, the duration of the occupant's gaze or pointing at the target vehicle screen can be calculated using the collected spatial pointing information, and the duration can be mapped to a normalized pointing duration feature value.

[0039] The pointing duration feature value refers to a dimensionless value obtained by normalizing the duration of an occupant's gaze or pointing at the target screen. The boundary conditions of the pointing duration feature value have a clear physical meaning. When the occupant's spatial pointing information is not directed at the target interaction area, or when the pointing duration is extremely short, close to zero seconds, the pointing duration feature value is zero. In this case, it is determined to be an unintentional glance or lack of attention, such as when the driver only glances quickly at the rearview mirror or out the window without stopping at the central control screen.

[0040] When the duration of a passenger's spatial pointing reaches or exceeds a preset saturation threshold, such as when the passenger stares at the screen for more than 2.8 seconds, the characteristic value of the pointing duration is capped at 1. At this point, it is determined to be a strong interaction intention, and even if the actual staring time continues to increase, the characteristic value will no longer increase accordingly.

[0041] This is to accurately quantify the intensity of intent: the feature value is set to zero for extremely short pointing times, which can prevent frequent interaction triggers due to brief glances; within the saturation threshold, the feature value increases linearly with the duration of gaze, and the longer the gaze, the stronger the competitiveness, reflecting fair competition; after exceeding the saturation threshold, the feature value is capped, which prevents a single passenger from forming a monopoly by gazing continuously for a long time. At this time, it is necessary to combine the gesture movement speed feature value to break the deadlock.

[0042] In practical implementation, the following formula can be used to calculate the characteristic value pointing to the duration:

[0043] in, This represents the characteristic value of the duration of the pointer of the i-th passenger. This indicates the duration for which the occupant gazes at or points at the target in-vehicle screen, in seconds. This represents the minimum effective threshold for triggering interaction. Below this time, the system considers it merely a scurrying glance rather than a gaze, and assigns a weight of 0. You can set it according to your actual situation, for example, set it to 0.02 seconds. This represents the first preset saturation threshold. Beyond this time, gaze duration will no longer increase the weight, and the weight will be capped at 1. You can set it yourself according to the actual situation, for example, set it to 2.8 seconds.

[0044] It should be noted that, due to the duration feature value In the calculation formula, the numerator and denominator have the same time dimension, such as seconds. Their ratio eliminates the influence of dimensions, thus the duration characteristic value is... It is a dimensionless numerical value.

[0045] In this embodiment of the application, for each occupant participating in the interaction, the instantaneous movement speed of the occupant's hand in three-dimensional space can be calculated using the collected gesture operation information, and the movement speed can be mapped into a normalized gesture movement speed feature value.

[0046] The gesture motion speed feature value refers to the dimensionless value obtained by normalizing the rate of change of displacement of key points of the occupant's hand in three-dimensional space. When the instantaneous motion speed of the occupant's hand in three-dimensional space is 0 meters per second, the gesture motion speed feature value is 0. At this time, the hand is stationary or naturally hanging down, such as the passenger's hand resting on their lap or armrest without moving. When the instantaneous motion speed of the hand reaches or exceeds 0.9 meters per second, the gesture motion speed feature value is capped at 1. At this time, it is judged as a high-priority, strong interaction, and the action is decisive and rapid, such as the driver quickly waving to cut off music or quickly pointing to a target. According to the calculation formula of the gesture motion speed feature value, as long as the occupant's hand makes a movement, even at a very slow speed, it will be assigned a minimum basic feature value of 0.1 to ensure that any conscious gesture can participate in the subsequent weight comparison.

[0047] When the hand movement speed reaches or exceeds the second preset saturation threshold, the feature value is capped at 1. At this time, it is judged as a high-priority, strong interaction, and the action is decisive and rapid, such as the driver quickly waving to cut off the music or quickly pointing to a certain target.

[0048] In this application, the second preset saturation threshold can be set according to the actual situation. For example, it can be set to 0.9 meters per second. As long as the occupant's hand makes a movement, even if the speed is extremely slow, it will be assigned a minimum basic feature value of 0.1, ensuring that any conscious hand gesture can participate in the subsequent weight comparison.

[0049] Among them, the faster the hand movement speed, the higher the gesture movement speed characteristic value, indicating that the occupant's intention to operate is more urgent.

[0050] In practical implementation, the following formula can be used to calculate the characteristic value of gesture movement speed:

[0051] in, This represents the characteristic value of the hand gesture movement speed of the i-th occupant. This indicates the actual instantaneous speed of the occupant's hand gesture, measured in m / s. This represents the reference speed, which can be 1 m / s.

[0052] Here, 0.1 is the preset minimum base score. This means that as long as an action is detected, even at extremely slow speeds, it will be assigned the lowest weight, with no dead zone. The second preset saturation threshold is 1.0, at which point the actual instantaneous speed can correspond to 0.9 m / s. Speeds higher than this are considered extremely fast operations, and the gesture movement speed feature value is capped. min() represents the minimum value function. When the parentheses contain multiple values ​​or expressions, the function outputs the result with the smallest value. For example, min(0.1+0.8, 1.0) means that 0.1 and 0.8 are added together to get 0.9, then 0.9 and 1.0 are compared, and the smaller value of 0.9 is taken as the output. When the calculation result within the parentheses exceeds the upper limit, the min function truncates it to the upper limit value, achieving numerical capping.

[0053] It should be noted that, due to the characteristic value of gesture movement speed In the calculation formula, the actual instantaneous speed of the occupant's gesture Compared with reference speed They have the same velocity dimension, such as m / s. Therefore, the actual instantaneous velocity Compared with reference speed The ratio is a dimensionless pure numerical value. The constants 0.1 and 1.0 are themselves preset dimensionless thresholds. The entire expression only involves the addition and minimum value operation of dimensionless quantities, so the final output is the characteristic value of the gesture movement speed. It is a dimensionless numerical value.

[0054] In practical implementation, the actual instantaneous speed of a gesture can be calculated in the following way: First, the in-vehicle camera can capture the continuous position coordinates (x, y, z) of key points of the occupant's wrist joint in three-dimensional space. △t is the time interval between two adjacent frames captured by the in-vehicle camera, for example, 0.03 seconds.

[0055] The displacement of key points of the wrist joint between two adjacent frames is calculated using the following formula:

[0056] Where d represents the displacement of the key points of the wrist joint between two adjacent frames. This represents the coordinates of the wrist in three-dimensional space at time t. It can be (x) t , y t , z t ). This represents the coordinates of the wrist in three-dimensional space at time t-1. It can be (x) t-1 , y t-1 , z t-1The `Sqrt()` function represents the square root function. The parentheses contain a numerical value or expression, and the function outputs the arithmetic square root of that value. For example, when calculating the displacement distance of a wrist keypoint in three-dimensional space, the displacement distance `d` is the square root of the sum of the squares of the displacement differences along the three coordinate axes between two adjacent frames, yielding the Euclidean distance between the two points.

[0057] In other words, the displacement distance d is and The Euclidean distance between two points.

[0058] Then, the instantaneous velocity V can be calculated based on the displacement distance d and the time interval Δt. actual =d / Δt. Instantaneous velocity V actual It equals the displacement distance d divided by the time interval Δt.

[0059] In practical implementation, in order to reduce data fluctuations caused by natural hand tremors, the instantaneous motion speed can be smoothed. The average or maximum instantaneous speed of the three most recent consecutive frames can be taken as the actual instantaneous speed for feature value calculation.

[0060] In this embodiment, for each occupant participating in the interaction, a preset role characteristic value can be determined based on the occupant's seat position in the vehicle cabin. Different seats correspond to different role characteristic values, used to reflect differences in the occupant's priority.

[0061] Occupant role characteristic values ​​can refer to fixed values ​​preset based on the occupant's seat position. These occupant role characteristic values ​​are dimensionless pure numerical values ​​and can be set according to actual conditions; for example, 1.0 for the driver's seat, 0.7 for the front passenger seat, and 0.4 for the rear seats. Occupant role characteristic values ​​are used to reflect the different priority levels of occupants in different seats during weighted calculations.

[0062] In a practical implementation, the crew role characteristic value can be represented as P. role The crew member role characteristic values ​​can be found in Table 1.

[0063] Table 1. Crew Role Characteristic Value Comparison Table

[0064] In this embodiment of the application, the occupant's pointing duration feature value, gesture movement speed feature value, and occupant role feature value can be assigned corresponding weight coefficients and then added together to obtain a comprehensive screen interaction weight value.

[0065] In practical implementation, screen interaction weight values ​​can be calculated in the following way:

[0066] in, This represents the screen interaction weight value of the i-th passenger. This represents the characteristic value of the duration of the pointer of the i-th passenger. This represents the characteristic value of the hand gesture movement speed of the i-th occupant. This represents the occupant role feature value of the i-th occupant. α represents the weighting coefficient for the duration feature value, which can be 0.4. β represents the weighting coefficient for the gesture speed feature value, which can be 0.4. γ represents the weighting coefficient for the occupant role feature value, which can be 0.2. The weighting coefficients can also be set according to the actual situation.

[0067] It should be noted that in the formula for calculating the screen interaction weight value, since the pointing duration feature value, gesture movement speed feature value, and occupant role feature value are all dimensionless values, and the weight coefficient corresponding to each feature value is also a dimensionless pure value without introducing any physical dimensions, the screen interaction weight value obtained by weighted summation is also a dimensionless pure value, representing the occupant's comprehensive weight score in screen interaction.

[0068] This application addresses the problem of existing technologies' inability to quantify the intensity of multi-occupant interaction intentions by weightedly fusing feature values ​​from three dimensions: spatial pointing duration, gesture movement speed, and occupant role priority. This enables a precise assessment of each occupant's overall interactive competitiveness. It can distinguish between unintentional glances and deliberate stares, calm operations and emergency operations, and the needs of the driver and rear passengers. This provides a comparable quantitative basis for subsequent arbitration of multi-occupant conflicts, avoiding misjudgments and unfairness caused by single-dimensional judgments.

[0069] Step 103: Determine the type of interaction conflict between occupants based on the spatial orientation information and occupant seating information of each occupant.

[0070] In this embodiment of the application, the spatial orientation landing point of each occupant and the occupant's seat information can be used to determine the type of interaction conflict between occupants.

[0071] Interaction conflict types refer to specific categories derived from classifying conflict scenarios where multiple occupants simultaneously vie for control of the same in-vehicle screen. Interaction conflict types can include two-person conflict on the same screen, multiple-person conflict on the same screen, single-person cross-screen conflict, and multiple-person cross-screen conflict, each corresponding to a different method of allocating operating rights.

[0072] Step 104: Determine the operation rights of the in-vehicle screen based on the interaction conflict type and the screen interaction weight value of each occupant.

[0073] In this embodiment of the application, after determining the type of interaction conflict, the operation right attribution information can be generated by combining the screen interaction weight values ​​of each occupant.

[0074] The operation right allocation information is used to indicate the result of operation right allocation for the target vehicle screen. The operation right allocation information can indicate that the operation right is exclusively held by a certain occupant, or that after the screen is divided into multiple interactive areas, each occupant has operation right in the corresponding area, or that multiple occupants across screens take turns having operation right in a time period during the operation cycle.

[0075] Step 105: Determine the order in which each passenger operates the in-vehicle screen based on the information on the ownership of operating rights.

[0076] In this embodiment, the order or parallel rule for each occupant to operate the target vehicle screen can be determined based on the permission allocation result indicated by the operation permission allocation information. The operation order serves as the basis for the execution of subsequent vehicle control commands.

[0077] This application identifies interaction conflict types based on the occupant's spatial orientation and seating information, and generates operation right attribution information by combining screen interaction weight values. This solves the problem that existing technologies cannot distinguish between same-screen conflicts and cross-screen conflicts, or between two-person conflicts and multiple-person conflicts. It enables the determination of multiple operation right allocation methods, ensuring that the occupant's screen operation order can be determined in an orderly manner under different conflict scenarios.

[0078] Step 106: Following the operation sequence, generate corresponding vehicle control commands through the in-vehicle screen based on the spatial orientation information and gesture operation information of each occupant.

[0079] In this embodiment of the application, once the operation sequence of each passenger on the vehicle screen is determined, the gesture operation of each passenger can be responded to sequentially or in sections according to the operation sequence.

[0080] For occupants who have gained control, the specific location of their attention on the in-vehicle screen can be determined based on their spatial orientation information. The specific operation that the occupant wants to perform can be identified based on their gesture operation information, and the gesture operation can be converted into a corresponding vehicle control command through the in-vehicle screen.

[0081] Among them, vehicle control commands refer to electronic commands that control a certain function or device of the vehicle to perform a specific action, such as adjusting the air conditioning temperature, changing the music track, adjusting the navigation destination, opening the sunroof, etc.

[0082] Step 107: Control the vehicle according to the vehicle control command.

[0083] In this embodiment of the application, after a vehicle control command is generated through the in-vehicle screen, the vehicle control command is sent to the relevant execution unit of the vehicle, and the execution unit performs the corresponding vehicle control operation.

[0084] In practical implementation, the execution unit may include the vehicle air conditioning system, audio entertainment system, navigation system, and sunroof control system, etc.

[0085] For example, the air conditioning temperature adjustment command is sent to the air conditioning controller to perform temperature adjustment, the music switching command is sent to the audio system to perform track switching, the navigation command is sent to the navigation module to perform route update, and the sunroof adjustment command is sent to the sunroof control system to perform sunroof opening adjustment.

[0086] This application acquires spatial directional information and gesture operation information from several occupants targeting the same in-vehicle screen, and also obtains the corresponding seat information for each occupant. By accurately identifying the spatial directional information and gesture operation information from several occupants targeting the same in-vehicle screen, the acquisition of occupant seat information provides a basis for subsequently distinguishing the priority of roles such as driver, front passenger, and rear passenger. For any given occupant, this application determines a directional duration feature value based on spatial directional information, a gesture speed feature value based on gesture operation information, and an occupant role feature value based on seat information. Furthermore, it performs a weighted sum of the directional duration feature value, gesture speed feature value, and occupant role feature value to obtain the occupant's screen interaction weight value. This comprehensive multi-dimensional information determination of screen interaction weight values ​​better reflects actual in-vehicle scenarios. This application determines the type of interaction conflict between occupants based on their spatial orientation and seating information. It then determines the ownership of operation rights for the in-vehicle screen based on the conflict type and each occupant's screen interaction weight value. Based on this ownership, it determines the order in which each occupant operates on the in-vehicle screen. Following this order, it generates corresponding vehicle control commands through the in-vehicle screen based on each occupant's spatial orientation and gesture operation information. By controlling the vehicle according to these commands, the application accurately determines the order in which each occupant operates on the in-vehicle screen, thereby achieving stable vehicle control and ensuring fast response times and a positive user experience.

[0087] Optionally, the same in-vehicle screen can be used as the first in-vehicle screen, and other in-vehicle screens other than the first in-vehicle screen can be used as the second in-vehicle screen.

[0088] In this embodiment of the application, among multiple in-vehicle screens, the screen that is simultaneously pointed to by multiple occupants and causes an interaction conflict can be defined as the first in-vehicle screen, and the remaining in-vehicle screens in the vehicle that are not involved in the current conflict can be defined as the second in-vehicle screen.

[0089] In other words, the first in-vehicle screen can refer to the target screen that is simultaneously pointed to by multiple occupants and a struggle for control occurs during multi-occupant interaction conflicts. The second in-vehicle screen can refer to any other in-vehicle screen besides the first one, and is used to determine whether an occupant has engaged in cross-screen transfer behavior.

[0090] The steps for determining the type of interaction conflict between occupants based on their spatial orientation information and seating information include: S1011, Obtain the number of passengers; S1012, if no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is equal to the preset first number, then the interaction conflict type between the occupants is determined to be the first conflict type. S1013, if no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is greater than the first number, then the interaction conflict type between the occupants is determined to be the second conflict type. S1014, if it is detected that the spatial orientation information of any occupant is transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the third conflict type. S1015, if it is detected that the spatial orientation information of multiple occupants has been transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the fourth conflict type.

[0091] In this embodiment of the application, the total number of occupants who send spatial pointing information and gesture operation information to the first vehicle screen can be obtained as the number of occupants.

[0092] In this embodiment of the application, if no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is equal to a preset first number, then the interaction conflict type between the occupants is determined to be the first conflict type.

[0093] Specifically, when all occupants' spatial orientations are initially on the first in-vehicle screen, no occupants have moved from other screens, and the number of occupants involved in the conflict is exactly equal to a preset first number, the current conflict is classified as the first conflict type. Taking a first number of 2 as an example, the first conflict type is a conflict between two people on the same screen.

[0094] In this embodiment of the application, if no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is greater than the first number, then the interaction conflict type between the occupants is determined to be the second conflict type.

[0095] Specifically, when all occupants' spatial orientation is on the first in-vehicle screen from the beginning and no cross-screen transfer occurs, but the number of occupants involved in the conflict exceeds a preset first number, such as two people, the current conflict is judged as the second type of conflict, namely, multi-person conflict on the same screen.

[0096] In this embodiment of the application, if it is detected that the spatial orientation information of any occupant is transferred from the second vehicle screen to the first vehicle screen, the interaction conflict type between the occupants is determined to be the third conflict type.

[0097] Specifically, when it is detected that a passenger's space pointer was originally on the second in-vehicle screen and then moved to the first in-vehicle screen, i.e. a cross-screen transfer behavior has occurred, the current conflict is judged as the third type of conflict, namely a single-person cross-screen conflict.

[0098] In this embodiment of the application, if it is detected that the spatial orientation information of multiple occupants has been transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the fourth conflict type.

[0099] Specifically, when it is detected that the spatial orientation of two or more occupants has been moved from the second in-vehicle screen to the first in-vehicle screen, that is, when a multi-person cross-screen transfer has occurred, the current conflict is determined to be the fourth type of conflict, namely a multi-person cross-screen conflict.

[0100] This application divides the vehicle screen into a first vehicle screen and a second vehicle screen, and combines the number of occupants with cross-screen transfer detection to classify multi-occupant interaction conflicts into four types of interaction conflicts. This solves the problem that existing technologies cannot distinguish the differences in conflict scenarios, and achieves accurate identification of different conflict types. It provides an accurate classification basis for adopting differentiated processing methods for each type in the future.

[0101] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and screen interaction weight value includes: S1021, If ​​the interaction conflict type is the first conflict type, then obtain the weight peak value and weight valley value from the screen interaction weight values ​​corresponding to each crew member. S1022, if the difference between the weight peak value and the weight valley value is greater than the preset weight difference threshold, then the first operation right attribution information is generated; the first operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the weight peak value.

[0102] In this embodiment of the application, when the current interaction conflict is determined to be the first type of conflict, the highest weight peak value and the lowest weight valley value are extracted from the screen interaction weight values ​​of each occupant.

[0103] The peak weight is the maximum value among the screen interaction weights of each passenger, representing the passenger with the stronger overall intent among the competing passengers. The trough weight is the minimum value among the screen interaction weights of each passenger, representing the passenger with the weaker overall intent among the competing passengers.

[0104] When the difference between the weight peak and the weight trough exceeds the preset weight difference threshold, it indicates that the intensity of the combined intent of the two occupants is significantly different, and at this time, the first operation right attribution information is generated.

[0105] The first operation right allocation information is used to indicate that the operation right of the first vehicle screen is fully granted to the occupant corresponding to the weight peak value, and that occupant has exclusive access to the screen. The instructions of the occupant corresponding to the weight valley value will not be responded to temporarily.

[0106] The weight difference threshold refers to a preset threshold used to determine whether the difference in the overall interaction weights between two occupants is sufficiently significant. When the difference exceeds this threshold, an exclusive authorization method is used to allocate operation rights. In the specific implementation, the weight difference threshold can be set according to the actual situation, for example, set to 0.25.

[0107] In the specific implementation, we will take the example of a preset first quantity of 2, where the driver and the front passenger control the central control screen simultaneously.

[0108] Assume that the in-car camera detects the driver's directional information on the central control screen for 2.3 seconds, and then the driver's right hand quickly slides from the steering wheel toward the screen, with an instantaneous hand movement speed of 0.8 meters per second.

[0109] Assuming the in-car camera detects that the front passenger's spatial pointing information falls on the central control screen for 1.8 seconds, and the front passenger slowly raises their right hand from the armrest with an instantaneous hand movement speed of 0.3 meters per second, both passengers' spatial pointing is on the same central control screen, and the start time of their gestures differs by 0.5 seconds, which is less than the preset 1-second time window threshold. At this point, interaction conflict detection can be triggered.

[0110] The screen interaction weight value was calculated for each passenger. According to the above calculation method, the characteristic value of the pointing duration of the driver passenger was (2.3-0.02) / (2.8-0.02)=0.82, the characteristic value of the gesture movement speed of the driver passenger was min(0.1+0.8,1.0)=0.9, and the characteristic value of the passenger role of the driver passenger was confirmed to be 1.0 according to Table 1. The screen interaction weight value of the driver passenger was obtained by weighting and summing the three according to the weight coefficient of the previous example, which is 0.888.

[0111] The characteristic value of the co-passenger's pointing duration is (1.8-0.02) / (2.8-0.02)=0.64, the characteristic value of the co-passenger's gesture movement speed is min(0.1+0.3,1.0)=0.4, the characteristic value of the co-passenger's occupant role is confirmed to be 0.7 according to Table 1, and the screen interaction weight value is 0.556 after weighting and summing according to the weight coefficients of the previous examples.

[0112] In the screen interaction weight values ​​between the driver and front passenger, the peak weight is 0.888, the valley weight is 0.556, and the difference between the two is 0.332, which is greater than the preset weight difference threshold of 0.25. This can generate the first operation right attribution information, indicating that the operation right of the central control screen belongs to the passenger corresponding to the weight peak, i.e., the driver.

[0113] After the driver gains exclusive interaction privileges, a blue light effect appears at the bottom of the central control screen to indicate that the driver is operating the system, and the passenger's gesture commands are temporarily stored and not responded to. During the authorization period, the central control screen only responds to the driver's spatial pointing and gesture commands. After the driver completes the operation and the spatial pointing moves off the central control screen, if no new spatial pointing point is detected after a 1.5-second timer, the driver's operating privileges are released, and the system returns to idle listening state, allowing for the next conflict arbitration.

[0114] In the case of the first conflict type, this application compares the difference between the weight peak value and the weight trough value, and generates the first operation right attribution information when the difference exceeds a preset threshold to grant the occupant with the highest weight exclusive operation right. This solves the problem of unreasonable adjudication when occupants compete for control of the same screen at the same time, and realizes efficient and accurate exclusive authorization in scenarios with significant differences in intent intensity. It avoids frequent competition and misjudgment when weights are close, and ensures the rapid establishment of interaction order.

[0115] Optionally, the method further includes the following steps: S1031, if the difference between the weight peak value and the weight valley value is less than or equal to the weight difference threshold, then the spatial pointing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger. S1032, determine the spatial distance between the spatial pointing points of each occupant based on the spatial pointing point of each occupant on the first vehicle screen; S1033, Based on the spatial distance and the screen interaction weight value of each occupant, determine the interaction area of ​​each occupant in the first vehicle screen; S1034, generate second operation right attribution information based on the interaction area of ​​each occupant on the first vehicle screen; the second operation right attribution information is used to indicate that each occupant with the interaction conflict type of the first conflict type has operation right in their respective corresponding interaction area.

[0116] In this embodiment of the application, if the difference between the weight peak value and the weight valley value is less than or equal to the weight difference threshold, the spatial pointing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger.

[0117] In practice, when the weight difference between two occupants in a conflict on the same screen is small and the conditions for exclusive authorization are not met, the winner-takes-all approach is not adopted. Instead, a partitioning process is initiated. First, based on each occupant's spatial pointing information, their specific gaze position or pointing position on the first in-vehicle screen is determined as the spatial pointing landing point.

[0118] In this embodiment, after obtaining the spatial pointing landing point of each occupant, the spatial distance between each pair of landing points is calculated. The spatial distance can be used to determine whether two occupants are viewing different positions on the screen or converging in the same area. In a specific implementation, the spatial distance can be the Euclidean distance between the spatial pointing landing points of the two occupants.

[0119] In this embodiment, the spatial distance between each pair of landing points and the screen interaction weight value of each occupant can be combined to divide an interactive operation area for each occupant on the first vehicle screen. The interactive area is an independently operable sub-area of ​​the screen allocated to each occupant on the first vehicle screen.

[0120] After the interactive operation area is divided, a second operation right allocation information can be generated. This second operation right allocation information indicates that each occupant with an interaction conflict type of the first conflict type has operation rights within their respective interactive area. This second operation right allocation information can distribute the operation rights of the first in-vehicle screen to each occupant by area, ensuring that each occupant has independent operation permissions within their own interactive area, without interference from others.

[0121] This application solves the problem of not being able to fairly allocate screen control when the weights are close by dividing each crew member into independent interaction areas based on the spatial pointing point and spatial distance and generating second operation right attribution information when the weight difference does not exceed the threshold under the first conflict type. It achieves the effect of a first preset number of crew members operating in parallel on the same screen without interfering with each other, and avoids the experience defects of one party waiting for a long time or the operation intention being completely suppressed due to the small weight difference.

[0122] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and screen interaction weight value includes: S1041, if the interaction conflict type is the second conflict type, then determine the spatial pointing landing point of each occupant on the first vehicle screen according to the spatial pointing information of each occupant. S1042, determine the spatial distance between the spatial pointing points of each occupant based on the spatial pointing point of each occupant on the first vehicle screen; S1043, Based on the spatial distance and the screen interaction weight value of each occupant, determine the interaction area of ​​each occupant in the first vehicle screen; S1044, generate third operation right attribution information based on the interaction area of ​​each occupant on the first vehicle screen; the third operation right attribution information is used to indicate that each occupant with the interaction conflict type of the second conflict type has operation right in their respective corresponding interaction area.

[0123] In this embodiment of the application, if the interaction conflict type is the second conflict type, the spatial pointing landing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger.

[0124] In the specific implementation, taking a preset first quantity of 2 as an example, if the interaction conflict type is the second conflict type, it means that when the number of occupants involved in the conflict exceeds two, the process can directly enter the partitioning process. First, based on the spatial pointing information of each occupant, the specific gaze position or pointing position of each person on the first vehicle screen is determined as the spatial pointing landing point.

[0125] In this embodiment, after obtaining the spatial pointing landing point of each occupant, the spatial distance between each pair of landing points is calculated. The spatial distance can be used to determine whether two occupants are viewing different positions on the screen or converging in the same area. In a specific implementation, the spatial distance can be the Euclidean distance between the spatial pointing landing points of the two occupants.

[0126] In this embodiment, the spatial distance between each pair of landing points and the screen interaction weight value of each occupant can be combined to divide an interactive operation area for each occupant on the first vehicle screen. The interactive area is an independently operable sub-area of ​​the screen allocated to each occupant on the first vehicle screen.

[0127] After the interactive operation area is divided, third operation right allocation information can be generated. This third operation right allocation information indicates that each occupant with an interaction conflict type of 2 has independent operation permissions within their respective corresponding interactive area through a partitioning method. This third operation right allocation information can distribute the operation rights of the first in-vehicle screen to multiple occupants involved in the conflict by area, ensuring that each occupant has independent operation permissions within their corresponding interactive area without interference.

[0128] This application addresses the technical gap in existing technologies by directly entering the partitioning process in the case of the second type of conflict. It divides independent interaction areas according to the spatial pointing landing point and spatial distance of each crew member and generates third operation right attribution information. This solves the technical gap that existing technologies cannot handle the interaction conflicts of three or more people. It achieves the effect of multiple crew members operating in parallel on the same screen without interfering with each other, avoiding the problem of increased decision-making complexity and long-term monopoly of screen resources by a single user due to the increase in the number of people.

[0129] Optionally, the step of determining the interaction area of ​​each occupant on the first in-vehicle screen based on spatial distance and screen interaction weight values ​​of each occupant includes: S1051, if the spatial distance between the spatial pointing landing points of each occupant is greater than the preset spatial distance threshold, then take the spatial pointing landing point of each occupant as the center and divide the non-overlapping interactive areas on the first vehicle screen as the interactive areas of each occupant on the first vehicle screen. S1052, if the spatial distance between the spatial pointing landing points of at least two occupants is less than or equal to the spatial distance threshold, then the screen division ratio is determined according to the screen interaction weight value of each occupant, and the first vehicle screen is divided into multiple interaction areas according to the screen division ratio, which serve as the interaction areas for each occupant in the first vehicle screen.

[0130] In this embodiment of the application, if the spatial distance between the spatial pointing points of each occupant is greater than a preset spatial distance threshold, that is, the spatial distance between the spatial pointing points of any two occupants is greater than the preset spatial distance threshold, then the first vehicle screen is divided into non-overlapping interactive areas centered on the spatial pointing points of each occupant, which serve as the interactive areas of each occupant on the first vehicle screen.

[0131] The spatial distance threshold is a preset distance used to determine whether the spatial pointing points of two occupants are dispersed or converged. When the spatial distance is greater than the threshold, it is determined that the line of sight is dispersed; when it is less than or equal to the threshold, it is determined that the line of sight is converged. The spatial distance threshold can be set according to actual conditions, for example, 50 pixels.

[0132] In the specific implementation, we will take the example of a preset first quantity of 2, where the driver, front passenger, and rear passenger A all want to control the central control screen at the same time.

[0133] The gesture start time windows of the driver, front passenger, and rear passenger A overlap, triggering interaction conflict detection. Interaction conflict detection is triggered when the spatial pointing information of all three individuals falls within the central control screen area, and the gesture start time difference is less than a preset 1-second time window threshold.

[0134] The screen interaction weight value is calculated for each passenger. Assuming that the characteristic value of the driver's pointing duration is 0.80, the characteristic value of the gesture movement speed is 0.85, and the characteristic value of the passenger role is 1.0, the screen interaction weight value of the driver is obtained by weighting and summing according to the weight coefficients of the previous examples, which is 0.86.

[0135] Assuming the co-passenger's pointing duration feature value is 0.75, the gesture movement speed feature value is 0.80, and the occupant role feature value is 0.7, the screen interaction weight value is 0.76 after being weighted and summed according to the weight coefficients of the previous examples.

[0136] Assuming that the characteristic value of the pointing duration of the rear passenger A is 0.70, the characteristic value of the gesture movement speed is 0.75, and the characteristic value of the passenger role is 0.4, the weighted summation yields the screen interaction weight value of the rear passenger A as 0.66.

[0137] The peak weight among the three is 0.86, the trough weight is 0.66, and the difference between the two is 0.20, which is less than the preset weight difference threshold of 0.25. No one has an absolute monopoly advantage, so dynamic partitioning judgment is initiated.

[0138] It can calculate the spatial distance between each pair of the spatial pointing points of the three people. When the spatial pointing points of the three people are all more than the preset spatial distance threshold, such as 50 pixels, it is determined to be a visual dispersion mode.

[0139] At this point, a stationary locking strategy can be adopted. Three non-overlapping interaction areas are generated, centered on each occupant's spatial pointing point. For example, if the driver's spatial pointing point is located in the upper left corner of the screen, the driver's interaction area is located in the upper left corner and highlighted in blue. If the front passenger's spatial pointing point is located in the center of the screen, the front passenger's interaction area is located in the center of the screen and highlighted in green. If the rear passenger's spatial pointing point is located in the lower right corner of the screen, the rear passenger's interaction area is located in the lower right corner and highlighted in orange. The interaction areas are illustrated in Table 2. Each occupant's actions will only be responded to when performed within their corresponding highlighted interaction area.

[0140] Table 2. Schematic diagram of interactive areas

[0141] In this embodiment, when any two occupants' gaze positions on the first in-vehicle screen are very close and the spatial distance is less than or equal to a spatial distance threshold, it indicates that the occupants' gazes are focused on the same area of ​​the screen and cannot be naturally separated by locking in place. In this case, a weighted proportional partitioning method is used. The screen partitioning ratio is calculated based on the proportion of each occupant's screen interaction weight value in the total weight. The screen is then divided into multiple interaction areas according to this ratio, with occupants with higher weights receiving a larger screen area. Screen partitioning ratio refers to the percentage of each occupant's screen interaction weight in the total weight of all participating occupants, within a scene where the viewer's gaze is focused. Higher weights correspond to larger screen partitioning ratios and larger allocated interaction areas.

[0142] In the specific implementation, we will still take the example of a preset first quantity of 2, and the driver, front passenger, and rear passenger A all wanting to control the central control screen at the same time.

[0143] The system can calculate the spatial distance between each pair of the three occupants' spatial pointing points. When the distance between each pair of the three occupants' spatial pointing points is less than or equal to a preset spatial distance threshold, such as 50 pixels, it is determined to be a line-of-sight overlap mode. A weighted proportional partitioning strategy is adopted. First, referring to the example above, the total screen interaction weight value of the three occupants is calculated. Since the driver's seat occupant has a weight value of 0.86, the front passenger seat occupant has a weight value of 0.76, and the rear passenger seat occupant has a weight value of 0.66, the total weight is 2.28. Among them, the driver's seat occupant accounts for approximately 37.7%, the front passenger seat occupant accounts for approximately 33.3%, and the rear passenger seat occupant accounts for approximately 29.0%.

[0144] When dividing the area, the screen interaction area is dynamically divided into three vertical strips according to the above proportions. Referring to Table 3, the left side of the central control screen is allocated to the driver and passenger, which is 0% to 37.7% of the width and is highlighted in blue. The middle part of the central control screen is allocated to the front passenger and passenger, which is 37.7% to 71% of the width and is highlighted in green. The right side of the central control screen is allocated to the rear passenger and passenger, which is 71% to 100% of the width and is highlighted in orange.

[0145] Table 3 Screen Allocation Area Diagram

[0146] The real-time screen interaction weight values ​​of the three occupants can be recalculated every 0.5 seconds. If the weight ratio changes by more than 5%, the screen area is re-divided. When the weight of an occupant increases, the area of ​​their corresponding interaction area expands proportionally, and when the weight decreases, the area of ​​their corresponding interaction area shrinks. The partition boundaries move smoothly to avoid a sense of jumpiness.

[0147] When the occupant's spatial orientation is within their own zone, screen area operations issued by the occupant are effective; however, general cockpit operations still require arbitration.

[0148] If the spatial pointer lands within another passenger compartment and triggers a screen area operation, it will be ineffective; the screen edge will flash to indicate that you should look at your own area. If the spatial pointer lands within another passenger compartment and triggers a general cabin operation, it will still be effective, but arbitration is required; the operation will be ineffective if the spatial pointer does not land on the screen. See Table 4 for details on screen interaction conditions and screen control validity.

[0149] Table 4 Correspondence between Screen Interaction Conditions and Screen Control Validity

[0150] Arbitration refers to the real-time arbitration triggered when multiple occupants simultaneously issue common cockpit operations. It compares the screen interaction weight values ​​of each occupant at the current moment, executing only the command from the occupant with the highest weight. The common cockpit operations of the remaining occupants are temporarily stored, and queuing information is displayed on the interface, such as the driver adjusting the volume while the passenger's command is queued. The screen uses semi-transparent color blocks or highlighted borders to indicate different zones, with corresponding seat icons such as the steering wheel icon, passenger icon, and rear seat icon displayed at the edges of each zone.

[0151] Occupant exit conditions include: a spatial pointer moving off the screen for more than 1.5 seconds, or three consecutive gestures being rejected due to eye movement across zones, deemed as unintentional competition. If either condition is met, the occupant exits the interaction. After exiting, the occupant's zone highlight turns off, and the screen interaction weights of the remaining occupants are renormalized and redistributed across the screen according to the new proportions. For example, after rear occupant A exits, the screen interaction weights of the driver and front passenger are 0.86 plus 0.76, totaling 1.62. The driver's share is approximately 53% (0.86 divided by 1.62), and the front passenger's share is approximately 47%. Once all occupants have exited, the screen zones disappear and the system returns to idle listening mode.

[0152] This application determines whether the line of sight is dispersed or concentrated based on spatial distance. When the line of sight is dispersed, a stationary locking strategy is used to divide the interaction areas into non-overlapping areas. When the line of sight is concentrated, weighted partitioning is performed based on the screen interaction weight value ratio. This solves the problem of unreasonable allocation of screen resources when the line of sight of multiple passengers is distributed in different ways, and achieves the effect of efficient use of screen space and non-interference between multiple passengers operating in parallel.

[0153] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and screen interaction weight value includes: S1061, if the interaction conflict type is the third conflict type, then determine the spatial direction transfer time of the occupant's spatial direction information from the second vehicle screen to the first vehicle screen based on the spatial direction information. S1062, Adjust the occupant's screen interaction weight value according to the spatial direction transfer duration to obtain the cross-screen interaction weight value; S1063, compare the cross-screen interaction weight value with the screen interaction weight values ​​of other occupants whose spatial orientation information is the first vehicle screen, and take the cross-screen interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​as the first interaction weight value. S1064, Generate fourth operation right attribution information based on the first interaction weight value; the fourth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the first interaction weight value.

[0154] In this embodiment of the application, if the interaction conflict type is the third conflict type, the spatial direction transfer time of the occupant's spatial direction information from the second vehicle screen to the first vehicle screen is determined based on the spatial direction information.

[0155] The third type of conflict is a single-person cross-screen conflict, which refers to a scenario where one occupant shifts their spatial orientation from the second in-vehicle screen to the first in-vehicle screen. This type requires adjusting the weight of the cross-screen occupant before they compete with the occupant who was originally looking at the screen.

[0156] In the specific implementation, when the current conflict is determined to be a single-person cross-screen conflict, the time interval between when the occupant's spatial pointing leaves the second in-vehicle screen and lands on the first in-vehicle screen is recorded based on the occupant's spatial pointing information, and is used as the spatial pointing transfer duration. The spatial pointing transfer duration is used to measure the urgency of the occupant's cross-screen operation; the shorter the transfer time, the stronger the occupant's cross-screen intention.

[0157] In this embodiment, the occupant's screen interaction weight value is adjusted based on the spatial direction shift duration to obtain a cross-screen interaction weight value. The shorter the shift duration, the greater the increase in the weight value, and the stronger the cross-screen occupant's competitiveness in subsequent competition. The cross-screen interaction weight value is used for fair comparison with occupants originally focused on the first in-vehicle screen.

[0158] In this embodiment of the application, the cross-screen interaction weight value is compared with the screen interaction weight values ​​of other occupants whose spatial orientation information is the first vehicle screen, and the cross-screen interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​is taken as the first interaction weight value.

[0159] In practice, the adjusted cross-screen interaction weight value of the cross-screen occupant can be compared with the screen interaction weight values ​​of other occupants who are already looking at the first in-vehicle screen. The larger value is selected as the first interaction weight value. The first interaction weight value represents the occupant with the stronger overall intent in the current competition.

[0160] In this embodiment, a fourth operation right attribution information is generated based on a first interaction weight value. This fourth operation right attribution information indicates that the operation right of the first in-vehicle screen belongs to the occupant corresponding to the first interaction weight value. The fourth operation right attribution information grants the operation right of the first in-vehicle screen to the occupant corresponding to the first interaction weight value, regardless of whether the occupant is transferred from another screen or was already looking at the screen; whoever has the higher first interaction weight value gains control.

[0161] In a practical implementation, for example, if there is an independent entertainment screen in the rear, and rear passenger A is operating the rear screen to watch a video, and then suddenly shifts their attention to the central control screen to adjust the air conditioning temperature, while the front passenger is also looking at the central control screen, then the interaction conflict type is the third type of conflict.

[0162] The system detected that the spatial orientation of rear passenger A shifted from the rear screen to the central control screen, for example, recording a spatial orientation shift duration of 0.3 seconds. According to the shift duration mapping rules in Table 5, a shift duration of less than 0.5 seconds is considered an extremely fast shift, indicating a very strong intent, and the shift coefficient is set to 1.2.

[0163] In practical implementation, the correspondence between the transfer duration and the transfer coefficient is k. transfer =f(Δt transfer ), where k transfer This represents the transfer coefficient, which ranges from [0.8, 1.2]. Δt transfer This represents the transition duration. f() represents the mapping rule between the transition duration and the transition coefficient. The specific mapping rule can be found in Table 5.

[0164] Table 5. Transfer Duration Mapping Rules

[0165] Detect the current interaction state of the rear screen. Playing a video on the rear screen is a low-priority task and can be interrupted at any time. The cross-screen penalty coefficient k... penalty The value is set to 1.0. For specific cross-screen penalty coefficients, please refer to Table 6.

[0166] Table 6 Cross-screen penalty coefficients

[0167] Calculate the screen interaction weight values ​​for the two occupants separately. Assuming the feature value of the co-occupant's pointing duration is 0.75, the feature value of the gesture movement speed is 0.70, and the feature value of the occupant's role is 0.7, the screen interaction weight value is obtained by weighting and summing according to the weight coefficients of the previous example.

[0168] Assuming that the characteristic value of the pointing duration of the rear passenger A is 0.65, the characteristic value of the gesture movement speed is 0.60, and the characteristic value of the passenger role is 0.4, the screen interaction weight value is obtained by weighting and summing according to the weight coefficients of the previous examples, which is 0.58.

[0169] In the specific implementation, the cross-screen interaction weight value is calculated as follows: W target，i =W i ×k penalty ×k transfer Among them, W target，iW represents the cross-screen interaction weight value for the i-th passenger. i k represents the screen interaction weight value of the i-th passenger. penalty k represents the cross-screen penalty coefficient. transfer This represents the transfer coefficient.

[0170] It should be noted that in the calculation of cross-screen interaction weight values, W i Let W be the screen interaction weight value for the i-th passenger, which has been previously proven to be dimensionless. The cross-screen penalty coefficient and transition coefficient are both dimensionless correction factors, and the product of these three dimensionless quantities is also dimensionless. Therefore, the cross-screen interaction weight value W... target，i It is also a dimensionless pure numerical value, which can be directly compared with the cross-screen interaction weight value of other occupants.

[0171] Passenger A in the rear row experienced a cross-screen transfer. Their screen interaction weight value was corrected by multiplying it by the cross-screen penalty coefficient and the transfer coefficient. The corrected cross-screen interaction weight value was 0.58 multiplied by 1.0 multiplied by 1.2, which equals 0.696.

[0172] The cross-screen interaction weight value of rear passenger A (0.696) is compared with the screen interaction weight value of the front passenger (0.72). The front passenger's weight value is higher, generating a fourth operation right attribution information, indicating that the operation right of the central control screen belongs to the front passenger. After the front passenger gains control of the central control screen, a green light effect is displayed at the bottom of the central control screen to indicate that the front passenger is operating the screen, and the gesture commands of rear passenger A on the central control screen are temporarily stored and not responded to.

[0173] The system automatically selects a user-friendly feedback method based on the scenario. Users can customize the feedback method by referring to Table 7. If rear passenger A's intention can be completed on the rear screen, the system will automatically switch to a notification indicating that air conditioning adjustments have been moved to the rear screen and requesting continued operation. If the front passenger's operation is short, the system will notify the front passenger that the operation is in progress and will notify them upon completion. If guidance is needed, the system will provide voice prompts indicating that the front passenger is using the central control screen and can adjust the air conditioning via the rear screen. The rear screen remains unchanged, and video playback continues unaffected.

[0174] Table 7 Friendly Feedback Methods

[0175] This application corrects and adjusts the screen interaction weight value of cross-screen occupants based on the spatial direction shift duration in single-person cross-screen conflict scenarios, and generates fourth operation right attribution information by comparing the corrected cross-screen interaction weight value with the occupant who originally looked at the target screen. This solves the problem that the intensity of intent cannot be quantified and compared when occupants shift their gaze from other screens to participate in the interaction, and realizes reasonable competition between cross-screen occupants and same-screen occupants based on comprehensive intent, avoiding the unfair phenomenon of cross-screen operations being unconditionally suppressed or unconditionally prioritized.

[0176] Optionally, the step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and screen interaction weight value includes: S1071, if the interaction conflict type is the fourth conflict type, then the sum of the screen interaction weight values ​​of multiple occupants whose spatial direction information is transferred from the second vehicle screen to the first vehicle screen is determined based on the spatial direction information, and is used as the team interaction weight value. S1072, compare the team interaction weight value with the screen interaction weight value of other occupants whose spatial orientation information is the first vehicle screen, and take the team interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​as the second interaction weight value. S1073, Generate fifth operation right attribution information based on the second interaction weight value; the fifth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the second interaction weight value.

[0177] In this embodiment, if the interaction conflict type is the fourth conflict type, the sum of the screen interaction weight values ​​of multiple occupants whose spatial pointing information is transferred from the second vehicle screen to the first vehicle screen is determined based on the spatial pointing information, and is used as the team interaction weight value. The fourth conflict type can refer to a multi-person cross-screen conflict type, which refers to a conflict scenario where two or more occupants simultaneously or sequentially transfer spatial pointing information from the second vehicle screen to the first vehicle screen.

[0178] In the specific implementation, when the current conflict is determined to be a multi-person cross-screen conflict, the screen interaction weight values ​​of all cross-screen occupants who have moved from the second in-vehicle screen to the first in-vehicle screen are summed to obtain the team interaction weight value. The team interaction weight value is a joint weight value obtained by summing the individual screen interaction weight values ​​of multiple cross-screen occupants. The team interaction weight value is used to compete with the cross-screen occupants as a whole against the occupants who were originally looking at the screen.

[0179] In this embodiment, the team interaction weight value is compared with the screen interaction weight values ​​of other occupants whose spatial orientation information is the first in-vehicle screen. The team interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​is used as the second interaction weight value. Specifically, the team interaction weight value of the cross-screen occupant team is compared with the screen interaction weight values ​​of other occupants who are already looking at the first in-vehicle screen, and the larger value is selected as the second interaction weight value. The second interaction weight value represents the party with a stronger overall intention in the multi-person cross-screen competition.

[0180] In this embodiment, a fifth operation right attribution information is generated based on the second interaction weight value. The fifth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the second interaction weight value. If the team interaction weight value wins, the operation right belongs to the cross-screen occupant team; if the weight value of the original screen occupant wins, the operation right belongs to that occupant.

[0181] In specific implementations, for example, it can detect that two occupants in the back row, namely occupant A and occupant B, simultaneously transfer spatial pointing information to the central control screen, and the overlap of the gesture start time window is less than 1 second.

[0182] Rear passenger A wants to lower the air conditioning temperature, rear passenger B wants to change the song, and at the same time the front passenger is also looking at the center console screen, hoping to adjust the navigation.

[0183] At this point, internal sub-arbitration within the back row is performed first, and screen interaction weight values ​​are calculated for the two occupants in the back row. Assuming that the characteristic value of the pointing duration of occupant A in the back row is 0.70, the characteristic value of the gesture movement speed is 0.65, and the characteristic value of the occupant role is 0.4, the screen interaction weight value is obtained by weighting and summing according to the weight coefficients of the previous example, which is 0.62.

[0184] The characteristic value of the pointing duration of the rear passenger B is 0.60, the characteristic value of the gesture movement speed is 0.55, and the characteristic value of the passenger role is 0.4. After weighting and summing according to the weight coefficients of the previous examples, the screen interaction weight value is 0.54.

[0185] Since rear passenger A's screen interaction weight value is higher than that of rear passenger B, rear passenger A can be selected as the rear representative. The rear representative can carry two sets of operational intentions. Figure 1 To lower the air conditioning temperature from rear passenger A, Figure 2 The song switching was initiated by passenger B in the back row. The sum of the screen interaction weights of the two passengers in the back row was used as the team interaction weight, i.e., 0.62 plus 0.54 equals 1.16. The total team weight can exceed 1.0, indicating that the intention of multiple people working together is stronger than the intention of a single person.

[0186] The team interaction weight value of 1.16 is compared with the screen interaction weight value of 0.72 for the front passenger. Since the team interaction weight value of 1.16 is greater than the screen interaction weight value of 0.72 for the front passenger, the fifth operation right attribution information is generated, indicating that the operation right of the central control screen belongs to the rear group with the larger screen interaction weight value, and the rear group obtains control of the central control screen.

[0187] This application solves the problem of unreasonable allocation of control rights when multiple back-row passengers simultaneously move from the back-row screen to the front-row screen to participate in interaction in multi-person cross-screen conflict scenarios. It generates a team interaction weight value by accumulating the screen interaction weight values ​​of multiple cross-screen passengers and comparing the weights with those of the passengers originally looking at the screen. This enables fair competition between cross-screen passenger teams and same-screen passengers based on joint intent.

[0188] Optionally, the method further includes the following steps: S1081, If ​​the second interaction weight value is the team interaction weight value, then the operation time ratio of each crew member is determined according to the ratio of the individual screen interaction weight value of each crew member to the team interaction weight value. S1082, according to the screen interaction weight value from high to low, within the preset operation cycle, the operation time period is allocated to multiple passengers in turn according to the operation time ratio. S1083, Generate sixth operation right attribution information based on the operation time period of each occupant; the sixth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the corresponding occupant in each operation time period within the operation cycle.

[0189] In this embodiment of the application, if the second interaction weight value is the team interaction weight value, then the operation time ratio of each passenger is determined based on the ratio of the individual screen interaction weight value of each passenger to the team interaction weight value. The operation time ratio can refer to the percentage of each cross-screen passenger's screen interaction weight value in the total team interaction weight value. The operation time ratio is used to determine the proportion of operation time obtained by each passenger within the operation cycle.

[0190] When a team of cross-screen participants wins a cross-screen conflict and gains control, this control needs to be further distributed within the team. The percentage of each participant's screen interaction weight relative to the team's total interaction weight is used to calculate the proportion of operation time allocated to each participant in subsequent operation cycles. Participants with higher weights receive a larger percentage of operation time. The preset operation cycle can be set according to actual needs, for example, to 4 seconds.

[0191] In this embodiment, within a preset operation period, operation time slots are allocated to multiple passengers according to their screen interaction weight values, from highest to lowest. An operation time slot refers to a segment of time within the operation period that grants each cross-screen passenger exclusive operation rights, and the operation time slots for each passenger are arranged sequentially from highest to lowest weight.

[0192] In practice, the pre-defined operation cycle can be divided according to the proportion of each crew member's operation time, and the operation time slots can be arranged sequentially according to the crew member's weight value from high to low. The crew member with the highest weight gets the operation time first, and the crew members with lower weights get the operation time in turn, ensuring that the crew members with stronger intentions perform their operations first.

[0193] In this embodiment of the application, a sixth operation right attribution information is generated based on the operation time period of each passenger; the sixth operation right attribution information is used to indicate that the operation right of the first vehicle screen is respectively assigned to the corresponding passenger in each operation time period within the operation cycle, so that each passenger in the cross-screen passenger team can obtain the exclusive operation right of the first vehicle screen in turn within their respective allocated time period.

[0194] In the specific implementation, after the rear-seat delegation, including rear-seat passengers A and B, is authorized, it can allocate time slices within the delegation according to the proportion of each passenger's sub-weight, and set the operation cycle, or rotation cycle, to 4 seconds.

[0195] Assume that the sub-weight of rear passenger A is 0.62 and the sub-weight of rear passenger B is 0.54, the sum of 0.62 and 0.54 is 1.16. The time slice allocated to rear passenger A is 4 seconds multiplied by 0.62 and divided by 1.16, resulting in approximately 2.14 seconds. The time slice allocated to rear passenger B is 4 seconds multiplied by 0.54 and divided by 1.16, resulting in approximately 1.86 seconds.

[0196] Within their assigned time slots, the vehicle only responds to the intended commands of the corresponding occupants. That is, it responds to user A's command to lower the air conditioning within 0 to 2.14 seconds, and to user B's command to change the song within 2.14 to 4.00 seconds. If the front passenger's request is rejected, the vehicle prompts the front passenger to revert to their previous state, indicating that the rear passengers are jointly operating the central control screen and the front passenger's command is queued and will be executed in 4 seconds, or prompting the front passenger to try again later. When the rear passenger representative or all passengers on the delegation look away from the central control screen for more than 1.5 seconds, or when all rear passengers have completed their operations and voluntarily withdrawn, the vehicle releases the interaction permissions and returns to idle listening mode.

[0197] This application solves the problem of disorderly allocation of operation rights after multiple cross-screen crew members win a team by determining the proportion of operation time based on the ratio of each crew member's screen interaction weight value to the team's interaction weight value, and then allocating operation time periods sequentially within a preset operation cycle according to the order of weight from high to low to generate the sixth operation right attribution information. This achieves a fair scheduling effect of taking turns operating according to weight and time ratio within the team.

[0198] Optionally, after generating operation right attribution information based on the interaction area of ​​each occupant on the first in-vehicle screen, the method further includes the following steps: S1091, determine the operation type based on gesture operation information; the operation type includes screen area operation and cockpit general operation. S1092, if the operation type is screen area operation, then respond to the screen area operation in the interaction area corresponding to the occupant who issued the gesture operation information. S1093, if the operation type is a cockpit general operation, then respond to the cockpit general operation of the occupant with the highest screen interaction weight value, and temporarily store the cockpit general operations of the remaining occupants.

[0199] In this embodiment, the operation type can be determined based on gesture operation information. The operation type includes screen area operation and general cockpit operation.

[0200] Screen area operations can refer to gesture operations related to specific display locations on the first in-vehicle screen, such as clicking icons, swiping lists, and opening applications. In partitioned mode, each occupant responds independently within their respective interaction area.

[0201] General cockpit operations refer to vehicle-level gesture operations that are independent of their specific display location on the main in-vehicle screen, such as volume adjustment, track switching, and air conditioning temperature adjustment. General cockpit operations are uniformly arbitrated in zone mode, responding only to commands from the occupant with the highest priority.

[0202] In the specific implementation, after completing the partitioning and generating the operation right ownership information, the gesture operation information issued by the occupant is classified and identified to determine whether the gesture command belongs to the screen area operation or the general cockpit operation.

[0203] In this embodiment of the application, if the operation type is a screen area operation, the screen area operation is responded to in the interaction area corresponding to the occupant who issued the gesture operation information.

[0204] In the implementation, when a passenger's gesture command pertains to a screen area operation, such as clicking an icon or swiping a list, the operation is only responded to within the interaction area allocated to the passenger who issued the gesture. Different passengers' screen area operations within their respective areas do not interfere with each other and are executed independently.

[0205] In this embodiment of the application, if the operation type is a general cockpit operation, the general cockpit operation of the occupant with the highest screen interaction weight value is responded to, and the general cockpit operations of the remaining occupants are temporarily stored.

[0206] In practice, when an occupant's gesture command is a general cabin operation, such as adjusting volume, changing tracks, or adjusting the air conditioning temperature—operations unrelated to the specific location of the screen—it is not executed separately by each zone, but rather through unified arbitration. Only the general cabin operation of the occupant with the highest screen interaction weight value among all current occupants is responded to; the general cabin operations of other occupants are temporarily suspended, awaiting further processing.

[0207] Gesture operation information can be categorized into two types: screen area operations and cockpit-wide operations. Screen area operations are related to specific screen locations, such as clicking icons, swiping lists, and opening applications. These are executed by area and do not interfere with each other. Cockpit-wide operations are vehicle-level operations independent of location, such as volume adjustment, track switching, and air conditioning temperature adjustment. These are uniformly arbitrated and only respond to the commands of the occupant with the highest priority. See Table 8 for a detailed illustration of operation type arbitration.

[0208] Table 8. Arbitration Diagram for Operation Types

[0209] This application solves the problem that location-independent global operations may cause secondary conflicts during partitioned interaction by dividing gesture operation information into two types: screen area operations and cockpit general operations. In partitioned mode, screen area operations are responded to independently within each occupant's own interaction area, while cockpit general operations are uniformly arbitrated and only the instructions of the occupant with the highest weight are responded to. This achieves the synergistic effect of screen area operations not interfering with each other and cockpit general operations being arbitrated in an orderly manner.

[0210] Optionally, the method further includes the following steps: S1101, if it is detected that the duration of any occupant's spatial pointing information moving out of the first vehicle screen exceeds the first duration threshold, or if no gesture operation information is issued within the second duration threshold, then the occupant's operation right in the first vehicle screen is released, and the remaining occupants are allowed to redistribute the interaction area of ​​the first vehicle screen.

[0211] In this embodiment, during the partitioned interaction process, the interaction status of each occupant is monitored in real time. When an occupant's spatial pointer moves away from the first in-vehicle screen for more than a first duration threshold, or when no gesture operation is performed for more than a second duration threshold, it is determined that the occupant no longer intends to participate in the interaction, and the operation rights held by the occupant on the first in-vehicle screen are actively released. After release, the remaining occupants still in the interaction state recalculate the proportions according to the original screen interaction weight values, and the interaction area of ​​the first in-vehicle screen is re-divided.

[0212] In practice, after the driver completes the operation, the spatial pointer moves out of the central control screen. If no new spatial pointer is detected after 1.5 seconds, the driver's operation authority is released, and the system returns to the idle listening state, allowing for the next conflict arbitration.

[0213] This application solves the problem of wasted screen area resources and rigid allocation after passengers leave midway in partitioned interaction by monitoring in real time whether the spatial pointer of the passenger moves out of the screen for more than a first time threshold or does not perform any gesture operation for more than a second time threshold. It automatically releases the interaction area of ​​the departing passenger and triggers the remaining passengers to redivide the screen area. It realizes the dynamic recycling and redistribution of screen resources and ensures that the interaction area always matches the actual number of participants and their intentions.

[0214] Reference Figure 2 The diagram shows a logical schematic of a vehicle control method provided in an embodiment of this application.

[0215] S201, Vehicle screen confirmed. In this embodiment, spatial pointing information and gesture operation information issued by several occupants to the same vehicle screen can be obtained, and the occupant seat information corresponding to each occupant can be obtained.

[0216] S202, Screen type determination. The system can determine the screen type of the in-vehicle screen. If the screen type is a central control screen, it is assumed that the in-vehicle screen faces all occupants and is divisible. If the screen type is a rear entertainment screen, it is only visible to rear occupants. Alternatively, the system can disregard the screen type and apply the vehicle control method of this application to any screen in the vehicle.

[0217] S203, if the number of occupants equals a preset first number, and the difference between the weight peak and the weight trough is greater than a preset weight difference threshold, then exclusive authorization is granted. If the number of occupants equals a preset first number, such as two people, and the difference between the weight peak and the weight trough is greater than a preset weight difference threshold, first operation right attribution information is generated; the first operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the weight peak.

[0218] If no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is equal to the preset first number, then the interaction conflict type between the occupants is determined to be the first conflict type.

[0219] If the interaction conflict type is the first type, then obtain the peak and valley values ​​of the weight from the screen interaction weight values ​​corresponding to each crew member. If the difference between the weight peak and the weight trough is greater than the preset weight difference threshold, then the first operation right attribution information is generated; the first operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the weight peak.

[0220] If the difference between the weight peak and the weight trough is less than or equal to the preset weight difference threshold, a second operation right attribution information is generated based on the interaction area of ​​each occupant on the first vehicle screen; the second operation right attribution information is used to indicate that each occupant with the interaction conflict type of the first conflict type has operation rights in their respective interaction areas.

[0221] S204. If the number of occupants is greater than the preset first number, or the difference between the weight peak and the weight trough is less than or equal to the preset weight difference threshold, then dynamic partitioning is performed.

[0222] If the difference between the weight peak and the weight trough is less than or equal to the weight difference threshold, the spatial pointing point of each passenger on the first vehicle screen is determined based on the spatial pointing information of each passenger. The spatial distance between the spatial pointing points of each passenger is determined based on the spatial pointing point of each passenger on the first in-vehicle screen. Based on spatial distance and screen interaction weight values ​​of each occupant, the interaction area of ​​each occupant in the first in-vehicle screen is determined; The second operation right attribution information is generated based on the interaction area of ​​each occupant on the first in-vehicle screen; the second operation right attribution information is used to indicate that each occupant with the first conflict type has operation rights in their respective interaction area.

[0223] If no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is greater than the first number, then the interaction conflict type between the occupants is determined to be the second conflict type. If the interaction conflict type is the second conflict type, then the spatial pointing landing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger. The spatial distance between the spatial pointing points of each passenger is determined based on the spatial pointing point of each passenger on the first in-vehicle screen. Based on spatial distance and screen interaction weight values ​​of each occupant, the interaction area of ​​each occupant in the first in-vehicle screen is determined; The third operation right attribution information is generated based on the interaction area of ​​each occupant on the first in-vehicle screen; the third operation right attribution information is used to indicate that each occupant with the interaction conflict type of the second conflict type has the operation right in their respective interaction area.

[0224] S205, if the spatial pointing points are dispersed, the system will lock in place. If the spatial distance between the spatial pointing points of each occupant is greater than a preset spatial distance threshold, then non-overlapping interactive areas will be divided on the first vehicle screen, centered on the spatial pointing points of each occupant, as the interactive areas for each occupant on the first vehicle screen.

[0225] S206, if the spatial pointing points are concentrated, the screen is divided into zones proportionally. If the spatial distance between the spatial pointing points of at least two occupants is less than or equal to the spatial distance threshold, the screen division ratio is determined according to the screen interaction weight value of each occupant, and the first vehicle screen is divided into multiple interaction areas according to the screen division ratio, which serve as the interaction areas for each occupant on the first vehicle screen.

[0226] S207, in the event of a single-person cross-screen conflict, the occupant's screen interaction weight value is adjusted, and the cross-screen interaction weight value is then used for sub-arbitration. If the interaction conflict type is the third type of conflict, the spatial orientation information of the occupant is transferred from the second vehicle screen to the first vehicle screen based on the spatial orientation information. The cross-screen interaction weight value is obtained by adjusting the occupant's screen interaction weight value based on the spatial direction transfer duration. Compare the cross-screen interaction weight value with the screen interaction weight values ​​of other occupants whose spatial orientation information is the first in-vehicle screen, and take the cross-screen interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​as the first interaction weight value. The fourth operation right attribution information is generated based on the first interaction weight value; the fourth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the first interaction weight value.

[0227] S208, in the case of multi-person cross-screen conflicts, within a preset operation cycle, operation time periods are allocated to multiple occupants according to the proportion of operation time. If the interaction conflict type is the fourth type of conflict, the sum of the screen interaction weight values ​​of multiple occupants whose spatial direction information is transferred from the second vehicle screen to the first vehicle screen is determined based on the spatial direction information, and this sum is used as the team interaction weight value. Compare the team interaction weight value with the screen interaction weight values ​​of other occupants whose spatial orientation information is the first in-vehicle screen, and take the team interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​as the second interaction weight value. The fifth operation right attribution information is generated based on the second interaction weight value; the fifth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the second interaction weight value.

[0228] If the second interaction weight value is the team interaction weight value, then the operation time ratio of each crew member is determined according to the ratio of the individual screen interaction weight value of each crew member to the team interaction weight value. Based on the screen interaction weight values ​​from high to low, within the preset operation cycle, operation time periods are allocated to multiple passengers according to the proportion of operation duration. The sixth operation right attribution information is generated based on the operation time period of each passenger; the sixth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the corresponding passenger in each operation time period within the operation cycle.

[0229] One embodiment of this application also provides a vehicle that may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0230] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0231] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0232] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0233] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0234] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other modifications and updates to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all modifications and updates falling within the scope of the embodiments of the present application.

[0235] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0236] The above provides a detailed description of the vehicle control method and the vehicle provided. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A vehicle control method, characterized in that, The vehicle includes several in-vehicle screens, and the method includes: Acquire spatial pointing information and gesture operation information sent by several occupants to the same vehicle screen, and acquire the occupant seat information corresponding to each occupant; For any of the occupants, a pointing duration feature value is determined based on the spatial pointing information, a gesture movement speed feature value is determined based on the gesture operation information, and an occupant role feature value is determined based on the occupant seating information. The pointing duration feature value, the gesture movement speed feature value, and the occupant role feature value are then weighted and summed to obtain the occupant's screen interaction weight value. Based on the spatial orientation information and seating information of each occupant, determine the type of interaction conflict between the occupants; The operation rights attribution information of the in-vehicle screen is determined based on the interaction conflict type and the screen interaction weight value of each occupant. The order in which each passenger operates the in-vehicle screen is determined based on the information regarding the ownership of operating rights. According to the operation sequence, corresponding vehicle control commands are generated through the vehicle screen based on the spatial orientation information and gesture operation information of each passenger. The vehicle is controlled according to the vehicle control command.

2. The method according to claim 1, characterized in that, The same vehicle screen is used as the first vehicle screen, and the other vehicle screens among the plurality of vehicle screens other than the first vehicle screen are used as the second vehicle screen. The step of determining the type of interaction conflict between occupants based on their spatial orientation information and seating information includes: Obtain the number of passengers; If no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is equal to a preset first number, then the interaction conflict type between the occupants is determined to be the first conflict type. If no spatial orientation information of the occupants is detected to be transferred from the second vehicle screen to the first vehicle screen, and the number of occupants is greater than the first number, then the interaction conflict type between the occupants is determined to be the second conflict type. If it is detected that the spatial orientation information of any occupant has been transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the third conflict type. If it is detected that the spatial orientation information of multiple occupants has been transferred from the second vehicle screen to the first vehicle screen, then the interaction conflict type between the occupants is determined to be the fourth conflict type.

3. The method according to claim 2, characterized in that, The step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the first conflict type, then obtain the weight peak value and weight valley value from the screen interaction weight values ​​corresponding to each passenger. If the difference between the weight peak value and the weight valley value is greater than a preset weight difference threshold, then first operation right attribution information is generated; the first operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the weight peak value.

4. The method according to claim 3, characterized in that, The method further includes: If the difference between the weight peak value and the weight valley value is less than or equal to the weight difference threshold, then the spatial pointing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger. The spatial distance between the spatial pointing points of each passenger is determined based on the spatial pointing point of each passenger on the first vehicle screen. Based on the spatial distance and the screen interaction weight value of each occupant, the interaction area of ​​each occupant in the first vehicle screen is determined; A second operation right attribution information is generated based on the interaction area of ​​each passenger on the first vehicle screen; the second operation right attribution information is used to indicate that each passenger whose interaction conflict type is the first conflict type has operation rights in their respective corresponding interaction area.

5. The method according to claim 2, characterized in that, The step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the second conflict type, then the spatial pointing point of each passenger on the first vehicle screen is determined according to the spatial pointing information of each passenger. The spatial distance between the spatial pointing points of each passenger is determined based on the spatial pointing point of each passenger on the first vehicle screen. Based on the spatial distance and the screen interaction weight value of each occupant, the interaction area of ​​each occupant in the first vehicle screen is determined; A third operation right attribution information is generated based on the interaction area of ​​each occupant on the first vehicle screen; the third operation right attribution information is used to indicate that each occupant whose interaction conflict type is the second conflict type has operation rights in their respective corresponding interaction areas.

6. The method according to claim 4 or 5, characterized in that, The step of determining the interaction area of ​​each occupant on the first in-vehicle screen based on the spatial distance and the screen interaction weight value of each occupant includes: If the spatial distance between the spatial pointing points of each occupant is greater than a preset spatial distance threshold, then the first vehicle screen is divided into non-overlapping interactive areas centered on the spatial pointing points of each occupant, which serve as the interactive areas for each occupant on the first vehicle screen. If the spatial distance between the spatial pointing landing points of at least two occupants is less than or equal to the spatial distance threshold, then the screen division ratio is determined according to the screen interaction weight value of each occupant, and the first vehicle screen is divided into multiple interaction areas according to the screen division ratio, which serve as the interaction areas for each occupant on the first vehicle screen.

7. The method according to claim 2, characterized in that, The step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the third conflict type, then the spatial orientation transfer time of the occupant from the second vehicle screen to the first vehicle screen is determined according to the spatial orientation information. The occupant's screen interaction weight value is adjusted according to the spatial direction transfer duration to obtain the cross-screen interaction weight value; The cross-screen interaction weight value is compared with the screen interaction weight value of other occupants whose spatial orientation information is the first vehicle screen. The cross-screen interaction weight value or screen interaction weight value that is greater than the other interaction weight value is taken as the first interaction weight value. A fourth operation right attribution information is generated based on the first interaction weight value; the fourth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the first interaction weight value.

8. The method according to claim 2, characterized in that, The step of determining the operation right ownership information of the first in-vehicle screen based on the interaction conflict type and the screen interaction weight value includes: If the interaction conflict type is the fourth conflict type, then the sum of the screen interaction weight values ​​of multiple occupants whose spatial direction information is transferred from the second vehicle screen to the first vehicle screen is determined based on the spatial direction information, and is used as the team interaction weight value. The team interaction weight value is compared with the screen interaction weight value of other occupants whose spatial orientation information is the first vehicle screen. The team interaction weight value or screen interaction weight value that is greater than the other interaction weight values ​​is taken as the second interaction weight value. A fifth operation right attribution information is generated based on the second interaction weight value; the fifth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the occupant corresponding to the second interaction weight value.

9. The method according to claim 8, characterized in that, The method further includes: If the second interaction weight value is the team interaction weight value, then the operation time ratio of each passenger is determined according to the ratio of the individual screen interaction weight value of each passenger to the team interaction weight value. According to the screen interaction weight values ​​from high to low, within the preset operation cycle, the operation time periods are allocated to the multiple passengers in sequence according to the operation duration ratio. A sixth operation right attribution information is generated based on the operation time period of each passenger; the sixth operation right attribution information is used to indicate that the operation right of the first vehicle screen belongs to the corresponding passenger for each operation time period within the operation cycle.

10. The method according to claim 6, characterized in that, After generating operation right attribution information based on the interaction area of ​​each occupant on the first vehicle screen, the method further includes: The operation type is determined based on the gesture operation information; the operation type includes screen area operation and general cockpit operation. If the operation type is the screen area operation, then the screen area operation is responded to in the interaction area corresponding to the occupant who issued the gesture operation information. If the operation type is a general cockpit operation, then the general cockpit operation of the occupant with the highest screen interaction weight value is responded to, and the general cockpit operations of the remaining occupants are temporarily stored.

11. The method according to claim 10, characterized in that, The method further includes: If it is detected that any occupant's spatial pointing information has been moved out of the first vehicle screen for a duration exceeding a first duration threshold, or if no gesture operation information is issued within a second duration threshold, then the occupant's operation rights on the first vehicle screen are released, and the remaining occupants are allowed to redistribute the interaction area of ​​the first vehicle screen.

12. A vehicle, characterized in that, It includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method as described in any one of claims 1-11.

Images