Vehicle-mounted intelligent armrest box based on multi-mode sensing fusion and control method thereof

Abstract

The invention relates to the technical field of automobile intelligent cabin man-machine interaction, in particular to a vehicle-mounted intelligent armrest box based on multi-mode sensing fusion and a control method thereof.The method comprises the steps that a multi-mode sensing module is used for obtaining environment information in an automobile cabin, the environment information comprises personnel state information, object state information, demand information and seating pressure signals in a vehicle cabin; the preset control center is used for performing intention recognition, reasoning and decision making on the environment information to generate a control instruction; the vehicle-mounted intelligent armrest box is used for controlling a corresponding mechanical structure to execute according to the control instruction and feeding back state information of the mechanical structure to the preset control center; and the power management module is used for supplying power to the vehicle-mounted intelligent armrest box. Therefore, the problem that the core requirements of efficient space utilization, man-machine engineering adaptation and scene non-inductive linkage in the intelligent cabin environment are difficult to meet in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of human-computer interaction in an intelligent automobile cockpit, and in particular to an on-board intelligent armrest box based on multimodal perception fusion and a control method thereof. Background Art

[0002] With the rapid development of smart cars and mobile office demand, vehicle cockpits are expanding from simple driving spaces to multifunctional living scenarios. Rear-seat passengers' demands for comfort, entertainment, and productivity have significantly increased. As the core interactive platform for the rear seats, the central armrest's limited functionality and lack of intelligence are becoming increasingly prominent.

[0003] The main technical limitations of the rear center armrest boxes of mainstream cars on the market are as follows:

[0004] (1) Lack of human-computer interaction: Existing armrests (such as those in the 2023 Volkswagen Passat and the standard Toyota Camry Hybrid) only provide basic storage and physical cup holder functions. They lack electronic control interfaces and are unable to respond to voice commands or connect to the vehicle's systems. Passengers must manually open and close the armrest lid and adjust components, which poses a safety hazard when operating while driving.

[0005] (2) Weak support for mobile office: A few high-end models (such as the optional office suite of the Mercedes-Benz S-Class) have tried to integrate folding tables, but there are significant defects: fixed structures (such as the extended table in the armrest box of the BMW 7 Series G12) cannot rotate horizontally, and passengers on the left and right need to twist their bodies to use the keyboard; the load-bearing stability is poor, and the engineering plastic bracket causes the keyboard to shake when the vehicle is bumpy (the measured amplitude is greater than 5mm); the power supply capacity is lacking, and external devices require additional wiring, which damages the appearance of the cabin.

[0006] (3) Fragmentation of intelligent scenarios: Although some models (such as the NIO ET7) are equipped with voice assistants, their control objects are limited to the air conditioning and entertainment systems, and are not deeply coupled with the physical structure of the armrest box. For example, when the rear screen is in office mode, passengers still need to manually unfold the input device; operations such as raising and lowering the cup holder and charging the device rely on physical buttons and cannot be automatically triggered through AI intent perception.

[0007] In summary, existing technologies struggle to meet the core requirements of "efficient space utilization, ergonomic adaptation, and seamless scene interaction" in a smart cockpit environment. Therefore, a revolutionary armrest solution integrating an adjustable input platform, intelligent power supply, and AI intent response is urgently needed. Summary of the Invention

[0008] This application provides an in-vehicle intelligent armrest box and its control method based on multimodal perception fusion to solve the problems that the existing technology is difficult to meet the core requirements of "efficient space utilization + ergonomic adaptation + seamless scene linkage" in the intelligent cockpit environment.

[0009] The first aspect of the present application provides an in-vehicle smart armrest box based on multimodal perception fusion, comprising the following steps: a multimodal perception module for acquiring environmental information in the vehicle cabin, wherein the environmental information includes occupant status information, object status information, demand information and seating pressure signals in the vehicle cabin; a preset control center for performing intention recognition, reasoning and decision-making on the environmental information to generate control instructions; an in-vehicle smart armrest box for controlling the execution of a corresponding mechanical structure according to the control instructions, and feeding back the status information of the mechanical structure to the preset control center; a power management module for supplying power to the in-vehicle smart armrest box.

[0010] Optionally, the multimodal perception module includes:

[0011] The in-cabin DMS / OMS camera is used to visually perceive people and objects in the cabin to collect the status information of the people and objects in the cabin; the microphone array is used to capture the voices of people in the cabin to perform offline voice wake-up and online command analysis based on the voices of people in the cabin to generate the demand information; the seat pressure sensor is used to identify whether there is an occupant in the seat and generate the seating pressure signal.

[0012] Optionally, the preset control center includes:

[0013] A consciousness recognition unit is used to perform intention recognition based on the environmental information to obtain the user's actual intention; a reasoning and decision unit is used to perform logical reasoning on the user's actual intention to determine the intention execution mode, and decide the corresponding control instructions based on the intention execution mode, wherein the intention execution mode is any one of the office mode, meeting mode and entertainment mode.

[0014] Optionally, the vehicle-mounted intelligent armrest box includes a top-level actuator, a middle actuator and a bottom-level actuator, wherein:

[0015] The top actuator is the mouse tray layer, which includes a mouse, a first USB port, a first wireless charging port, and an electric lifting mechanism; the middle actuator is the keyboard tray layer, which includes a keyboard, a second USB port, a second wireless charging port, an electric worm gear mechanism, and an angle encoder; the bottom actuator includes a cup holder, a storage box, a temperature control sensor, an electric push rod mechanism, and a telescopic guide rail.

[0016] A second embodiment of the present application provides a method for controlling an in-vehicle intelligent armrest box based on multimodal perception fusion, including:

[0017] Acquire environmental information within the vehicle cabin, wherein the environmental information includes occupant status information, object status information, demand information, and seating pressure signals within the vehicle cabin; perform intent recognition, reasoning, and decision-making on the environmental information to generate control instructions; and trigger corresponding mechanical structures in the vehicle-mounted smart armrest box according to the control instructions.

[0018] Optionally, obtaining the environmental information in the vehicle cabin includes:

[0019] The DMS / OMS camera in the cabin is used to perform visual perception of people and objects in the cabin to collect the status information of the people in the cabin and the status information of the objects in the cabin; the microphone array is used to capture the voices of people in the cabin to perform offline voice wake-up and online command analysis based on the voices of people in the cabin to generate the demand information; the seat pressure sensor is used to identify whether there is an occupant in the seat and generate the seating pressure signal.

[0020] Optionally, performing intention recognition, reasoning, and decision-making on the environmental information to generate a control instruction includes:

[0021] Intent recognition is performed based on the environmental information to obtain the user's actual intention; logical reasoning is performed on the user's actual intention to determine the intention execution mode, and corresponding control instructions are decided based on the intention execution mode, wherein the intention execution mode is any one of office mode, meeting mode and entertainment mode.

[0022] Optionally, triggering a corresponding mechanical structure in the vehicle-mounted intelligent armrest box according to the control instruction includes:

[0023] When the top-level actuator in the on-board smart armrest box is triggered according to the control instruction, the electric lifting mechanism in the top-level actuator is used to convert the mouse, the first USB interface, and the first wireless charging interface in the top-level actuator from a retracted state to an open state or from an open state to a retracted state; when the middle actuator in the on-board smart armrest box is triggered according to the control instruction, the electric worm gear mechanism in the middle actuator is used to convert the keyboard, the second USB interface, and the second wireless charging interface in the middle actuator from a rolled-up state to an unfolded state or from an unfolded state to a rolled-up state, and the angle encoder in the middle actuator is used to adjust the information in real time; when the bottom-level actuator in the on-board smart armrest box is triggered according to the control instruction, the electric push rod mechanism and the telescopic guide rail in the bottom-level actuator are used to withdraw or retract the cup holder in the bottom-level actuator.

[0024] The third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion as described in the above embodiment.

[0025] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion as described in the above embodiment.

[0026] The embodiment of the present invention proposes a vehicle-mounted intelligent armrest box machine control method based on multimodal perception fusion, which realizes intelligent control of the mouse layer, keyboard layer, and cup holder through multimodal perception and control (voice, vision, gesture) and the vehicle-mounted artificial intelligence system. The keyboard layer can realize multi-angle adaptive adjustment to meet the personalized comfort experience of the left and right passengers in the rear row, and the central armrest system with equipment power supply and intelligent control of the cup holder.

[0027] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0029] Figure 1 This is a structural diagram of an in-vehicle intelligent armrest box based on multimodal perception fusion according to an embodiment of the present application;

[0030] Figure 2 This is a schematic diagram of the specific structure of an in-vehicle intelligent armrest box based on multimodal perception fusion according to an embodiment of the present application;

[0031] Figure 3 A schematic diagram of an execution of an intermediate actuator provided according to an embodiment of the present application;

[0032] Figure 4 A specific decision flow chart of an in-vehicle intelligent armrest box based on multimodal perception fusion according to an embodiment of the present application;

[0033] Figure 5 This is a flow chart of a method for controlling a vehicle-mounted intelligent armrest box based on multimodal perception fusion according to an embodiment of the present application;

[0034] Figure 6 A schematic structural diagram of a vehicle provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0036] The following describes an in-vehicle intelligent armrest box and a control method thereof based on multimodal perception fusion according to an embodiment of the present application with reference to the accompanying drawings.

[0037] Figure 1 A block diagram of an in-vehicle intelligent armrest box based on multimodal perception fusion provided in an embodiment of the present application.

[0038] like Figure 1 As shown, the in-vehicle smart armrest box 10 based on multimodal perception fusion includes: a multimodal perception module 101, a preset control center 102, an in-vehicle smart armrest box 103 and a power management module 104.

[0039] The multi-mode sensing module 101 is used to obtain environmental information within the vehicle cabin, including information on the status of people and objects within the cabin, demand information, and seating pressure signals. The preset control center 102 is used to identify intent, infer, and make decisions based on environmental information to generate control instructions. The onboard intelligent armrest box 103 is used to control the execution of the corresponding mechanical structure according to the control instructions and to feed back the status information of the mechanical structure to the preset control center. The power management module 104 is used to supply power to the onboard intelligent armrest box.

[0040] In some embodiments, the multimodal sensing module 101 includes:

[0041] The in-cabin DMS / OMS camera is used to visually perceive people and objects in the cabin to collect information about the status of people and objects in the cabin;

[0042] A microphone array is used to capture the voices of people in the vehicle cabin, perform offline voice wake-up and online command analysis based on the voices of people in the cabin, and generate demand information;

[0043] The seat pressure sensor is used to identify whether there is an occupant in the seat and generate a seating pressure signal.

[0044] Specifically, if Figure 2As shown, the in-cabin DMS / OMS camera included in the multi-mode perception module 101 of the embodiment of the present invention is used to perceive people and objects in the cabin, including the ID of the people in the cabin, the number of people in the cabin, the seating position, appearance characteristics, gestures, clothing, handheld objects, emotional state, objects in the cabin and other in-cabin people status information and object status information; the microphone array is used to pick up sound as input for voice interaction to complete offline voice wake-up and online command analysis to generate corresponding demand information; the seat pressure sensor identifies whether there is an occupant in the seat to generate a corresponding seating pressure signal.

[0045] For example, the in-cabin DMS / OMS cameras primarily perform occupant recognition (including left and right seat occupants), Face ID recognition, personalized parameter synchronization, gesture recognition, and occupant status recognition (e.g., handheld computer, handheld document, etc.). The microphone array supports offline wakeup (e.g., "Open Office Mode") and online command parsing (e.g., "Turn the keyboard 15° to the left"). Seat pressure sensors are installed on each seat to measure their respective signals.

[0046] In some embodiments, the preset control hub includes:

[0047] Awareness recognition unit, used to identify intentions based on environmental information to obtain the user's actual intentions;

[0048] The reasoning and decision-making unit is used to perform logical reasoning on the user's actual intention to determine the intention execution mode and decide the corresponding control instructions based on the intention execution mode.

[0049] Specifically, if Figure 2 As shown, the preset control center 102 of the embodiment of the present invention supports multi-modal fusion perception of vision + voice + cabin environment + vehicle status + vehicle signal. The consciousness recognition unit is based on the accurate intention recognition capability of the large model, and performs intention recognition according to environmental information to obtain the user's actual intention, thereby realizing scene recognition, user single intention / multiple intentions and complex intention recognition under multi-modal fusion perception. The reasoning and decision-making unit performs logical reasoning based on the result of intention recognition (i.e., the user's actual intention) to obtain the intention execution mode, and decides the corresponding control instructions according to the intention execution mode, wherein the intention execution mode is any one of the office mode, meeting mode and entertainment mode.

[0050] It should be noted that the office mode links the office software and the cabin scene mode, the meeting mode links the meeting software and the cabin scene mode, and the entertainment mode links the game or ecological entertainment software and the cabin scene mode.

[0051] In some embodiments, the vehicle-mounted smart armrest box 103 includes a top-level actuator, a middle actuator, and a bottom-level actuator, wherein:

[0052] The top actuator is the mouse tray layer, which includes the mouse, the first USB port, the first wireless charging port, and the electric lifting mechanism;

[0053] The middle actuator is the keyboard drag layer, which includes a keyboard, a second USB port, a second wireless charging port, an electric worm gear mechanism, and an angle encoder;

[0054] The bottom-level actuators include cup holders, storage boxes, temperature control sensors, electric push rod mechanisms, and telescopic rails.

[0055] Specifically, if Figure 2 As shown, the top-level actuator covers a panel integrated with a mouse device, USB and wireless charging devices, and a magnetic positioning slot, wherein the device is powered by an integrated power management module 104 and expands the USB-C / TYPE-A power port and wireless charging interface (with an embedded wireless charging coil, supporting Qi protocol); the upper layer of the top-level actuator is covered with a nano-anti-slip coating (generally with a friction coefficient > 0.8); an electric lifting mechanism is provided inside the top-level actuator to realize the mechanical execution of the top layer from the retracted state to the open state.

[0056] The intermediate actuator covers a high-strength aviation aluminum substrate (bending strength ≥150GPa) and integrates a keyboard device, USB and wireless charging device. It powers the device through the integrated power management module 104 and expands the USB-C / TYPE-A power port and wireless charging interface (embedded wireless charging coil, supporting Qi protocol), and has a magnetic positioning slot; the worm gear transmission box structure supports electric control and horizontal adjustment of 180° (±90°) left and right, and its rotating axis is equipped with an encoder to record real-time adjustment information.

[0057] In addition to the storage box, the bottom actuator also includes an electric push rod and a telescopic guide rail mechanism to drive the cup holder (generally load-bearing ≥ 2kg) to extend and retract. It also integrates a temperature control module (optional semiconductor refrigeration) to achieve temperature control of the storage box.

[0058] It should be noted that the on-board intelligent armrest box 103 establishes communication with the cockpit-related domain controllers (including the body domain and the cockpit domain) through the vehicle's CAN network as a whole, displays information status and realizes cockpit scene linkage, receives control instructions from the preset control center 102 to complete the mechanical structure control execution, and synchronizes the status information of the actuator and equipment to the preset control center 102.

[0059] In some embodiments, the power management module 104 provides power management services for the electric mechanism and interface of the vehicle-mounted smart armrest box.

[0060] like Figure 3As shown, the decision-making process of the vehicle-mounted intelligent armrest box based on multimodal perception fusion proposed in the embodiment of the present invention is further explained below through a specific embodiment.

[0061] When a user enters the vehicle cabin and takes a seat, the pressure sensor senses the seating signal and sends it to the preset control center 102 to sense the seat information of the passenger; the camera recognizes the visual information in the cabin, including the ID of the current passenger, and sends it to the preset control center 102 to sense the personalized configuration information and user habits of the current passenger.

[0062] The user opens the ceiling screen and starts the conference software. The preset control center 102 senses the conditional trigger of the conference scene in the intention execution mode through the linkage with the cabin ecological information, and actively pushes the linkage control service of the in-vehicle smart armrest box to the user.

[0063] After the user authorizes control, the preset control center 102 generates control instructions for the corresponding actuators in the vehicle-mounted intelligent armrest box 103 based on the user's authorization, the personalized information of the current user ID, and the user's preferences. The confidence score of the multimodal perception decision satisfies Score = 0.4\cdot V_{det} + 0.3\cdot F_{seat} + 0.2\cdot A_{voice} + 0.1\cdot C_{cloud}, where:

[0064] V_{det}: confidence of visual recognition (0-1);

[0065] F_{seat}: confidence of the pressure sensor;

[0066] A_{voice}: the matching degree of the voice command;

[0067] C_{cloud}: The matching degree between user habits and personalized data.

[0068] Actuators such as the top, middle, and bottom layers of the vehicle's intelligent armrest are activated separately. For example, the armrest automatically unfolds the keyboard and mouse mechanical structure, visually identifies the user ID, and automatically associates the ID's structural angle parameter settings. Furthermore, it also links cockpit controls such as air conditioning, seats, ambient lighting, and windows, saving a cumulative 83 seconds of operation time each time.

[0069] Among them, such as Figure 4 As shown, when the intermediate actuator receives a rotation angle instruction from the preset control hub 102, for example, θ∈[-90°, 90°], the rotation mechanism works as follows:

[0070] Step 1: receiving a rotation angle instruction θ∈[-90°,90°] from the preset control center 102;

[0071] Step 2: The drive motor drives the worm wheel to rotate through the worm. The worm wheel is an 18-tooth polyimide gear with a tooth surface pressure angle of 22°, a tooth addendum coefficient of 0.8, and a tooth root height coefficient of 1.0. The worm is a stainless steel component with a lead angle γ∈[18°,20°], satisfying γ=arctan(L / πd) (L: lead, d: pitch circle diameter).

[0072] Step 3: The angle encoder provides real-time feedback of the rotation angle error;

[0073] Step 4: When the angle error is less than 0.5°, the drive is stopped, the self-locking solenoid valve is activated, and the keyboard is rotated to the specified position;

[0074] Step 5: Damping adaptation: During the driving process, the worm axial pressure F = k*v can be adaptively adjusted according to the vehicle speed. 2 (k:0.05Ns 2 / km 2 ).

[0075] In summary, the in-vehicle intelligent armrest box based on multimodal perception fusion proposed in the embodiments of the present application has the following beneficial effects:

[0076] (1) Built-in power and data interfaces are integrated, eliminating the need for passengers to use external devices (such as mobile power or USB hubs). The wiring is clear, enhancing the overall cabin experience and safety.

[0077] (2) It is integrated into the overall ecosystem of the AI ​​cockpit, solving the problem of system isolation and actively sensing user intentions. As a result, the application scenarios are no longer limited to physical storage. Instead, the system can automatically trigger corresponding scenarios based on user voice, behavior, and other intentions, thus achieving linkage with the overall cockpit scenario.

[0078] (3) The structure is highly flexible and can automatically trigger corresponding scenarios based on the user's voice, behavior, and other intentions, thereby meeting the flexible use needs of multiple rear-seat passengers, such as the different use needs of left and right passengers; the different support angle requirements of passengers of different heights and body shapes; and the dynamic adaptive adjustment requirements under different vehicle driving conditions, such as turning.

[0079] (4) Interactive AI intelligence, through multimodal perception fusion, improves the accuracy of user intent recognition to over 92%, and the response speed reaches within 0.8s, as shown below:

[0080] index Traditional solutions Patent for this invention Intent recognition accuracy Voice control ≤70% Multimodal fusion ≥ 92% Response speed 1.5-2s 0.3-0.8s (predicted execution) False trigger rate 15% (caused by environmental noise) ＜3％

[0081] (5) The innovative design of the three-layer mechanical structure, the middle layer rotation mechanism supports electric adjustment and ±90° wide range horizontal adjustment, which can realize the adaptive adjustment of the mechanical structure related to the vehicle status.

[0082] Next, a control method for an in-vehicle intelligent armrest box based on multimodal perception fusion proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0083] Figure 5 This is a flow chart of a control method for an in-vehicle intelligent armrest box based on multimodal perception fusion according to an embodiment of the present application.

[0084] like Figure 5 As shown, the control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion includes the following steps:

[0085] In step S501 , the environment information in the vehicle cabin is obtained, wherein the environment information includes the status information of people in the vehicle cabin, the status information of objects, demand information and a seating pressure signal.

[0086] In some embodiments, obtaining environmental information within the vehicle cabin includes:

[0087] Use the DMS / OMS camera in the cabin to visually perceive people and objects in the cabin to collect information about the status of people and objects in the cabin;

[0088] Use a microphone array to capture the voices of people in the cabin, perform offline voice wake-up and online command analysis based on the voices of people in the cabin, and generate demand information;

[0089] The seat pressure sensor detects whether there is an occupant in the seat and generates a seating pressure signal.

[0090] In step S502, intention recognition, reasoning and decision making are performed on the environmental information to generate control instructions.

[0091] In some embodiments, performing intent recognition, reasoning, and decision-making on environmental information to generate control instructions includes:

[0092] Perform intent recognition based on environmental information to obtain the user's actual intention;

[0093] Logical reasoning is performed on the user's actual intention to determine the intention execution mode, and corresponding control instructions are decided based on the intention execution mode, where the intention execution mode is any one of the office mode, meeting mode and entertainment mode.

[0094] In step S503, the corresponding mechanical structure in the vehicle-mounted intelligent armrest box is triggered according to the control instruction.

[0095] In some embodiments, triggering a corresponding mechanical structure in the vehicle-mounted smart armrest box according to a control instruction includes:

[0096] When the top-level actuator in the vehicle-mounted intelligent armrest box is triggered according to the control command, the electric lifting mechanism in the top-level actuator is used to convert the mouse, the first USB interface, and the first wireless charging interface in the top-level actuator from a retracted state to an open state or from an open state to a retracted state;

[0097] When the middle actuator in the vehicle-mounted smart armrest box is triggered according to the control command, the electric worm gear mechanism in the middle actuator is used to convert the keyboard, the second USB port, and the second wireless charging port in the middle actuator from the rolled-up state to the unfolded state or the unfolded state to the rolled-up state, and the angle encoder in the middle actuator is used to adjust the information in real time;

[0098] When the bottom actuator in the vehicle-mounted intelligent armrest box is triggered according to the control instruction, the electric push rod mechanism and telescopic guide rail in the bottom actuator are used to withdraw or retract the cup holder in the bottom actuator.

[0099] It should be noted that the above explanation of the embodiment of the in-vehicle intelligent armrest box based on multimodal perception fusion is also applicable to the control method of the in-vehicle intelligent armrest box based on multimodal perception fusion in this embodiment, and will not be repeated here.

[0100] The control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion proposed in the embodiment of the present application has the following beneficial effects:

[0101] (1) Built-in power and data interfaces are integrated, eliminating the need for passengers to use external devices (such as mobile power or USB hubs). The wiring is clear, enhancing the overall cabin experience and safety.

[0102] (2) It is integrated into the overall ecosystem of the AI ​​cockpit, solving the problem of system isolation and actively sensing user intentions. As a result, the application scenarios are no longer limited to physical storage. Instead, the system can automatically trigger corresponding scenarios based on user voice, behavior, and other intentions, thus achieving linkage with the overall cockpit scenario.

[0103] (3) The structure is highly flexible and can automatically trigger corresponding scenarios based on the user's voice, behavior, and other intentions, thereby meeting the flexible use needs of multiple rear-seat passengers, such as the different use needs of left and right passengers; the different support angle requirements of passengers of different heights and body shapes; and the dynamic adaptive adjustment requirements under different vehicle driving conditions, such as turning.

[0104] (4) Interactive AI intelligence, through multimodal perception fusion, improves the accuracy of user intent recognition to over 92%, and the response speed reaches within 0.8s;

[0105] (5) The innovative design of the three-layer mechanical structure, the middle layer rotation mechanism supports electric adjustment and ±90° wide range horizontal adjustment, which can realize the adaptive adjustment of the mechanical structure related to the vehicle status.

[0106] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0107] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0108] When the processor 602 executes the program, it implements the control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion provided in the above embodiment.

[0109] Furthermore, the electronic device further includes:

[0110] The communication interface 603 is used for communication between the memory 601 and the processor 602 .

[0111] The memory 601 is used to store computer programs that can be run on the processor 602 .

[0112] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0113] If the memory 601, processor 602, and communication interface 603 are implemented independently, the communication interface 603, memory 601, and processor 602 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0114] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0115] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0116] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the above-mentioned control method of the in-vehicle intelligent armrest box based on multimodal perception fusion.

[0117] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0118] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0119] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0120] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0121] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0122] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0123] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0124] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A vehicle-mounted intelligent armrest box based on multimodal perception fusion, characterized in that: include: A multi-mode sensing module is used to obtain environmental information in the vehicle cabin, wherein the environmental information includes occupant status information, object status information, demand information, and seating pressure signals in the vehicle cabin; A preset control center is used to perform intention recognition, reasoning and decision-making on the environmental information to generate control instructions; An on-board intelligent armrest box, used to control the execution of the corresponding mechanical structure according to the control instructions, and feed back the status information of the mechanical structure to the preset control center; A power management module is used to supply power to the vehicle-mounted intelligent armrest box.

2. The vehicle-mounted intelligent armrest box based on multimodal perception fusion according to claim 1 is characterized in that: The multimodal sensing module includes: The DMS / OMS camera in the cabin is used to visually perceive people and objects in the cabin to collect status information of people and objects in the cabin; A microphone array is used to capture the voices of people in the vehicle cabin, and perform offline voice wake-up and online command analysis based on the voices of people in the vehicle cabin to generate the demand information; The seat pressure sensor is used to identify whether there is an occupant in the seat and generate the seating pressure signal.

3. The vehicle-mounted intelligent armrest box based on multimodal perception fusion according to claim 1 is characterized in that: The preset control center includes: A consciousness recognition unit, configured to perform intention recognition based on the environmental information to obtain the user's actual intention; The reasoning and decision-making unit is used to perform logical reasoning on the actual intention of the user to determine the intention execution mode, and decide the corresponding control instructions based on the intention execution mode, wherein the intention execution mode is any one of the office mode, meeting mode and entertainment mode.

4. The vehicle-mounted intelligent armrest box based on multimodal perception fusion according to claim 1 is characterized in that: The vehicle-mounted intelligent armrest box includes a top-level actuator, a middle actuator and a bottom-level actuator, wherein: The top actuator is a mouse tray layer, which includes a mouse, a first USB port, a first wireless charging port, and an electric lifting mechanism; The intermediate actuator is a keyboard drag layer, which includes a keyboard, a second USB port, a second wireless charging port, an electric worm gear mechanism and an angle encoder; The bottom actuator includes a cup holder, a storage box, a temperature control sensor, an electric push rod mechanism and a telescopic guide rail.

5. A control method for an in-vehicle intelligent armrest box based on multimodal perception fusion, characterized in that: The vehicle-mounted intelligent armrest box based on multimodal perception fusion according to any one of claims 1 to 4 comprises the following steps: Acquiring environmental information in the vehicle cabin, wherein the environmental information includes occupant status information, object status information, demand information, and a seating pressure signal in the vehicle cabin; Performing intention recognition, reasoning, and decision-making on the environmental information to generate control instructions; The corresponding mechanical structure in the vehicle-mounted intelligent armrest box is triggered according to the control instruction.

6. The control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion according to claim 5 is characterized in that: The obtaining of the environmental information in the vehicle cabin includes: Using the DMS / OMS camera in the cabin to visually perceive people and objects in the cabin to collect status information of people and objects in the cabin; Using a microphone array to capture the voices of people in the vehicle cabin, and performing offline voice wake-up and online command analysis based on the voices of people in the vehicle cabin to generate the demand information; A seat pressure sensor is used to identify whether there is an occupant in the seat and generate the seating pressure signal.

7. The control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion according to claim 5 is characterized in that: The performing of intention recognition, reasoning, and decision-making on the environmental information to generate control instructions includes: Performing intention recognition based on the environmental information to obtain the user's actual intention; Logical reasoning is performed on the actual intention of the user to determine the intention execution mode, and corresponding control instructions are decided based on the intention execution mode, wherein the intention execution mode is any one of office mode, meeting mode and entertainment mode.

8. The control method of the vehicle-mounted intelligent armrest box based on multimodal perception fusion according to claim 7 is characterized in that: The triggering of the corresponding mechanical structure in the vehicle-mounted intelligent armrest box according to the control instruction includes: When the top-level actuator in the vehicle-mounted intelligent armrest box is triggered according to the control instruction, the electric lifting mechanism in the top-level actuator is used to convert the mouse, the first USB interface, and the first wireless charging interface in the top-level actuator from a retracted state to an open state or from an open state to a retracted state; When the intermediate actuator in the vehicle-mounted intelligent armrest box is triggered according to the control instruction, the electric worm gear mechanism in the intermediate actuator is used to convert the keyboard, the second USB interface, and the second wireless charging interface in the intermediate actuator from a rolled-up state to an unfolded state or vice versa, and the angle encoder in the intermediate actuator is used to adjust the information in real time; When the bottom actuator in the vehicle-mounted intelligent armrest box is triggered according to the control instruction, the cup holder in the bottom actuator is withdrawn or retracted by utilizing the electric push rod mechanism and the telescopic guide rail in the bottom actuator.

9. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the control method for the vehicle-mounted intelligent armrest box based on multimodal perception fusion as described in any one of claims 5 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a control method for an in-vehicle intelligent armrest box based on multimodal perception fusion as described in any one of claims 5 to 8.

Images