Method for intelligently processing driving disturbance data and related apparatus

Abstract

Embodiments of the present application provide a method for intelligently processing driving interference data and a related device, the method is applied to a vehicle terminal of a vehicle, the vehicle comprises a plurality of sensors and the vehicle terminal, the vehicle terminal comprises a controller and a center control screen, and the controller is connected with the center control screen. When detecting that a driving interference early warning function is started, multi-modal information in the vehicle is acquired through the plurality of sensors; whether a passenger in the vehicle has an interference behavior for a driver is determined according to the multi-modal information; if the interference behavior exists, a target strategy corresponding to the interference behavior is queried; and the target strategy is executed to perform a protection operation for the interference behavior. In this way, the vehicle terminal can have a safety warning capability, and at the same time, a protection measure for the interference behavior is provided in combination with the plurality of sensors in the vehicle, thereby improving the safety of driving.

Description

Technical Field

[0001] This application belongs to the technical field of data processing systems or methods specifically applicable to prediction purposes, and specifically relates to a method and related apparatus for intelligently processing driving interference data. Background Technology

[0002] Currently, during vehicle operation, passengers may interfere with the driver by making loud noises, pulling on the driver, striking the driver's seat, or grabbing the steering wheel. These situations can all cause traffic accidents, and in severe cases, even endanger lives. However, existing technology lacks protective measures to address these situations. Summary of the Invention

[0003] This application provides a method and related apparatus for intelligently processing driving interference data, aiming to improve the safety warning capability of vehicle terminals.

[0004] In a first aspect, embodiments of this application provide a method for intelligently processing driving interference data, applied to an in-vehicle terminal of a vehicle. The vehicle includes multiple sensors and the in-vehicle terminal, which includes a controller and a central control screen, the controller being connected to the central control screen. The method includes:

[0005] Driving interference warning function detected;

[0006] Multimodal information inside the vehicle is acquired through the multiple sensors, including any one or more of image information, voice information, and perception information;

[0007] Based on the multimodal information, determine whether any passengers in the vehicle are engaging in disruptive behavior towards the driver;

[0008] If the interference behavior exists, then query the target policy corresponding to the interference behavior;

[0009] The target policy is executed to perform protective operations against the interference behavior.

[0010] Secondly, embodiments of this application provide an apparatus for intelligently processing driving interference data, applied to an in-vehicle terminal of a vehicle. The vehicle includes multiple sensors and the in-vehicle terminal, which includes a controller and a central control screen, the controller being connected to the central control screen. The apparatus includes:

[0011] The detection unit is used to detect when the driving interference warning function is activated;

[0012] The acquisition unit is used to acquire multimodal information inside the vehicle through the plurality of sensors, wherein the multimodal information includes any one or more of image information, voice information, and perception information;

[0013] The determining unit is used to determine, based on the multimodal information, whether there is any interference behavior by passengers inside the vehicle targeting the driver;

[0014] The query unit is used to query the target strategy corresponding to the interference behavior when the interference behavior exists;

[0015] The processing unit is used to execute the target policy to perform protective operations against the interference behavior.

[0016] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, a communication interface, and one or more programs, the one or more programs being stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of either the first or second aspect of embodiments of this application.

[0017] Fourthly, embodiments of this application provide a computer storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in either the first or second aspect of this embodiment.

[0018] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in either the first or second aspect of the embodiments of this application. The computer program product may be a software installation package.

[0019] As can be seen from the embodiments of this application, firstly, when the driving interference warning function is detected to be activated, multimodal information inside the vehicle is acquired through multiple sensors; then, based on the multimodal information, it is determined whether there are any interfering behaviors by passengers inside the vehicle targeting the driver; if interfering behaviors exist, the target policy corresponding to the interfering behaviors is queried; finally, the target policy is executed to perform protective operations against the interfering behaviors. In this way, the in-vehicle terminal can have a safety warning capability, and at the same time, combined with multiple sensors inside the vehicle, it provides protective measures against interfering behaviors, thereby improving driving safety. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1aThis is a schematic diagram of the structure of a vehicle provided in an embodiment of this application;

[0022] Figure 1b This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating a method for intelligently processing driving interference data provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of an intelligent device for processing driving interference data provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0026] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, systems, products, or apparatuses.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] Currently, during vehicle operation, passengers may interfere with the driver by making loud noises, pulling on the driver, striking the driver's seat, or grabbing the steering wheel. These situations can all cause traffic accidents, and in severe cases, even endanger lives. However, existing technology lacks protective measures to address these situations.

[0029] To address the aforementioned issues, this application provides a method for intelligently processing driving interference data. This method can be applied to scenarios involving vehicle driving interference detection, early warning, and protection. When a driving interference warning function is detected and activated, multimodal information from within the vehicle is acquired through multiple sensors. Based on this multimodal information, it is determined whether passengers are engaging in disruptive behavior towards the driver. If such behavior is found, the corresponding target policy is queried. Finally, the target policy is executed to perform protective operations against the disruptive behavior. This solution is applicable to various scenarios, including but not limited to the aforementioned application scenarios.

[0030] The system architecture involved in the embodiments of this application is described below.

[0031] Figure 1a This is a structural schematic diagram of a vehicle 100 provided in an embodiment of this application; the vehicle 100 is equipped with an in-vehicle terminal 110 and multiple sensors installed inside the vehicle, wherein the multiple sensors are used to acquire multimodal information inside the vehicle, the in-vehicle terminal 110 includes a controller 111 and a central control screen 112, the controller 111 is connected to the central control screen 112, the central control screen 112 is used for human-computer interaction with the user to input configuration information for the driving interference warning function, and the controller 111 is used to process the multimodal information and configuration information to perform protective operations against the interference behavior.

[0032] Specifically, the multiple sensors include cameras, pressure sensors, microphones, etc., and are not considered unique here. The controller 111 is an intelligent cockpit domain controller 111, which can realize the following data processing in L1 and L2 autonomous driving: instrument panel, body control, multimedia system, engine control, as well as safety system and independent system.

[0033] This application also provides an electronic device 10, such as... Figure 1b As shown, it includes at least one processor 11, a display screen 12, and a memory 13, and may also include a communications interface 15 and a bus 14. The processor 11, display screen 12, memory 13, and communications interface 15 can communicate with each other via the bus 14. The display screen 12 is configured to display a preset user guide interface in the initial setup mode. The communications interface 15 can transmit information. The processor 11 can call logical instructions in the memory 13 to execute the methods described in the above embodiments.

[0034] Optionally, the electronic device 10 may be a mobile electronic device, an electronic device or other device, and is not limited to a single type.

[0035] Furthermore, the logic instructions in the aforementioned memory 13 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0036] The memory 13, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 11 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 13, thereby implementing the methods in the above embodiments.

[0037] The memory 13 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the electronic device 10. Furthermore, the memory 13 may include high-speed random access memory (RAM) and may also include non-volatile memory. For example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, may be used, or they may be transient storage media.

[0038] The specific methods will be described in detail below.

[0039] Figure 2 This is a flowchart illustrating a method for intelligently processing driving interference data according to an embodiment of this application. This method is applied to an in-vehicle terminal of a vehicle, which includes multiple sensors and the in-vehicle terminal. The in-vehicle terminal includes a controller and a central control screen, with the controller connected to the central control screen. The method includes:

[0040] Step 201: Driving interference warning function detected.

[0041] Step 202: Acquire multimodal information inside the vehicle through the multiple sensors.

[0042] The multimodal information includes any one or more of image information, voice information, and perception information.

[0043] Step 203: Based on the multimodal information, determine whether there are any interfering behaviors by passengers in the vehicle towards the driver.

[0044] In one possible embodiment, determining whether there is any interfering behavior from passengers in the vehicle towards the driver based on the multimodal information includes: determining the warning type that has been activated in the driving interference warning function; and performing behavior matching based on the multimodal information in a first dataset to determine whether the interfering behavior exists, wherein the first dataset is the dataset corresponding to the warning type.

[0045] In its specific implementation, the driving interference warning function includes multiple warning types, such as child interference, front passenger interference, and rear passenger interference, which can be set according to specific circumstances and are not limited to a single type. After detecting the driving interference warning function, the vehicle terminal needs to determine the warning type set by the user, and then analyze the multimodal information based on the parameter status corresponding to the warning type to determine whether the multimodal information meets the conditions of the warning type.

[0046] As can be seen, in this embodiment, multimodal information is analyzed for the warning type of the driving interference warning function, which improves the targeting of behavior matching.

[0047] Specifically, the step of performing behavior matching based on the multimodal information in the first dataset includes: if the multimodal information only includes the image information, then determining a first matching degree between each first pose in the first dataset and a second pose in the image information; if the multimodal information only includes the speech information, then determining a second matching degree between each first semantic in the first dataset and a second semantic corresponding to the speech information; if the multimodal information only includes the perception information, then determining a third matching degree between a first state in the first dataset and a second state corresponding to the perception information.

[0048] In practice, different types of warnings require different multimodal information. For example, in cases where the action directly interferes with the driver, only image information is generally needed. Therefore, in this case, the multimodal information only includes image information, and it is only necessary to determine the first matching degree between each first pose in the first dataset and the second pose in the image information. Specifically, the similarity between the first pose and the second pose is determined through image analysis, and the similarity is determined as the first matching degree.

[0049] For example, in cases where noise interferes with the driver's concentration, only voice information is generally needed. Therefore, in this case, the multimodal information only includes voice information. It is only necessary to determine the second matching degree between each first semantic in the first dataset and the second semantic corresponding to the voice information. Specifically, the matching probability is output based on the degree of closeness between the first semantic and the second semantic, and the matching probability is the second matching degree.

[0050] For example, in cases of impact or collision affecting the driver's seat, only sensory information may be needed. In such cases, the multimodal information only includes sensory information. Therefore, it is only necessary to determine the third matching degree between the first state in the first dataset and the second state corresponding to the sensory information. Specifically, by comparing multiple first pressure data in the first state with multiple second pressure data in the second state, it is determined how many of the multiple pressure data reach the threshold requirement of the second pressure data. Then, the third matching degree is calculated based on the number of data reaching the threshold requirement. For example, if there are 5 first pressure data and 3 of them reach the threshold requirement, then the third matching degree is 60%.

[0051] After the multimodal information has been matched with all the data in the first dataset, if the matching degree requirement is not met, then there is no interference behavior; if the matching degree requirement is met, then there is interference behavior.

[0052] Furthermore, the step of performing behavior matching based on the multimodal information in the first dataset further includes: if the multimodal information includes the image information, the voice information, and the perception information, then determining the target matching degree based on the first matching degree, the second matching degree, and the third matching degree; if the target matching degree is greater than a first threshold, then determining that the interfering behavior exists.

[0053] In specific implementation, within the child interference category, the multimodal information includes the image information, the voice information, and the perceptual information. In this case, after determining multiple first matching degrees, multiple second matching degrees, and multiple third matching degrees, the highest matching degree among these is identified. The number of matching degrees that meet the matching degree requirement is determined. If the preset number is reached, the interference behavior is determined to exist; otherwise, the interference behavior is determined to not exist.

[0054] As can be seen, in this embodiment, the accuracy of judging interference behavior is improved through precise matching degree calculation.

[0055] Step 204: If the interference behavior exists, query the target policy corresponding to the interference behavior.

[0056] Step 205: Execute the target strategy to perform protection operations against the interference behavior.

[0057] In one possible embodiment, executing the target strategy to perform protective operations against the interference behavior includes: when the target strategy is a first strategy, outputting a prompt message to issue a warning; when the target strategy is a second strategy, locking the interfered device or directly issuing a reminder to the interference source to prohibit input control of the device; and when the target strategy is a third strategy, applying vehicle braking.

[0058] In practice, each warning type can be graded according to the degree of interference with the driver, resulting in multiple levels. Each level is associated with a different strategy. If the interference level is low, the first strategy is executed, simply outputting a warning message. If the interference level is medium, the second strategy is executed, locking the interfered device or directly alerting the source of interference to prevent input control of the device and avoid other passengers from altering driving data or arbitrarily shifting gears via the in-vehicle terminal. If the interference level is high, vehicle braking is applied to prevent the driver from being severely interfered with and losing control of the vehicle, thus avoiding a traffic accident. Furthermore, direct alerts are issued to the source of interference.

[0059] Taking the "medium" level of interference as an example, if the target interference behavior is "kicking the driver's seat" and this behavior is not considered a "medium" level, the cockpit domain controller can control the vibration of the child's seat and simultaneously control the speakers in that seat area to play warning voice messages to remind the child to stop the interference behavior.

[0060] As can be seen, in this embodiment, different target strategies are provided to protect against the interference behavior based on different warning types, thereby improving the intelligence of the vehicle terminal.

[0061] In one possible embodiment, before the driving interference warning function is activated, the method further includes: when the driving interference warning function is detected not to be activated, displaying a first prompt message on the central control screen; obtaining a user's confirmation instruction for the first prompt message; and activating the driving interference warning function according to the confirmation instruction.

[0062] In specific implementation, when the driving interference warning function is detected as not being enabled, a first prompt message is displayed on the central control screen, such as "Activate driving interference warning function?". If a user's confirmation command is detected, the driving interference warning function is activated; if a user's rejection command is detected, the driving interference warning function is not activated. Furthermore, each time the vehicle terminal starts up, if the driving interference warning function is detected as not being enabled, the first prompt message is displayed on the central control screen, prompting the user to activate the driving interference warning function.

[0063] As can be seen, in this embodiment, when the driving interference warning function is detected as not being enabled, the user is prompted to enable the driving interference warning function, thus realizing the safety reminder function.

[0064] In one possible embodiment, the central control screen displays a settings interface, and the method further includes: obtaining a settings instruction input by the user; displaying the settings interface on the central control screen according to the settings instruction; obtaining configuration information input by the user through the settings interface; and determining the warning type of the driving interference warning function according to the configuration information.

[0065] In practice, before the user activates the driving interference warning function, the cockpit domain controller controls the central control screen to display the settings interface. This settings interface can remind the driver to set the "driving interference warning function" and simultaneously display a brief introduction to the function, such as informing the user that after the function is activated, it will detect various behaviors in the cabin that interfere with the driver's driving during driving and issue warnings. The interference behaviors can be customized by the driver or preset in multiple ways for the driver to flexibly choose from. If the driver selects the child interference category, further detailed settings options can be displayed, such as "shouting loudly", "kicking the driver's seat", "standing in the center and reaching out to touch the front control buttons of the vehicle", etc.

[0066] Understandably, users can set multiple alert types to be active simultaneously.

[0067] As can be seen, this embodiment provides users with detailed warning type settings, which improves the warning capabilities of the vehicle terminal for different scenarios.

[0068] In summary, in this embodiment, when the driving interference warning function is detected to be activated, multimodal information inside the vehicle is first acquired through multiple sensors; then, based on the multimodal information, it is determined whether passengers inside the vehicle are engaging in interfering behavior towards the driver; if interfering behavior exists, the target policy corresponding to the interfering behavior is queried; finally, the target policy is executed to perform protective operations against the interfering behavior. This enables the in-vehicle terminal to have safety warning capabilities, and, combined with multiple sensors inside the vehicle, provides protective measures against interfering behavior, thereby improving driving safety.

[0069] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, mobile electronic devices include corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0070] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0071] Please see Figure 3 This application also provides a device 30 for intelligently processing driving interference data, applied to an in-vehicle terminal of a vehicle. The vehicle includes multiple sensors and the in-vehicle terminal, which includes a controller and a central control screen, the controller being connected to the central control screen. The device 30 for intelligently processing driving interference data includes:

[0072] Detection unit 31 is used to detect when the driving interference warning function is activated;

[0073] The acquisition unit 32 is used to acquire multimodal information inside the vehicle through the plurality of sensors, wherein the multimodal information includes any one or more of image information, voice information and perception information;

[0074] The determining unit 33 is used to determine, based on the multimodal information, whether there is any interference behavior by passengers in the vehicle targeting the driver;

[0075] The query unit 34 is used to query the target strategy corresponding to the interference behavior when the interference behavior exists;

[0076] The processing unit 35 is used to execute the target strategy to perform protective operations against the interference behavior.

[0077] As can be seen from the embodiments of this application, firstly, when the driving interference warning function is detected to be activated, multimodal information inside the vehicle is acquired through multiple sensors; then, based on the multimodal information, it is determined whether there are any interfering behaviors by passengers inside the vehicle targeting the driver; if interfering behaviors exist, the target policy corresponding to the interfering behaviors is queried; finally, the target policy is executed to perform protective operations against the interfering behaviors. In this way, the in-vehicle terminal can have a safety warning capability, and at the same time, combined with multiple sensors inside the vehicle, it provides protective measures against interfering behaviors, thereby improving driving safety.

[0078] In one possible embodiment, the determining unit 33, which determines whether there is any interfering behavior by passengers in the vehicle towards the driver based on the multimodal information, is specifically configured to: determine the type of warning that has been activated in the driving interference warning function; and perform behavior matching based on the multimodal information in a first dataset to determine whether the interfering behavior exists, wherein the first dataset is the dataset corresponding to the warning type.

[0079] In one possible embodiment, regarding the aspect of performing behavior matching based on the multimodal information in the first dataset, the determining unit 33 is specifically configured to: if the multimodal information only includes the image information, determine a first matching degree between each first pose in the first dataset and a second pose in the image information; if the multimodal information only includes the speech information, determine a second matching degree between each first semantic in the first dataset and a second semantic corresponding to the speech information; if the multimodal information only includes the perception information, determine a third matching degree between a first state in the first dataset and a second state corresponding to the perception information.

[0080] In one possible embodiment, the aspect of performing behavior matching based on the multimodal information in the first dataset, the determining unit 33 is specifically configured to: if the multimodal information includes the image information, the voice information, and the perception information, determine the target matching degree based on the first matching degree, the second matching degree, and the third matching degree; if the target matching degree is greater than a first threshold, determine that the interfering behavior exists.

[0081] In one possible embodiment, before the driving interference warning function is activated, the device further includes: a display unit, configured to display a first prompt message on the central control screen when the driving interference warning function is detected not being activated; the acquisition unit 32 is further configured to: acquire a user's confirmation instruction for the first prompt message; and a activation unit, configured to activate the driving interference warning function according to the confirmation instruction.

[0082] In one possible embodiment, the central control screen displays a settings interface, and the device further includes: the acquisition unit 32 is further configured to: acquire settings instructions input by the user; the display unit is further configured to: display the settings interface on the central control screen according to the settings instructions; the acquisition unit 32 is further configured to: acquire configuration information input by the user through the settings interface; and the determination unit 33 is further configured to: determine the warning type of the driving interference warning function according to the configuration information.

[0083] In one possible embodiment, the processing unit 35 is specifically configured to: when the target strategy is a first strategy, output a warning message; when the target strategy is a second strategy, lock the interfered device or directly remind the interference source to prohibit input control of the device; and when the target strategy is a third strategy, apply vehicle braking.

[0084] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0085] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0086] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0087] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0089] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0091] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, volatile memory, or non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM), etc., various media capable of storing program code.

[0092] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions without departing from the spirit and scope of the present invention, and various modifications and alterations can be made, including combinations of the different functions and implementation steps described above, as well as software and hardware implementation methods, all of which are within the protection scope of the present invention.

Claims

1. A method for intelligently processing driving interference data, characterized in that, An in-vehicle terminal for use in a vehicle, the vehicle including multiple sensors and the in-vehicle terminal, the in-vehicle terminal including a controller and a central control screen, the controller being connected to the central control screen; the method includes: Driving interference warning function detected; Multimodal information inside the vehicle is acquired through the multiple sensors, including any one or more of image information, voice information, and perception information; Based on the multimodal information, determine whether any passengers in the vehicle are engaging in disruptive behavior towards the driver; If the interference behavior exists, then query the target policy corresponding to the interference behavior; The target strategy is executed to perform protective operations against the interference behavior; The step of determining whether passengers in the vehicle are engaging in disruptive behavior towards the driver based on the multimodal information includes: determining the type of warning that has been activated in the driving interference warning function; and performing behavior matching based on the multimodal information in a first dataset to determine whether the disruptive behavior exists, wherein the first dataset is the dataset corresponding to the warning type. The step of performing behavior matching based on the multimodal information in the first dataset includes: if the multimodal information only includes the image information, then determining a first matching degree between each first pose in the first dataset and a second pose in the image information; if the multimodal information only includes the speech information, then determining a second matching degree between each first semantic in the first dataset and a second semantic corresponding to the speech information; if the multimodal information only includes the perception information, then determining a third matching degree between a first state in the first dataset and a second state corresponding to the perception information; if the multimodal information includes the image information, the speech information, and the perception information, then determining a target matching degree based on the first matching degree, the second matching degree, and the third matching degree; if the target matching degree is greater than a first threshold, then determining that the interfering behavior exists. The execution of the target strategy to perform protective operations against the interference behavior includes: when the target strategy is a first strategy, outputting a prompt message to issue a warning; when the target strategy is a second strategy, locking the interfered device or directly issuing a reminder to the interference source to prohibit input control of the device; and when the target strategy is a third strategy, applying vehicle braking.

2. The method according to claim 1, characterized in that, Before the driving interference warning function is activated, the method further includes: When it is detected that the driving interference warning function is not activated, a first prompt message is displayed on the central control screen; Obtain the user's confirmation instruction regarding the first prompt message; The driving interference warning function is activated according to the confirmation command.

3. The method according to claim 1, characterized in that, The central control screen displays a settings interface; the method further includes: Get the user's input settings instructions; The settings interface is displayed on the central control screen according to the settings instructions; Obtain the configuration information input by the user through the settings interface; The warning type of the driving interference warning function is determined based on the configuration information.

4. A device for intelligently processing driving interference data, characterized in that, An in-vehicle terminal for use in a vehicle, the vehicle including multiple sensors and the in-vehicle terminal, the in-vehicle terminal including a controller and a central control screen, the controller being connected to the central control screen; the device includes: The detection unit is used to detect when the driving interference warning function is activated; The acquisition unit is used to acquire multimodal information inside the vehicle through the plurality of sensors, wherein the multimodal information includes any one or more of image information, voice information, and perception information; A determining unit is configured to determine whether in-vehicle passengers are engaging in disruptive behavior towards the driver based on the multimodal information, including: determining the activated warning type in the driving interference warning function; performing behavior matching based on the multimodal information in a first dataset to determine whether the disruptive behavior exists, wherein the first dataset is the dataset corresponding to the warning type; wherein, if the multimodal information only includes the image information, a first matching degree is determined between each first pose in the first dataset and a second pose in the image information; if the multimodal information only includes the voice information, a second matching degree is determined between each first semantic in the first dataset and a second semantic corresponding to the voice information; if the multimodal information only includes the perception information, a third matching degree is determined between a first state in the first dataset and a second state corresponding to the perception information; if the multimodal information includes the image information, the voice information, and the perception information, a target matching degree is determined based on the first matching degree, the second matching degree, and the third matching degree; if the target matching degree is greater than a first threshold, the disruptive behavior is determined to exist; the first dataset is the dataset corresponding to the warning type. The query unit is used to query the target strategy corresponding to the interference behavior when the interference behavior exists; The processing unit is configured to execute the target strategy to perform protective operations against the interference behavior, including: when the target strategy is a first strategy, outputting a prompt message to issue a warning; when the target strategy is a second strategy, locking the interfered device or directly issuing a reminder to the interference source to prohibit input control of the device; and when the target strategy is a third strategy, applying vehicle braking.

5. An electronic device, characterized in that, The method includes a processor, a memory, a communication interface, and one or more programs, said one or more programs being stored in the memory and configured to be executed by the processor, said programs including instructions for performing the steps of the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange is provided, wherein the computer program causes a computer to execute instructions for the steps of the method as described in any one of claims 1-3.

Images