Multimedia information processing method, device, electronic device and storage medium

Abstract

The present invention discloses a multimedia information processing method, apparatus, electronic device, and storage medium. The method comprises: acquiring multimedia information; processing the multimedia information to obtain a processing result; and transmitting the multimedia information and the processing result to a vehicle's cabin domain controller, where the cabin domain controller determines and displays a control result based on the multimedia information and the processing result. The present invention utilizes a split design of an intelligent coprocessor and a cabin domain controller, improving the efficiency of multimedia information processing at a lower chip cost, thereby meeting the growing number of human-computer interaction application scenarios.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a multimedia information processing method, device, electronic device and storage medium. Background Art

[0002] In-vehicle intelligent cockpit systems serve as the gateway to human-vehicle interaction, providing users with an intelligent and safe driving experience. With the advancement of visual AI technology, intelligent cockpit domain controllers (DCs) are increasingly equipped with functions based on external cameras, including driver monitoring, passenger monitoring, 360-degree perception, and gesture recognition. These increasingly diverse functions place increasing demands on the processing power of DCs. Currently, the development of integrated DCs has failed to keep pace with the rapid increase in functional requirements. Summary of the Invention

[0003] In order to solve the problems of the prior art, the embodiments of the present invention provide a multimedia information processing method, device, electronic device and storage medium. The technical solution is as follows:

[0004] In one aspect, a multimedia information processing method is provided, which is applied to an intelligent coprocessor of a vehicle, and the method includes:

[0005] Acquire multimedia information collected by the vehicle-mounted multimedia acquisition module;

[0006] Processing multimedia information to obtain processing results;

[0007] The multimedia information and processing results are sent to the vehicle's cockpit domain controller, which determines and displays the control results based on the multimedia information and processing results.

[0008] In another aspect, a multimedia information processing device is provided, which is applied to an intelligent coprocessor of a vehicle, and the device includes:

[0009] An acquisition module, used to acquire multimedia information collected by the vehicle-mounted multimedia acquisition module;

[0010] A processing module, used for processing multimedia information and obtaining processing results;

[0011] The sending module is used to send multimedia information and processing results to the vehicle's cockpit domain controller, which determines and displays the control results based on the multimedia information and processing results.

[0012] In an exemplary embodiment, the processing module is specifically configured to:

[0013] The multimedia information is input into the intent recognition model for intent recognition processing to obtain the intent recognition result.

[0014] In an exemplary embodiment, the multimedia information includes video information and / or audio information, and the acquisition module includes:

[0015] A video receiving module is used to receive video information collected by an external camera module;

[0016] The audio receiving module is used to receive audio information collected by the external microphone module; wherein the external camera module and the external microphone module are respectively connected to the vehicle-mounted multimedia collection module.

[0017] In an exemplary embodiment, the sending module is specifically configured to:

[0018] Send video information to the video receiving module in the cockpit domain controller through the preset video channel;

[0019] The audio information and processing results are sent to the cockpit domain controller through the first transmission interface.

[0020] On the other hand, another multimedia information processing method is provided, which is applied to a cockpit domain controller of a vehicle, and the method includes:

[0021] Receive multimedia information and processing results sent by the vehicle's intelligent coprocessor;

[0022] Determine and display the control result based on the multimedia information and the processing result; wherein the processing result is obtained by the intelligent coprocessor obtaining the multimedia information collected by the vehicle-mounted multimedia acquisition module and processing the multimedia information.

[0023] On the other hand, another multimedia information processing device is provided, which is applied to a cockpit domain controller of a vehicle, and the device includes:

[0024] A receiving module, used to receive multimedia information and processing results sent by the vehicle's intelligent coprocessor;

[0025] The determination module is used to determine and display the control result based on the multimedia information and the processing result; wherein the processing result is obtained by the intelligent coprocessor obtaining the multimedia information collected by the vehicle-mounted multimedia acquisition module and processing the multimedia information.

[0026] In an exemplary embodiment, the multimedia information includes video information and / or audio information, and the receiving module is specifically configured to:

[0027] Receive video information sent by the intelligent coprocessor through the preset video channel through the video channel docking module;

[0028] The audio information and processing result sent by the intelligent coprocessor through the first transmission interface are received through the second transmission interface.

[0029] In an exemplary embodiment, the determining module includes:

[0030] The parsing module is used to parse the multimedia information and the processing result according to a preset format to obtain the parsed multimedia information and the processing result;

[0031] A control result generating module, used for generating a control result according to the parsed multimedia information and the processing result;

[0032] Display module, used to display control results.

[0033] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the above-mentioned multimedia information processing method.

[0034] On the other hand, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored. The at least one instruction or the at least one program is loaded and executed by a processor to implement the multimedia information processing method as described above.

[0035] In another aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the multimedia information processing method provided in the above aspects.

[0036] In an embodiment of the present invention, an intelligent coprocessor applied to a vehicle acquires multimedia information collected by an onboard multimedia acquisition module; processes the multimedia information to obtain a processing result; and transmits the multimedia information and the processing result to the vehicle's cockpit domain controller. The cockpit domain controller determines and displays a control result based on the multimedia information and the processing result. By connecting the intelligent coprocessor to the existing cockpit domain controller, the cockpit domain controller's insufficient computing power for multimedia information can be compensated without changing the cockpit domain controller chip platform design, thereby achieving flexible expansion of the cockpit domain controller's computing power. This split design achieves the requirements for multimedia functions that require computing processing power at a lower chip cost and chip capability, expands the application scenarios of human-computer interaction, and enriches the passenger experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 Schematic diagram of the architecture of a multimedia information processing method provided by an embodiment of the present invention;

[0039] Figure 2 This is a flowchart of a multimedia information processing method provided by an embodiment of the present invention;

[0040] Figure 3 is a flowchart of another multimedia information processing method provided by an embodiment of the present invention;

[0041] Figure 4 is a flowchart of another multimedia information processing method provided by an embodiment of the present invention;

[0042] Figure 5 This is a structural block diagram of a multimedia information processing device provided by an embodiment of the present invention;

[0043] Figure 6 is a structural block diagram of another multimedia information processing device provided by an embodiment of the present invention;

[0044] Figure 7 This is a hardware structure block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] See also Figure 1 , which is a schematic diagram of the architecture of a multimedia information processing method provided by an embodiment of the present invention, including a vehicle multimedia information processing system 100 and a vehicle display 110. The vehicle multimedia information processing system 100 includes an intelligent coprocessor 500 and a cockpit domain controller 600.

[0048] The intelligent coprocessor 500 (coprocessor) is a chip that assists the central processing unit (CPU) in completing tasks that it cannot perform, or performs inefficiently or poorly. Tasks that the CPU cannot perform include signal transmission between devices and management of connected devices; tasks that the CPU cannot perform efficiently or poorly include graphics processing and audio processing.

[0049] The cockpit domain controller 600 supports cockpit applications such as driver monitoring, passenger monitoring, and the integration of visual perception and audio recognition. It also provides excellent display performance, supporting single-chip multi-screen displays to meet the display needs of instrument screens and central control screens of various sizes.

[0050] As cockpit domain controllers gain more and more functionality based on external cameras, their computing power must be increased accordingly. This embodiment of the present invention adds an intelligent coprocessor to the existing cockpit domain controller, eliminating the need to develop a new cockpit domain controller chip platform. This reduces the computing power deficit of the cockpit domain controller at a lower chip cost, flexibly expanding the computing power of the cockpit domain controller.

[0051] See also Figure 2 , which is a flowchart of a multimedia information processing method provided by an embodiment of the present invention, which can be applied to Figure 1The intelligent coprocessor 500 of the vehicle multimedia information processing system 100. It should be noted that this specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on conventional or non-creative work. The order of steps listed in the embodiments is only one way of executing the steps among many steps, and does not represent the only execution order. When the actual system or product is executed, it can be executed sequentially or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0052] S201: Acquire multimedia information collected by the vehicle-mounted multimedia collection module.

[0053] S203: Process the multimedia information to obtain a processing result.

[0054] As an optional implementation, S203 includes:

[0055] Multimedia information is input into the intent recognition model for intent recognition processing to obtain the intent recognition result. Among them, intent recognition models include rule-based, traditional machine learning algorithm-based, and deep learning algorithm-based models. Traditional machine learning algorithms include Deep Belief Networks (DBN) and Support Vector Machines (SVM); deep learning algorithms include Convolutional Neural Networks (CNN), Long Short-Term Memory Networks (LSTM), and Bidirectional Recurrent Neural Networks (Bi-RNN).

[0056] For example, when the multimedia information is a video of a driver yawning continuously, the multimedia information is input into the intent recognition model for intent recognition processing, and the intent recognition result is obtained, that is, the processing result: the driver is currently driving fatigued. Similarly, when the multimedia information is a video of a driver making a phone call, the intent recognition result (processing result) is obtained: the driver is currently making a phone call. Similarly, when the multimedia information is a video of a passenger pointing at the right exterior rearview mirror, making a gesture to rotate it closer to the car body, and emitting an audio of "turn that inward," the intent recognition result (processing result) is obtained: the passenger wants to rotate the right exterior rearview mirror closer to the car body.

[0057] Applying the intent recognition model to the intelligent coprocessor, combined with the collected multimedia information, can more accurately understand the user's needs. For example, in the above example of rotating the right exterior mirror, the pronoun "that" in the audio is not clear enough to refer to the content, making it difficult to determine the user's needs based solely on the audio. However, because the user's gestures are also collected, the user's intention can be accurately determined: they want the right exterior mirror to rotate closer to the car body.

[0058] For example, even when audio processing alone is sufficient, users may express multiple intents. For example, a passenger might say, "Play Farewell My Concubine," but they don't specify whether they want the movie or the song. Therefore, a specialized intent recognition model is needed to accurately meet user needs. Of course, different intent recognition models have varying accuracy, so the specific model or models used can be adjusted based on the specific situation.

[0059] Based on multimedia information and intent recognition models, the intelligent coprocessor accurately obtains the user's intention and sends the processing result to the cockpit domain controller, making it easier for the subsequent cockpit domain controller to perform its work, thereby achieving the ideal effect of human-computer interaction more smoothly.

[0060] S205: The multimedia information and processing results are sent to the vehicle's cockpit domain controller, which determines and displays the control result based on the multimedia information and processing results.

[0061] Continuing with the example of a driver yawning continuously, after obtaining the processing results, the intelligent coprocessor simultaneously sends the yawning video and the processing result that "the driver is currently driving fatigued" to the cockpit domain controller. Similarly, in the example of the driver making a phone call, the intelligent coprocessor simultaneously sends the phone call video and the processing result that "the driver is currently making a phone call" to the cockpit domain controller. Similarly, in the example of rotating the exterior mirror, the intelligent coprocessor simultaneously sends the video of the passenger's gesture changes, the audio of the passenger's request, and the processing result that "the passenger wants the right exterior mirror to rotate closer to the car body" to the cockpit domain controller.

[0062] The embodiment of the present invention utilizes an intelligent coprocessor to obtain multimedia information and process and send the multimedia information. Based on the existing cockpit domain controller, the computing power of the overall system is expanded without developing a new cockpit domain controller chip platform, thereby greatly meeting the increasingly rich multimedia function requirements.

[0063] In an exemplary embodiment, the multimedia information includes video information and / or audio information. The above step S201 of acquiring the multimedia information collected by the vehicle-mounted multimedia collection module may include the following steps:

[0064] Receiving video information collected by an external camera module; and / or

[0065] Receives audio information collected by an external microphone module.

[0066] Among them, the peripheral camera module and the peripheral microphone module are respectively connected to the vehicle-mounted multimedia acquisition module.

[0067] In the embodiment of the present invention, it is preferred that there are at least two camera modules and microphone modules, so that the multimedia information obtained by the intelligent coprocessor can be diverse and accurate.

[0068] In an exemplary embodiment, the above step S205 of sending the multimedia information and processing results to the vehicle's cockpit domain controller may include the following steps:

[0069] Sending video information to the video receiving module in the cockpit domain controller through a preset video channel; and / or

[0070] The audio information and processing results are sent to the cockpit domain controller through the first transmission interface.

[0071] In practical applications, the preset video channel can transmit low-voltage differential signaling (LVDS); the first transmission interface can be a USB interface or an Ethernet interface.

[0072] The embodiment of the present invention "isolates" video information from other information and sends them exclusively through a preset video channel, thereby improving transmission efficiency.

[0073] See also Figure 3 , which is a flow chart of another multimedia information processing method provided by an embodiment of the present invention, which can be applied to Figure 1 The cockpit domain controller 600 of the vehicle multimedia information processing system 100. Specifically, the method may include:

[0074] S301: Receive multimedia information and processing results sent by the vehicle's intelligent coprocessor.

[0075] S303: Determine and display a control result based on the multimedia information and the processing result.

[0076] Among them, the processing result is obtained by the intelligent coprocessor obtaining the multimedia information collected by the on-board multimedia acquisition module and processing the multimedia information. In the aforementioned example of the driver yawning continuously, after the cockpit domain processor receives the multimedia information sent by the intelligent coprocessor (the video of the yawning) and the processing result of "the driver is currently driving fatigued", it determines the control result: remind the passenger to rest. In the aforementioned example of the driver making a phone call, after the cockpit domain processor receives the multimedia information sent by the intelligent coprocessor (the video of the phone call) and the processing result of "the driver is currently making a phone call", it determines the control result: remind the passenger to pull over and make a phone call. In the aforementioned example of rotating the exterior rearview mirror, after the cockpit domain processor receives the multimedia information sent by the intelligent coprocessor (the video of the passenger's gestures, the audio of the passenger's request) and the processing result of "the passenger wants the right exterior rearview mirror to rotate closer to the vehicle body", it determines the control result: control the right exterior rearview mirror to rotate closer to the vehicle body.

[0077] Compared with the cockpit domain controller that has to shoulder all the "heavy responsibilities" of acquiring multimedia information, processing multimedia information, and determining and displaying control results, the embodiment of the present invention reduces the "burden" on the cockpit domain controller with the help of an intelligent coprocessor without the need to develop a new cockpit domain controller chip platform, making the entire multimedia information processing system efficient and flexible.

[0078] In an exemplary embodiment, the multimedia information includes video information and / or audio information. The above step S301 of receiving the multimedia information and processing results sent by the intelligent coprocessor of the vehicle may include the following steps:

[0079] Receive video information sent by the intelligent coprocessor through the preset video channel through the video channel docking module;

[0080] The audio information and processing result sent by the intelligent coprocessor through the first transmission interface are received through the second transmission interface.

[0081] In actual applications, the video channel docking module can receive video information transmitted by the preset video channel in the form of a low-voltage differential signal; the second transmission interface is the existing interface of the cockpit domain controller, which can be a USB interface or an Ethernet interface, as long as it is the same type as the first transmission interface.

[0082] The embodiment of the present invention "isolates" video information from other information. Video information is received through the video channel docking module, and other information is received through the second transmission interface, which can improve transmission efficiency and save total computing and processing time.

[0083] In an exemplary embodiment, the above step S303 of determining and displaying the control result based on the multimedia information and the processing result may include the following steps:

[0084] Parsing the multimedia information and the processing result respectively according to a preset format to obtain the parsed multimedia information and the processing result;

[0085] Generate control results based on the parsed multimedia information and processing results;

[0086] Display control results.

[0087] The multimedia information and processing results sent by the intelligent coprocessor cannot directly achieve human-computer interaction. They must first be parsed into data formats that meet human-computer interaction requirements. Then, control results are generated based on the parsed data formats, and then the control results are displayed. Specifically, the cockpit domain controller sends the control results to a display device, including the instrument display, central control display, and passenger display.

[0088] Thanks to the intelligent coprocessor, the cockpit domain controller in this embodiment of the present invention only performs three computational tasks: 1. receiving and parsing data; 2. generating control results; and 3. displaying control results. This simplified computational processing improves the efficiency of human-computer interaction.

[0089] In order to improve the accuracy of multimedia information processing, in an exemplary embodiment, the multimedia information processing method applicable to the vehicle cockpit domain controller may further include:

[0090] Get vehicle status information;

[0091] The vehicle status is controlled based on the status information.

[0092] Vehicle status information includes engine shutdown status. Continuing with the aforementioned example of a driver making a phone call, if the engine remains off for half an hour, it indicates the vehicle has not been driven, eliminating the need to remind the driver not to make phone calls while driving. This allows the cockpit domain controller to comprehensively consider vehicle status information when generating control decisions, reducing the probability of errors.

[0093] See also Figure 4 , which is a flow chart of another multimedia information processing method provided by an embodiment of the present invention, the method comprising:

[0094] S401: The intelligent coprocessor obtains multimedia information collected by the vehicle-mounted multimedia collection module;

[0095] S403: The intelligent coprocessor processes the multimedia information and obtains a processing result;

[0096] S405: The intelligent coprocessor sends the multimedia information and processing results to the vehicle's cockpit domain controller;

[0097] S407: The cockpit domain controller receives the multimedia information and processing results sent by the vehicle's intelligent coprocessor;

[0098] S409: The cockpit domain controller determines and displays the control result based on the multimedia information and processing results.

[0099] The specific implementation of steps S401 to S409 can be found in the aforementioned embodiments of the present disclosure. Figure 2 and Figure 3 The relevant steps in the illustrated method embodiment will not be repeated here.

[0100] Corresponding to the multimedia information processing methods provided in the above-mentioned embodiments, the embodiments of the present invention also provide several multimedia information processing devices. Since the multimedia information processing devices provided in the embodiments of the present invention correspond to the multimedia information processing methods provided in the above-mentioned embodiments, the implementation methods of the aforementioned multimedia information processing methods are also applicable to the multimedia information processing devices provided in this embodiment and will not be described in detail in this embodiment.

[0101] See also Figure 5 , which shows a schematic diagram of the structure of a multimedia information processing device provided by an embodiment of the present invention, and the multimedia information processing device is applied to the intelligent coprocessor 500 of the vehicle. The multimedia information processing device has the function of implementing the multimedia information processing method in the above method embodiment, and the function can be implemented by hardware or by hardware executing corresponding software. Figure 5 As shown, the multimedia information processing device may include:

[0102] An acquisition module 510 is used to acquire multimedia information collected by the vehicle-mounted multimedia collection module;

[0103] The processing module 520 is used to process the multimedia information and obtain a processing result;

[0104] The sending module 530 is used to send the multimedia information and processing results to the vehicle's cockpit domain controller, and the cockpit domain controller determines and displays the control results based on the multimedia information and processing results.

[0105] In an exemplary embodiment, the processing module 520 is specifically configured to:

[0106] The multimedia information is input into the intent recognition model for intent recognition processing to obtain the intent recognition result.

[0107] In an exemplary embodiment, the multimedia information includes video information and / or audio information, and the acquisition module 510 may include:

[0108] The video receiving module 5101 is used to receive video information collected by the external camera module 5102;

[0109] The audio receiving module 5103 is used to receive audio information collected by the external microphone module 5104.

[0110] Among them, the external camera module 5102 and the external microphone module 5104 are respectively connected to the vehicle-mounted multimedia acquisition module.

[0111] In an exemplary embodiment, the sending module 530 is specifically configured to:

[0112] The video information is sent to the video receiving module in the cockpit domain controller 600 through the preset video channel 5301;

[0113] The audio information and processing results are sent to the cockpit domain controller 600 through the first transmission interface 5302 .

[0114] The multimedia information processing device of the embodiment of the present invention realizes the reception and intent recognition processing of multimedia information, and sends the multimedia information and processing results to the cockpit domain controller. This split arrangement shortens the subsequent cockpit domain controller calculation and processing time without the need for a newly developed cockpit domain controller chip platform, and flexibly expands the multimedia information processing function.

[0115] See also Figure 6 , which shows a schematic diagram of the structure of another multimedia information processing device provided by an embodiment of the present invention, and the multimedia information processing device is applied to the cockpit domain processor 600 of the vehicle. The multimedia information processing device has the function of implementing the multimedia information processing method in the above method embodiment, and the function can be implemented by hardware or by hardware executing corresponding software. Figure 6 As shown, the multimedia information processing device may include:

[0116] The receiving module 601 is used to receive multimedia information and processing results sent by the vehicle's intelligent coprocessor 500;

[0117] The determination module 602 is used to determine and display the control result based on the multimedia information and the processing result; wherein the processing result is obtained by the intelligent coprocessor 500 obtaining the multimedia information collected by the vehicle-mounted multimedia collection module and processing the multimedia information.

[0118] In an exemplary embodiment, the multimedia information includes video information and / or audio information, and the receiving module is specifically configured to:

[0119] Receive video information sent by the intelligent coprocessor 500 through a preset video channel through the video channel docking module 6011;

[0120] The second transmission interface 6012 receives the audio information and processing results sent by the intelligent coprocessor 500 through the first transmission interface.

[0121] In an exemplary embodiment, the determining module includes:

[0122] Parsing module 6021, used to parse the multimedia information and processing results according to a preset format to obtain the parsed multimedia information and processing results;

[0123] A control result generating module 6022 is used to generate a control result based on the parsed multimedia information and the processing result;

[0124] The display module 6023 is used to display the control results.

[0125] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0126] The multimedia information processing device of the embodiment of the present invention achieves the result of human-computer interaction based on the existing transmission channel and intelligent coprocessor computing and processing without changing the existing chip platform design, avoiding the situation where the computing and processing capabilities of multimedia information are not matched with the increasing richness of multimedia information.

[0127] An embodiment of the present invention provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement any multimedia information processing method provided in the above method embodiments.

[0128] The memory can be used to store software programs and modules. The processor executes the software programs and modules stored in the memory to perform various functional applications and monitor power anomalies. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data generated based on the use of the device. In addition, the memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0129] The method embodiments provided in the embodiments of the present invention can be executed in a computer terminal, server, or similar computing device. The embodiments of the present invention provide a terminal comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the processor loads and executes the at least one instruction, at least one program, or the code set or instruction set to implement the multimedia information processing method provided in the above method embodiments.

[0130] The memory can be used to store software programs and modules. The processor executes various functional applications and multimedia information processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. In addition, the memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0131] The method embodiments provided in the embodiments of the present invention can be executed in a computer terminal, a server or a similar computing device. Taking running on a terminal as an example, Figure 7 This is a hardware structure block diagram of a terminal running a multimedia information processing method provided by an embodiment of the present invention. Specifically:

[0132] The terminal may include components such as an RF (Radio Frequency) circuit 710, a memory 720 including one or more computer-readable storage media, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a WiFi (wireless fidelity) module 770, a processor 780 including one or more processing cores, and a power supply 790. It will be understood by those skilled in the art that Figure 7 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0133] The RF circuit 710 can be used to receive and send signals during information transmission or calls. Specifically, it receives downlink information from the base station and transmits it to one or more processors 780 for processing. Furthermore, it transmits uplink data to the base station. Typically, the RF circuit 710 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, and the like. Furthermore, the RF circuit 710 can communicate with the network and other terminals via wireless communication. Such wireless communication can utilize any communication standard or protocol, including but not limited to GSM (Global System of Mobile Communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), and the like.

[0134] The memory 720 can be used to store software programs and modules. The processor 780 executes various functional applications and data processing by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, the memory 720 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 720 may also include a memory controller to provide access to the memory 720 by the processor 780 and the input unit 730.

[0135] The input unit 730 can be used to receive digital or character input and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control. Specifically, the input unit 730 may include a touch-sensitive surface 731 and other input devices 732. The touch-sensitive surface 731, also known as a touch display or touchpad, can detect user touch operations on or near it (for example, operations performed by a user using a finger, stylus, or any other suitable object or accessory on or near the touch-sensitive surface 731) and drive corresponding connected devices according to a pre-set program. Optionally, the touch-sensitive surface 731 may include a touch detection device and a touch controller. The touch detection device detects the user's touch position and detects signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 780. It can also receive and execute commands from the processor 780. In addition, the touch-sensitive surface 731 can be implemented using various types of touch devices, including resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 731, the input unit 730 may further include other input devices 732. Specifically, the other input devices 732 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, and a joystick.

[0136] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the terminal. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 740 may include a display panel 741. Optionally, the display panel 741 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like. Furthermore, the touch-sensitive surface 731 can cover the display panel 741. When the touch-sensitive surface 731 detects a touch operation on or near it, it transmits the information to the processor 780 to determine the type of touch event. The processor 780 then provides a corresponding visual output on the display panel 741 based on the type of touch event. Among them, the touch-sensitive surface 731 and the display panel 741 can be two independent components to implement input and output functions. However, in some embodiments, the touch-sensitive surface 731 and the display panel 741 can also be integrated to implement input and output functions.

[0137] The terminal may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 741 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 741 and / or the backlight when the terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the terminal posture (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the terminal may also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0138] Audio circuit 760, speaker 761, and microphone 762 provide an audio interface between the user and the terminal. Audio circuit 760 can convert received audio data into electrical signals and transmit them to speaker 761, which then converts them into sound signals for output. Microphone 762, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 760 and converted into audio data. The audio data is then processed by output processor 780 and transmitted via RF circuit 710 to, for example, another terminal. Alternatively, the audio data can be output to memory 720 for further processing. Audio circuit 760 may also include an earphone jack to allow communication between an external headset and the terminal.

[0139] WiFi is a short-range wireless transmission technology. The terminal can help users send and receive emails, browse web pages and access streaming media through the WiFi module 770. It provides users with wireless broadband Internet access. Figure 7 A WiFi module 770 is shown, but it is understandable that it is not an essential component of the terminal and can be omitted as needed without changing the essence of the invention.

[0140] Processor 780 is the terminal's control center, connecting various components of the terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 720 and accessing data stored in memory 720, it performs various terminal functions and processes data, thereby providing overall terminal monitoring. Optionally, processor 780 may include one or more processing cores; preferably, processor 780 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 780.

[0141] The terminal also includes a power supply 790 (e.g., a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 780 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 790 can also include any of one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other components.

[0142] Although not shown, the terminal may also include a camera, a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for executing the multimedia information processing provided by the above method embodiment.

[0143] An embodiment of the present invention also provides a computer-readable storage medium, which can be set in a terminal to store at least one instruction, at least one program, code set or instruction set related to implementing a multimedia information processing method. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the multimedia information processing method provided by the above method embodiment.

[0144] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0145] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0146] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0147] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimedia information processing method, characterized in that: An intelligent coprocessor applied to a vehicle, the method comprising: Acquire multimedia information collected by the vehicle-mounted multimedia acquisition module; Processing the multimedia information to obtain a processing result; Sending the multimedia information and the processing result to a cockpit domain controller of the vehicle; The cockpit domain controller parses the multimedia information and the processing results according to a preset format to obtain the parsed multimedia information and the processing results; generates a control result based on the parsed multimedia information and the processing results; and displays the control result. The cockpit domain controller and the intelligent coprocessor are designed as split units.

2. The multimedia information processing method according to claim 1, wherein: The processing of the multimedia information to obtain a processing result includes: The multimedia information is input into the intent recognition model for intent recognition processing to obtain an intent recognition result.

3. The multimedia information processing method according to claim 1, wherein: The multimedia information includes video information and / or audio information; The acquiring of multimedia information collected by the vehicle-mounted multimedia collection module includes: Receive video information collected by an external camera module; and / or receive audio information collected by an external microphone module; wherein the external camera module and the external microphone module are respectively connected to the vehicle-mounted multimedia acquisition module.

4. The multimedia information processing method according to claim 3, wherein: The sending of the multimedia information and the processing result to the cockpit domain controller of the vehicle includes: Sending the video information to the video receiving module in the cockpit domain controller through a preset video channel; and / or The audio information and the processing result are sent to the cockpit domain controller through the first transmission interface.

5. A multimedia information processing method, characterized in that: Applied to a vehicle cockpit domain controller, the method includes: Receiving multimedia information and processing results sent by the vehicle's intelligent coprocessor; the processing results are obtained by the intelligent coprocessor obtaining multimedia information collected by the vehicle-mounted multimedia acquisition module and processing the multimedia information; Parsing the multimedia information and the processing result respectively according to a preset format to obtain the parsed multimedia information and the processing result; generating a control result according to the parsed multimedia information and the processing result; Display control results; the cockpit domain controller and the intelligent coprocessor are split designs.

6. The multimedia information processing method according to claim 5, characterized in that: The multimedia information includes video information and / or audio information; The multimedia information and processing results sent by the intelligent coprocessor of the receiving vehicle include: Receiving, through a video channel docking module, the video information sent by the intelligent coprocessor through a preset video channel; The audio information and the processing result sent by the intelligent coprocessor through the first transmission interface are received through the second transmission interface.

7. A multimedia information processing device, characterized in that: An intelligent coprocessor for a vehicle, the device comprising: An acquisition module, used to acquire multimedia information collected by the vehicle-mounted multimedia acquisition module; A processing module, configured to process the multimedia information and obtain a processing result; A sending module is used to send the multimedia information and the processing result to the vehicle's cockpit domain controller, and the cockpit domain controller parses the multimedia information and the processing result according to a preset format to obtain the parsed multimedia information and processing result; generates a control result based on the parsed multimedia information and processing result; and displays the control result. The cockpit domain controller and the intelligent coprocessor are of split design.

8. A multimedia information processing device, characterized in that: A cockpit domain controller for a vehicle, the device comprising: A receiving module, configured to receive multimedia information and processing results sent by the vehicle's intelligent coprocessor; the processing results are obtained by the intelligent coprocessor acquiring multimedia information collected by the vehicle-mounted multimedia acquisition module and processing the multimedia information; A determination module is used to parse the multimedia information and the processing results respectively according to a preset format to obtain the parsed multimedia information and the processing results; generate a control result based on the parsed multimedia information and the processing results; and display the control result. The cockpit domain controller and the intelligent coprocessor are of a split design.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the multimedia information processing method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the multimedia information processing method according to any one of claims 1 to 6.

Images