Audio and video data processing method, device and equipment of vehicle, storage medium and vehicle

Abstract

Embodiments of the present disclosure provide a video and audio data processing method and device of a vehicle, equipment, a storage medium and the vehicle. The method comprises: acquiring basic configuration information of the vehicle; judging whether the vehicle can normally communicate with a car machine server based on the basic configuration information; if the vehicle can normally communicate with the car machine server, acquiring source configuration information of each video and audio data from the car machine server; acquiring corresponding video and audio data from each source server according to the source configuration information of each video and audio data; performing heterogeneous fusion processing on the video and audio data to obtain fused video and audio data; and outputting the fused video and audio data. In this way, multi-source video and audio data can be acquired from a unified source configuration management center and heterogeneous fusion can be achieved, thereby solving the problem of user switching between multiple APPs and fragmented experience caused by the application island mode in the background art.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle data playback, and more particularly to the field of vehicle audio-visual data processing technology. Background Technology

[0002] Currently, intelligent vehicles have evolved from simple means of transportation into mobile intelligent terminals integrating complex digital ecosystems. Powerful in-vehicle infotainment (IVI) systems serve as the user's interaction center, reshaping the in-vehicle experience, with audio-visual entertainment being a crucial part of meeting users' visual and auditory needs.

[0003] While current in-vehicle infotainment (IVI) systems generally support multi-source media access, most still remain at the hardware interface level (such as HDMI / USB / Bluetooth / CarPlay), employing an "application silo" model for the content layer: each streaming app independently maintains its own movie and music libraries, requiring users to switch between multiple apps and repeat content, resulting in a fragmented experience. Moreover, directly deploying various streaming apps within the vehicle's infotainment system can consume significant CPU and memory resources due to insufficient compatibility, leading to sluggish system response, increased heat generation, and even affecting the stability of core functions such as navigation and driver assistance systems. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, storage medium, and vehicle for processing audio and video data of a vehicle.

[0005] According to a first aspect of this disclosure, a method for processing audio-visual data of a vehicle is provided. The method includes: Obtain the basic configuration information of the vehicle; Based on the basic configuration information, determine whether the vehicle can communicate normally with the vehicle server; If the vehicle can communicate normally with the vehicle-mounted server, then the source configuration information of each audio-visual data is obtained from the vehicle-mounted server; Based on the source configuration information of each audio-visual data, obtain the corresponding audio-visual data from each source server respectively; The audio and video data are heterogeneously fused to obtain the fused audio and video data; Output the fused audio and video data.

[0006] In addition to the aspects and any possible implementations described above, a further implementation is provided in which obtaining the basic configuration information of the vehicle includes: Obtain the unique identification code of the vehicle; Obtain the model information of the vehicle; Obtain the manufacturer information of the vehicle; Obtain the network connection status of the vehicle; Obtain the current system time and preset reference time of the vehicle.

[0007] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein determining whether the vehicle can communicate normally with the vehicle-mounted server based on the basic configuration information includes: Based on the basic configuration information, it is determined whether the vehicle's current system time is correct and whether the vehicle can connect to the network normally, and the determination result is obtained. Based on the judgment result, it is determined whether the vehicle can communicate normally with the vehicle-mounted server.

[0008] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein determining whether the vehicle can communicate normally with the vehicle-mounted server based on the determination result includes: If the determination result is that the vehicle's current system time is correct and the vehicle can connect to the network normally, then it is determined that the vehicle can communicate normally with the vehicle server.

[0009] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein determining whether the vehicle can communicate normally with the vehicle-mounted server based on the determination result further includes: If the vehicle's current system time is incorrect or the vehicle cannot connect to the network normally, it is determined that the vehicle cannot communicate normally with the vehicle's computer server. The method further includes: Read stored audio and video data from the local cache; The stored audio and video data is output to the HMI interface for display; The system continuously monitors changes in the current system time and network status until the vehicle can communicate normally with the vehicle-mounted server.

[0010] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein if the vehicle can communicate normally with the vehicle-mounted server, the source configuration information of each audio-visual data is obtained from the vehicle-mounted server, including: If the vehicle can communicate normally with the vehicle server, then the vehicle's preset identifier is determined; Based on the vehicle's preset identifier, the vehicle's audio source configuration, video source configuration, HMI content configuration, and theme configuration are obtained from the vehicle's infotainment server, wherein the configuration is used to characterize the resource type.

[0011] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein the method further includes: Based on the HMI content configuration, determine the display content of the controls, buttons, cards, and lists on the current HMI interface; Based on the theme configuration, adapt the HMI interface mode and / or system theme style.

[0012] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein obtaining the corresponding audio-visual data from each source server according to the source configuration information of each audio-visual data includes: Obtain the preset correspondence between the source configuration information and the source server; Based on the source configuration information of each audio-visual data and the preset correspondence, determine the source server corresponding to each audio-visual data. Obtain the corresponding audio and video data from each of the aforementioned source servers.

[0013] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein obtaining the corresponding audio and video data from each source server includes: Each source server initiates an authentication request to each source server; upon receiving the authentication request, each source server parses the token to be authenticated in the authentication request, verifies the token to be authenticated, and sends the authentication verification result to the vehicle. If the authentication verification result of any source server is that the vehicle authentication is successful, then the corresponding audio and video data is obtained from that source server.

[0014] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein the heterogeneous fusion processing of the various audio-visual data to obtain fused audio-visual data includes: Clean the aforementioned audio and video data; The audio and video data are spatiotemporally aligned. The aforementioned audio and video data are standardized. The audio and video data are filtered and cut according to user habits.

[0015] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein the method further includes: Obtain the language information currently set for the vehicle; When the audio-visual data contains multilingual fields, the corresponding language content is automatically selected and output based on the language information.

[0016] According to a second aspect of this disclosure, a vehicle audio-visual data processing apparatus is provided. The apparatus includes: The first acquisition module is used to acquire the basic configuration information of the vehicle; The judgment module is used to determine whether the vehicle can communicate normally with the vehicle server based on the basic configuration information. The second acquisition module is used to acquire source configuration information of each audio-visual data from the vehicle server if the vehicle can communicate normally with the vehicle server. The third acquisition module is used to acquire the corresponding audio and video data from each source server according to the source configuration information of each audio and video data; The fusion module is used to perform heterogeneous fusion processing on the various audio-visual data to obtain fused audio-visual data; The acquisition module is used to output the fused audio and video data.

[0017] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0018] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0019] According to a fifth aspect of this disclosure, a vehicle is provided, comprising: the apparatus as described in the second aspect above; and the electronic equipment as described in the third aspect above.

[0020] In this disclosure, when the vehicle can communicate normally with the vehicle-mounted server, the source configuration information of each audio-visual data is obtained from the vehicle-mounted server. Then, based on the source configuration information of each audio-visual data, the corresponding audio-visual data is obtained from each source server. This allows for the direct acquisition of different audio-visual data from different servers using the source configuration information obtained from the vehicle-mounted server without the need to install different apps. The audio-visual data is then heterogeneously fused to obtain and output the fused audio-visual data. This achieves centralized acquisition and heterogeneous fusion of multi-source audio-visual data through unified source configuration management, solving the problem of users having to switch between multiple apps and experiencing fragmented experiences caused by the application silo mode in the background technology. At the same time, it avoids deploying multiple apps directly on the vehicle-mounted terminal, reducing CPU and memory usage and ensuring system response speed and core function stability.

[0021] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A flowchart of a vehicle audio-visual data processing method according to an embodiment of the present disclosure is shown; Figure 2 A flowchart of another method for processing audio-visual data of a vehicle according to an embodiment of the present disclosure is shown; Figure 3 A block diagram of a vehicle audio-visual data processing apparatus according to an embodiment of the present disclosure is shown; Figure 4 A block diagram of another vehicle audio-visual data processing apparatus according to an embodiment of the present disclosure is shown; Figure 5 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0024] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0025] Figure 1 A flowchart of a vehicle audio-visual data processing method 100 according to an embodiment of the present disclosure is shown. Method 100 may include: Step 110: Obtain the basic configuration information of the vehicle; Step 120: Based on the basic configuration information, determine whether the vehicle can communicate normally with the vehicle server; Step 130: If the vehicle can communicate normally with the vehicle-mounted server, then obtain the source configuration information of each audio-visual data from the vehicle-mounted server; Step 140: Based on the source configuration information of each audio-visual data, obtain the corresponding audio-visual data from each source server respectively; Step 150: Perform heterogeneous fusion processing on the various audio and video data to obtain fused audio and video data; Step 160: Output the fused audio-visual data.

[0026] When the vehicle can communicate normally with the vehicle-mounted server, the source configuration information of each audio-visual data is obtained from the vehicle-mounted server. Then, based on the source configuration information of each audio-visual data, the corresponding audio-visual data is obtained from each source server. This allows for the direct acquisition of different audio-visual data from different servers without the need to install different apps. The audio-visual data is then heterogeneously fused to obtain the fused audio-visual data, which is then output. This achieves centralized acquisition and heterogeneous fusion of multi-source audio-visual data through unified source configuration management. This solves the problem of users having to switch between multiple apps and experiencing fragmented performance caused by the application silo mode in the background technology. At the same time, it avoids deploying multiple apps directly on the vehicle-mounted system, reducing CPU and memory usage and ensuring system response speed and core function stability.

[0027] In some embodiments, obtaining the basic configuration information of the vehicle includes: Obtain the unique identification code of the vehicle; Obtain the model information of the vehicle; Obtain the manufacturer information of the vehicle; Obtain the network connection status of the vehicle; Obtain the current system time and preset reference time of the vehicle.

[0028] By collecting multi-dimensional basic vehicle information, a comprehensive data foundation is provided for subsequent communication status judgment and personalized configuration acquisition, ensuring the accuracy and adaptability of subsequent processing procedures.

[0029] In some embodiments, determining whether the vehicle can communicate normally with the vehicle-mounted server based on the basic configuration information includes: Based on the basic configuration information, it is determined whether the vehicle's current system time is correct and whether the vehicle can connect to the network normally, and the determination result is obtained. Based on the judgment result, it is determined whether the vehicle can communicate normally with the vehicle-mounted server.

[0030] By determining whether the vehicle's current system time is correct and whether the vehicle can connect to the network normally, the correctness of the system time and network connectivity can be used as the core conditions for judging the communication status, avoiding the failure of information source configuration acquisition due to time abnormalities or network problems, and improving the accuracy of communication prediction.

[0031] In some embodiments, determining whether the vehicle can communicate normally with the vehicle-mounted server based on the determination result includes: If the determination result is that the vehicle's current system time is correct and the vehicle can connect to the network normally, then it is determined that the vehicle can communicate normally with the vehicle server.

[0032] By using the vehicle's current system time accuracy and the vehicle's ability to connect to the network normally as the criteria for determining whether the vehicle can communicate normally with the vehicle's computer server, online data acquisition is only triggered when both time and network conditions are met, thus preventing invalid requests from wasting resources.

[0033] In some embodiments, determining whether the vehicle can communicate normally with the vehicle-mounted server based on the determination result further includes: If the vehicle's current system time is incorrect or the vehicle cannot connect to the network normally, it is determined that the vehicle cannot communicate normally with the vehicle's computer server. The method further includes: Read stored audio and video data from the local cache; The stored audio and video data is output to the HMI interface for display; The system continuously monitors changes in the current system time and network status until the vehicle can communicate normally with the vehicle-mounted server.

[0034] If the current system time is incorrect or the vehicle cannot connect to the network normally, it is determined that the vehicle cannot communicate normally with the vehicle server. At this time, it can automatically switch to local cached data when communication is abnormal, so as to ensure that the user's audio-visual experience is not interrupted. At the same time, it continuously monitors the time and network changes, and seamlessly switches to online mode after recovery, improving system fault tolerance and user experience continuity.

[0035] In some embodiments, if the vehicle can communicate normally with the vehicle-mounted server, obtaining the source configuration information of each audio-visual data from the vehicle-mounted server includes: If the vehicle can communicate normally with the vehicle server, then the vehicle's preset identifier is determined; Based on the vehicle's preset identifier, the vehicle's audio source configuration, video source configuration, HMI content configuration, and theme configuration are obtained from the vehicle's infotainment server, wherein the configuration is used to characterize the resource type.

[0036] By utilizing vehicle preset identifiers to obtain multi-dimensional configuration information, personalized resource type adaptation can be achieved for different vehicle models or users, avoiding the problem of mismatched display content caused by uniform configuration.

[0037] In some embodiments, the method further includes: Based on the HMI content configuration, determine the display content of the controls, buttons, cards, and lists on the current HMI interface; Based on the theme configuration, adapt the HMI interface mode and / or system theme style.

[0038] By dynamically adjusting the interface display based on HMI content and theme configuration, the presentation of audio-visual integrated data is consistent with the overall UI style of the vehicle system, improving user visual consistency and ease of operation.

[0039] In some embodiments, obtaining the corresponding audio-visual data from each source server according to the source configuration information of each audio-visual data includes: Obtain the preset correspondence between the source configuration information and the source server; Based on the source configuration information of each audio-visual data and the preset correspondence, determine the source server corresponding to each audio-visual data. Obtain the corresponding audio and video data from each of the aforementioned source servers.

[0040] By obtaining the preset correspondence between the source configuration information and the source server, and then quickly locating the source server corresponding to each audio and video data based on the preset correspondence, the corresponding audio and video data can be obtained from each source server. This can improve data acquisition efficiency, avoid blind requests, and reduce network overhead.

[0041] In some embodiments, obtaining the corresponding audio and video data from each source server includes: Each source server initiates an authentication request to each source server; upon receiving the authentication request, each source server parses the token to be authenticated in the authentication request, verifies the token to be authenticated, and sends the authentication verification result to the vehicle. If the authentication verification result of any source server is that the vehicle authentication is successful, then the corresponding audio and video data is obtained from that source server.

[0042] By sending authentication requests to each of the aforementioned source servers, each source server parses the token to be authenticated and verifies it. Once the authentication verification result of any source server passes, the corresponding audio and video data is obtained from that source server. In this way, a unified authentication mechanism is used to connect to each source server, ensuring the legality and security of data acquisition, while avoiding the interruption of the entire process due to the failure of authentication of a single source.

[0043] In some embodiments, the heterogeneous fusion processing of the audio-visual data to obtain fused audio-visual data includes: Clean the aforementioned audio and video data; The audio and video data are spatiotemporally aligned. The aforementioned audio and video data are standardized. The audio and video data are filtered and cut according to user habits.

[0044] By cleaning, aligning, standardizing, and customizing the various audio and video data, heterogeneous data from different sources can be integrated into content with a unified format and style, effectively eliminating data differences and improving the consistency of the merged audio and video data and user satisfaction.

[0045] In some embodiments, the method further includes: Obtain the language information currently set for the vehicle; When the audio-visual data contains multilingual fields, the corresponding language content is automatically selected and output based on the language information.

[0046] After obtaining the vehicle's current language settings, the system can automatically match multilingual content based on the vehicle's language settings, reducing the need for manual switching and improving usability in international scenarios.

[0047] In some embodiments, the method further includes: The fused audio and video data is stored persistently in a local cache and / or database. Record the timestamp of the most recent data refresh.

[0048] By caching and persistently storing the fused audio-visual data and refreshing the timestamp records, data reuse in offline scenarios can be supported, providing a basis for subsequent incremental updates, reducing repeated network requests, and improving system efficiency.

[0049] Figure 2 An architectural diagram of the audio-visual data processing apparatus of the present invention is shown: The audio-visual system adopts a three-layer architecture: a foundation layer, a capability layer, and an HMI layer. Each layer depends only on the layer below it, adhering to the principle of unidirectional dependency. This ensures module decoupling, clear interfaces, and facilitates tailoring, independent upgrades, or reuse by car manufacturers for different models. It also reduces coupling risks and improves system stability and maintainability.

[0050] 100-1 Network Status Monitoring: Monitors network connection status, network availability, and network link quality (4G / 5G, Wi-Fi, V2X).

[0051] 100-2 Network Requests: Provides a unified encapsulation for the server-side RESTful / HTTPS interfaces, supporting dynamic injection of vehicle VINs, user tokens, and region codes at the header level, facilitating copyright and territorial control on the information source side.

[0052] 100-3 Data Caching: Employs the MMKV scheme for fast storage and retrieval of large binary objects. As a basic tool, it can be reused across multiple platforms, including in-vehicle and mobile devices.

[0053] 100-4 Data storage: The SQLite solution is used to achieve persistent storage of structured metadata.

[0054] 100-5 Log Tool: Enables real-time uploading of critical copyright errors to the server-side cloud platform. Log content is anonymized to prevent user privacy leaks. Logs can be easily retrieved on-site.

[0055] 100-6 Image Library: Enables image decoding, scaling, cropping, and caching, as well as image preloading. Achieves zero-wait visual matching for HMI. As a basic tool, it can be reused across multiple platforms, including in-vehicle and mobile devices.

[0056] 100-7 Cross-process communication tool: Encapsulates AIDL+Binder connection pool, exposing the capability layer Service to the host APP. Supports "subscribe-publish" pattern; when the aggregator updates its business, it broadcasts a one-time notification to all APPs, reducing duplicate Binder calls.

[0057] 100-8 Multithreading Tool: Based on Kotlin coroutines and a custom thread pool, it offloads CPU-intensive tasks and I / O-intensive tasks (downloading, writing libraries) to different schedulers to prevent thread jitter.

[0058] 200-1 Source Server API: Audio and video source providers offer source resource server access interfaces. Building upon the basic network request capabilities of 100-2, it enables access to various source server interfaces, data acquisition, and parsing. It also provides isolated management for access to each source.

[0059] 200-2 Source SDK: Audio and video source providers offer SDKs, which allow users within the audio-visual aggregation system to access audio and video resources by calling methods exposed by the SDKs. This module manages the isolation of access to each source SDK to prevent any runtime exception or crash from causing the entire audio-visual aggregation service to become unavailable.

[0060] 200-3 Source Data Management: Building upon the basic capabilities of 100-3 data caching and 100-8 multi-threading tools, this feature enables the asynchronous storage and retrieval of source data acquired from various sources.

[0061] 200-4 Heterogeneous Data Fusion from Information Sources: Reorganize and integrate the different data structures of various information sources into a unified standard data structure, and output the organized data to the HMI interface.

[0062] 200-5 Data Preloading: On top of the 100-8 storage, a "business cache" layer is encapsulated. For events such as vehicle shutdown, hibernation, power failure, and no network, the current key data is encrypted and stored in MMKV the moment the event is triggered. After the vehicle or system status is restored, the data is automatically read out and the interface status is restored.

[0063] 200-6 Data Publishing: Non-hosted applications and other applications running on the system need to obtain some data from the media aggregator. Building upon the basic capabilities of the 100-7 inter-process communication tool, the media aggregator completes data publishing to other applications in this module.

[0064] 200-7 Data Update Strategy: Scheme for updating data content from audio and video sources.

[0065] 200-8 Device Information Management: Reads vehicle information such as screen size, screen resolution, number of audio channels, vehicle code, and vehicle-mounted entertainment screens.

[0066] 200-9 Configuration Management: Building upon the basic network request capabilities of 100-2, this feature enables access to the vehicle infotainment system manufacturer's server-side business interfaces. It retrieves project configuration lists, such as: audio source from supplier A and video source from supplier B for a specific vehicle model project; HMI content configured for a specific vehicle model project; and system theme content for a specific vehicle model project.

[0067] 200-10 Voice Control: Automatically generate "visible and speakable" semantic labels for all buttons, cards, and lists in the current HMI, and the labels are dynamically refreshed with the interface.

[0068] 200-11 Multi-theme adaptation: Adapts the control styles on the HMI based on the current vehicle model project system theme content obtained from configuration management in 200-9.

[0069] 200-12 Multilingual Adaptation: The content side returns multilingual fields from the source API, and the aggregate automatically selects the appropriate language based on the voice settings in the vehicle's infotainment system.

[0070] 200-11 Network Status Monitoring: Building upon the basic capabilities of 100-1, and considering real-world scenarios such as weak in-vehicle networks, tunnels, and underground parking lots, the audio-visual aggregator can detect and dynamically downgrade in real time, automatically pausing preloading and enabling local caching to avoid lag; at the same time, it writes the status to the log for diagnosing network dead spots.

[0071] 300-1 Default HMI Style: Adapts to vehicle screen resolution and reserved space size. Supports day / night switching and Chinese / English switching.

[0072] 300-2 Customized HMI: Showcases HMI design content created by vehicle manufacturers.

[0073] This invention, an audio-visual aggregator, solves the problem of application silos, providing users with a one-stop aggregated display of content from multiple sources, avoiding the fragmented experience caused by users repeatedly switching between different source applications.

[0074] The video aggregator runs within the host application, eliminating the need to request independent app resources from the system, thus avoiding the simultaneous operation of multiple source apps and reducing system resource consumption.

[0075] As a complete service, the video-audio aggregation platform is easy for host applications to integrate, allows for rapid development, and reduces the technical and human resource costs for enterprise software development. It also addresses copyright and compliance risks.

[0076] By injecting the VIN+regional code through the unified network in 100-2, the source side verifies the regional copyright in real time; and by uploading the 100-5 logs with anonymized data, it complies with GDPR / China's Personal Information Protection Law.

[0077] The specific implementation steps of the audio-visual data processing method of the present invention are as follows: Figure 2 As shown: The process requires starting hardware devices (such as vehicle ignition and vehicle store). After the vehicle system starts, when the host process starts, it calls the multimedia aggregator startup method, and the multimedia aggregator starts running.

[0078] Once the multimedia aggregator starts running, it will listen for system startup broadcasts, system time broadcasts, and network status broadcasts to obtain information on changes in system time and network status.

[0079] The audio-visual aggregator reads basic vehicle information.

[0080] If the system time is incorrect, or the system network is unavailable (not connected to the internet), the audio / video aggregator cannot communicate with the source servers. It then reads the cached data stored in the device and refreshes it onto the HMI page. Simultaneously, it continuously monitors changes in system time and system network status.

[0081] If the system time is correct and the system network is available (connected to the Internet), the audio / video aggregator connects and communicates with the vehicle infotainment system manufacturer's server. The audio / video aggregator obtains the current vehicle model's audio source configuration, video source configuration, HMI content configuration, theme configuration, etc. from the vehicle infotainment system manufacturer's server.

[0082] Based on the source configuration obtained, authentication requests are sent to each source server respectively.

[0083] Obtain trusted authentication tokens from each source server. (The video aggregator is mentioned here, but the context is unclear.) Figure 1 In module 200-3, authentication tokens for each source server are managed, including: encrypted storage, reading, and expiration verification.

[0084] Use the authentication token obtained from the source server through the above steps to communicate with the source server multiple times to obtain source business data. Use such as Figure 3 The 200-3 module shown stores and retrieves data from multiple asynchronously acquired sources.

[0085] Use such as Figure 3 The 200-4 module shown performs heterogeneous data fusion processing on the multiple information source service data obtained in

[009] to obtain information source data with a unified data structure.

[0086] Refresh the unified data structure source data from the above steps to the HMI page and display it to the user. This completes one instance of multi-source heterogeneous data fusion and output.

[0087] Use such as Figure 3 The 200-3 module shown caches the source data of the unified data structure in

[010] , and some data is persistently saved using a database.

[0088] Update the last data refresh time and execute the new round of update strategy.

[0089] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0090] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0091] Figure 4 A block diagram of a vehicle audio-visual data processing apparatus 400 according to an embodiment of the present disclosure is shown. Figure 4 As shown, the device 400 includes: The first acquisition module 410 is used to acquire the basic configuration information of the vehicle; The judgment module 420 is used to determine, based on the basic configuration information, whether the vehicle can communicate normally with the vehicle server; The second acquisition module 430 is used to acquire source configuration information of each audio-visual data from the vehicle server if the vehicle can communicate normally with the vehicle server. The third acquisition module 440 is used to acquire the corresponding audio and video data from each source server according to the source configuration information of each audio and video data; The fusion module 450 is used to perform heterogeneous fusion processing on the various audio-visual data to obtain fused audio-visual data; The acquisition module 460 is used to output the fused audio and video data.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0093] According to embodiments of the present disclosure, the present disclosure also provides an electronic device and a non-transitory computer-readable storage medium storing computer instructions.

[0094] According to embodiments of this disclosure, this disclosure also provides a vehicle.

[0095] Figure 5 A schematic block diagram of an electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0096] Device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0097] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform method 100 by any other suitable means (e.g., by means of firmware).

[0099] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0104] Computing systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0105] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for processing audio-visual data of a vehicle, characterized in that, include: Obtain the basic configuration information of the vehicle; Based on the basic configuration information, determine whether the vehicle can communicate normally with the vehicle server; If the vehicle can communicate normally with the vehicle-mounted server, then the source configuration information of each audio-visual data is obtained from the vehicle-mounted server; Based on the source configuration information of each audio-visual data, obtain the corresponding audio-visual data from each source server respectively; The audio and video data are heterogeneously fused to obtain the fused audio and video data; Output the fused audio and video data.

2. The method as described in claim 1, characterized in that, The process of obtaining the basic configuration information of the vehicle includes: Obtain the unique identification code of the vehicle; Obtain the model information of the vehicle; Obtain the manufacturer information of the vehicle; Obtain the network connection status of the vehicle; Obtain the current system time and preset reference time of the vehicle.

3. The method as described in claim 1, characterized in that, The step of determining whether the vehicle can communicate normally with the vehicle-mounted server based on the basic configuration information includes: Based on the basic configuration information, it is determined whether the vehicle's current system time is correct and whether the vehicle can connect to the network normally, and the determination result is obtained. Based on the judgment result, it is determined whether the vehicle can communicate normally with the vehicle-mounted server.

4. The method as described in claim 3, characterized in that, The step of determining whether the vehicle can communicate normally with the vehicle-mounted server based on the determination result includes: If the determination result is that the vehicle's current system time is correct and the vehicle can connect to the network normally, then it is determined that the vehicle can communicate normally with the vehicle server.

5. The method as described in claim 3, characterized in that, The step of determining whether the vehicle can communicate normally with the vehicle-mounted server based on the determination result further includes: If the vehicle's current system time is incorrect or the vehicle cannot connect to the network normally, it is determined that the vehicle cannot communicate normally with the vehicle's computer server. The method further includes: Read stored audio and video data from the local cache; The stored audio and video data is output to the HMI interface for display; The system continuously monitors changes in the current system time and network status until the vehicle can communicate normally with the vehicle-mounted server.

6. The method as described in claim 1, characterized in that, If the vehicle can communicate normally with the vehicle-mounted server, then the source configuration information of each audio-visual data is obtained from the vehicle-mounted server, including: If the vehicle can communicate normally with the vehicle server, then the vehicle's preset identifier is determined; Based on the vehicle's preset identifier, the vehicle's audio source configuration, video source configuration, HMI content configuration, and theme configuration are obtained from the vehicle's infotainment server.

7. The method as described in claim 6, characterized in that, The method further includes: Based on the HMI content configuration, determine the display content of the controls, buttons, cards, and lists on the current HMI interface; Based on the theme configuration, adapt the HMI interface mode and / or system theme style.

8. The method as described in claim 1, characterized in that, The step of obtaining the corresponding audio-visual data from each source server according to the source configuration information of each audio-visual data includes: Obtain the preset correspondence between the source configuration information and the source server; Based on the source configuration information of each audio-visual data and the preset correspondence, determine the source server corresponding to each audio-visual data. Obtain the corresponding audio and video data from each of the aforementioned source servers.

9. The method as described in claim 8, characterized in that, The step of obtaining the corresponding audio and video data from each source server includes: Each of the aforementioned information source servers is sent an authentication request. If the authentication verification result of any source server is that the vehicle authentication is successful, then the corresponding audio and video data is obtained from that source server.

10. The method as described in claim 1, characterized in that, The process of heterogeneously fusing the various audio-visual data to obtain fused audio-visual data includes: Clean the aforementioned audio and video data; The audio and video data are spatiotemporally aligned. The aforementioned audio and video data are standardized. The audio and video data are filtered and cut according to user habits.

11. The method as described in claim 1, characterized in that, The method further includes: Obtain the language information currently set for the vehicle; When the audio-visual data contains multilingual fields, the corresponding language content is automatically selected and output based on the language information.

12. A vehicle audio-visual data processing device, characterized in that, include: The first acquisition module is used to acquire the basic configuration information of the vehicle; The judgment module is used to determine whether the vehicle can communicate normally with the vehicle server based on the basic configuration information. The second acquisition module is used to acquire source configuration information of each audio-visual data from the vehicle server if the vehicle can communicate normally with the vehicle server. The third acquisition module is used to acquire the corresponding audio and video data from each source server according to the source configuration information of each audio and video data; The fusion module is used to perform heterogeneous fusion processing on the various audio-visual data to obtain fused audio-visual data; The acquisition module is used to output the fused audio and video data.

13. An electronic device, characterized in that, include: Memory and processor The memory stores a computer program, and when the processor executes the program, it implements the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor corresponding to the electronic device, the electronic device is able to implement the vehicle audio-visual data processing method as described in any one of claims 1-11.

15. A vehicle, characterized in that, include: The apparatus as described in claim 12; as well as The electronic device as claimed in claim 13.

Images