A vehicle-mounted audio and video processing method based on SOA architecture

Abstract

The application relates to a vehicle-mounted audio and video processing method based on an SOA architecture, in particular to the technical field of audio and video data processing, which comprises the following steps: S1, audio and video around a vehicle are respectively collected and uploaded; S2, the uploaded video and audio are subjected to optimization processing; S3, the video subjected to the optimization processing is transmitted to a server end; S4, the video is transmitted to a client end through the server end; S5, the video is subjected to real-time display; S6, the video and the audio are subjected to time synchronization processing; S7, the audio and the video are subjected to audio and video synthesis, and the synthesized audio and video are obtained; and S8, the synthesized audio and video are transmitted for storage and display, and the vehicle-mounted audio and video processing process is controlled. The application processes the vehicle-mounted audio and video based on hardware devices of a driving domain controller and a cabin domain controller of a whole vehicle, so that a driving record function under a new architecture is realized, and the processing efficiency of the vehicle-mounted audio and video is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio and video data processing, and particularly relates to a vehicle-mounted audio and video processing method based on an SOA architecture. BACKGROUND

[0002] Under the whole intelligent networked vehicle environment, the electronic and electrical architecture (EEA) is facing changes, and the service-oriented architecture SOA is mentioned many times. In terms of the development trend of the automobile industry, there are about 150 ECUs in each vehicle in the traditional architecture. Under the SOA architecture, the vehicle ECU is greatly reduced. The originally dispersed ECUs and their corresponding basic software are modularized and standardized, and are redeployed as a layered software architecture. The vehicle can provide different services for the driver through different software configurations without increasing or replacing the hardware. For traditional vehicles, the independent driving recorder has become the first ECU to be "cut off" on the vehicle, but the driving record function is essential for users in daily use. Therefore, how to realize the driving record function based on the existing hardware devices of the vehicle is a difficult problem that needs to be solved.

[0003] Chinese Patent Publication No. CN106204805A discloses a driving recorder and a parking monitoring method and system based on the driving recorder. The parking monitoring method includes configuring the interrupt trigger function of the acceleration sensor of the driving recorder and keeping the acceleration sensor in a powered state. The driving recorder detects whether the acceleration sensor generates an interrupt trigger immediately after starting. When the interrupt trigger is generated, the driving recorder is started and enters a parking monitoring mode to perform emergency recording. The scheme is an independent driving recorder controller, and cannot realize the driving record function based on the existing hardware devices of the vehicle under the SOA architecture. SUMMARY

[0004] Therefore, the present application provides a vehicle-mounted audio and video processing method based on an SOA architecture to overcome the problem that it is difficult to realize the driving record function based on the existing hardware devices of the vehicle under the SOA architecture and the low audio and video processing efficiency in the prior art.

[0005] To achieve the above-mentioned purpose, the present application provides a vehicle-mounted audio and video processing method based on an SOA architecture, comprising:

[0006] Step S1, audio and video around the vehicle are collected respectively, the collected audio is uploaded to a cabin domain controller, and the collected video is uploaded to a driving domain controller;

[0007] Step S2, the uploaded video is optimized by a video processor, and the uploaded audio is optimized by a sound processor;

[0008] Step S3: The optimized video is transmitted to the server. After receiving the video, the server sends user parameters to the video processor.

[0009] Step S4: The video is transmitted from the server to the client of the cockpit domain controller;

[0010] Step S5: The video is transmitted to the in-vehicle infotainment system via the client for real-time display;

[0011] Step S6: While displaying the video in real time, perform time synchronization processing on the video in the video processor, the video in the client, and the audio in the sound processor.

[0012] Step S7: After the time synchronization process is completed, the optimized audio and the video in the client are combined to obtain the combined audio and video.

[0013] Step S8: The synthesized audio and video are transmitted to the interface module for storage and display, and the in-vehicle audio and video processing process is controlled by the application in the client.

[0014] Furthermore, in step S1, during video acquisition, the video of the vehicle's surroundings recorded by the camera is acquired through the video interface;

[0015] In step S2, the video optimization processing includes stabilizing the video, dehazing, scaling the video, overlaying the video, and encoding the video.

[0016] Furthermore, in step S1, audio from around the vehicle is collected through a microphone interface, and in step S2, the audio is optimized by a sound processor. The audio optimization includes noise suppression and audio format encoding.

[0017] Furthermore, in step S3, the optimized video is transmitted to the server via a video data transmitter, and user parameters are sent to the video processor via a user parameter receiver on the server. The video data transmitter transmits the optimized video to the server via a real-time streaming protocol and audio / video bridging technology.

[0018] Furthermore, in step S4, the video is transmitted from the server of the driving domain controller to the client of the cockpit domain controller via Ethernet based on the real-time streaming protocol, and the video format is H.264.

[0019] Furthermore, in step S6, the video in the video processor, the video in the client, and the audio in the sound processor are synchronized in time using a global clock. In step S7, the audio and video are synthesized using a real-time transmission processor.

[0020] Further, in the step S8, the synthesized audio and video are stored in the mobile hard disk of the interface module, and when the adaptive automatic brake signal is collected by the SOA interface, the synthesized audio and video are transmitted to the interface module for storage.

[0021] Further, in the step S8, the control mode of the vehicle-mounted audio and video processing process by the application end includes video display control, audio and video recording control, factory setting recovery control, and fault state control, wherein:

[0022] When the video display control is performed, the vehicle-mounted audio and video processing application end sends a signal of displaying video and hiding video to the interface module, the interface module sends a notification signal to the rendering module, the rendering module sends a rendering video or a hidden video to the screen of the client according to the notification signal, and when the video display control is performed, the interface exposed by the vehicle-mounted audio and video processing application end to the interface module includes a display video interface and a hidden video interface;

[0023] When the audio and video recording control is performed, the vehicle-mounted audio and video processing application end sends a signal of application end interface calling to the interface module, the interface module sends a notification signal to the rendering module, the rendering module sends an audio and video file to the disk, the disk stores the audio and video file, the disk returns an interface calling result to the rendering module after storage, the rendering module returns the interface calling result to the interface module after receiving the return interface calling result of the disk, and the interface module returns the interface calling result to the vehicle-mounted audio and video processing application end;

[0024] When the factory setting recovery control is performed, the vehicle-mounted audio and video processing application end sends a signal of factory setting recovery to the interface module, and the interface module sends a notification signal to the factory setting recovery module;

[0025] When the fault state control is performed, the vehicle-mounted audio and video processing application end sends a signal of registering callback to the interface module, the interface module sends a notification signal to the fault state module, the fault state module performs state change notification after receiving the notification signal, and sends a callback notification to the interface module, and the interface module sends the callback notification to the vehicle-mounted audio and video processing application end after receiving the callback notification.

[0026] Further, the driving domain controller includes a video processor, a server-side video data transmitter, and a server-side user parameter receiver, the cabin domain controller includes a sound processor, a real-time transmission processor, a client, and an interface module, the client includes a video data receiver, a video parameter transmitter, and an application end, the interface module includes a data storage module and a display module, the data storage module includes a storage strategy unit and a mobile hard disk, and the display module includes an encoder and a vehicle-mounted entertainment information display.

[0027] Further, the interface attribute of the mobile hard disk includes USB2.0 and USB3.0, and the interface attribute of the driving domain controller is Ethernet.

[0028] Compared with the prior art, the present application has the beneficial effect that the present application utilizes the hardware devices of the vehicle itself to process the vehicle audio and video without increasing the hardware devices, thereby realizing the vehicle recording function under the new architecture without independent devices and improving the processing efficiency of the vehicle audio and video.

[0029] Especially, the video quality is improved by anti-shake, defogging, video zooming, video superimposition and encoding, the vehicle audio and video processing efficiency is further improved, the audio quality is improved by optimizing the audio, and the vehicle audio and video processing efficiency is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 Fig. 1 is a flowchart of the vehicle audio and video processing method based on the SOA architecture according to the present application. DETAILED DESCRIPTION

[0031] In order to make the objects and advantages of the present application more clear, the present application will be further described below in conjunction with the embodiments; it should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0032] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and do not limit the protection scope of the present application.

[0033] Please refer to Figure 1 Fig. 1 is a flowchart of the vehicle audio and video processing method based on the SOA architecture according to the present application, which includes:

[0034] Step S1, the audio and video around the vehicle are collected respectively, the collected audio is uploaded to the cabin domain controller, and the collected video is uploaded to the driving domain controller;

[0035] Step S2, the uploaded video is optimized by the video processor, and the uploaded audio is optimized by the sound processor;

[0036] Step S3, the optimized video is transmitted to the server end, and the server end sends the user parameters to the video processor after receiving the video;

[0037] Step S4, the video is transmitted to the client end of the cabin domain controller by the server end;

[0038] Step S5, transmitting the video to the vehicle infotainment system through the client for real-time display;

[0039] Step S6, while the video is being displayed in real time, time synchronization processing is performed on the video in the video processor, the video in the client, and the audio in the sound processor;

[0040] Step S7, after the time synchronization processing is completed, the optimized audio and the video in the client are synthesized to obtain synthesized audio and video;

[0041] Step S8, the synthesized audio and video are transmitted to the interface module for storage and display, and the vehicle audio and video processing process is controlled through the application end in the client.

[0042] Specifically, in step S1, the audio around the vehicle is collected through the microphone interface, and in step S2, the audio is optimized by the sound processor, and the optimization of the audio includes noise suppression processing and audio format coding processing.

[0043] Specifically, the video is deblurred, defogged, scaled, overlaid, and encoded to improve the video quality and further improve the vehicle audio and video processing efficiency. The optimization method of the video is not limited in the embodiment, and the person skilled in the art can freely set it as long as the demand for improving the video quality is met, such as also increasing the frame rate of the video.

[0044] Specifically, the audio is optimized to improve the audio quality and further improve the vehicle audio and video processing efficiency. The optimization method of the audio is not limited in the embodiment, and the person skilled in the art can freely set it as long as the demand for improving the audio quality is met, such as also performing equalization processing on the audio.

[0045] Specifically, in step S3, the video processor is transmitted to the server end through the video data transmitter, and the user parameters are sent to the video processor through the user parameter receiver in the server end, and the video data transmitter transmits the optimized video to the server end through the real-time streaming protocol and the audio and video bridging technology.

[0046] Specifically, in step S4, the video is transmitted from the server end of the driving domain controller to the client of the cabin domain controller through the real-time streaming protocol based on Ethernet, and the format of the video is H.264.

[0047] Specifically, in step S6, the video in the video processor, the video in the client, and the audio in the sound processor are time-synchronized through a global clock, and in step S7, the audio and the video are synthesized through a real-time transmission processor.

[0048] Specifically, in the step S8, the synthesized audio and video are stored in the mobile hard disk of the interface module, and when the adaptive automatic brake signal is collected by the SOA interface, the synthesized audio and video are transmitted to the interface module for storage.

[0049] Specifically, in the step S8, only the synthesized audio and video are transmitted to the interface module for storage when the adaptive automatic brake signal is collected by the SOA interface, so that the video is written to the disk in an emergency. The detection of the emergency is not specifically limited in the present application, and other detection methods of the emergency can also be set by those skilled in the art, such as setting the collision signal collected by the SOA interface as the detection method of the emergency, as long as the accurate detection of the emergency is met. The present application takes the adaptive automatic brake signal collected by the SOA interface as the detection method of the emergency as an example. When the adaptive automatic brake signal is collected by the SOA interface, the synthesized audio and video are transmitted to the interface module for storage. When the adaptive automatic brake signal is not collected by the SOA interface, the synthesized audio and video are transmitted to the interface module for display and storage.

[0050] Specifically, the client of the vehicle information entertainment system is provided with a vehicle audio and video processing application end. In the step S8, the control mode of the application end on the vehicle audio and video processing process includes video display control, audio and video recording control, factory setting recovery control and fault state control, wherein:

[0051] When the video display control is performed, the vehicle audio and video processing application end sends a signal of displaying video and hiding video to the interface module, the interface module sends a notification signal to the rendering module, and the rendering module sends a rendering video or a hidden video to the screen of the client according to the notification signal. When the video display control is performed, the interface exposed by the vehicle audio and video processing application end includes a display video interface and a hidden video interface. The display video interface is an interface for displaying video and rendering video to the screen, and its function name is addSurface, its parameter is in Surface surface, and its return value type is void. The hidden video interface is an interface for hiding video and stopping video rendering, and its function name is removeSurface, its parameter is in Surface surface, and its return value type is void.

[0052] In the audio and video recording control, the vehicle-mounted audio and video processing application end sends an application interface calling signal to the interface module, the interface module sends a notification signal to the rendering module, the rendering module sends an audio and video file to the disk, the disk stores the audio and video file, the disk returns an interface calling result to the rendering module after storage, the rendering module returns the interface calling result to the interface module after receiving the return interface calling result of the disk, the interface module returns the interface calling result to the vehicle-mounted audio and video processing application end, the interface exposed by the interface module to the vehicle-mounted audio and video processing application end in the audio and video recording control includes an open recording interface, a close recording interface, a start recording file interface, a stop recording file interface, a photographing interface, a save manual emergency data interface and a save automatic emergency data interface, the open recording interface refers to an interface for recording an mp4 file containing audio, the function name of which is startAudioRecord, the parameter of which is void, and the return value type of which is int, the close recording interface refers to an interface for recording an mp4 file without audio, the function name of which is stopAudioRecord, the parameter of which is void, and the return value type of which is int, the start recording file interface refers to an interface for starting to record an mp4 file, the function name of which is startRecord, the parameter of which is void, and the return value type of which is int, the stop recording file interface refers to an interface for stopping to record an mp4 file, the function name of which is stopRecord, the parameter of which is void, and the return value type of which is int, the photographing interface refers to an interface for capturing a current time picture, the function name of which is captureImage, the parameter of which is long currentTime, and the return value type of which is int, the save manual emergency data interface refers to an interface for saving manual emergency video data as an mp4 file, the function name of which is captureUrgentData, the parameter of which is void, and the return value type of which is int, and the save automatic emergency data interface refers to an interface for saving automatic emergency video data as an mp4 file, the function name of which is captureAutoUrgentData, the parameter of which is void, and the return value type of which is int;

[0053] In the factory setting recovery control, the vehicle-mounted audio and video processing application end sends a factory setting recovery signal to the interface module, the interface module sends a notification signal to the factory setting recovery module, the interface exposed by the interface module to the vehicle-mounted audio and video processing application end in the factory setting recovery control includes a factory setting recovery interface, and the factory setting recovery interface refers to an interface for restoring the application end to default settings, the function name of which is restorFactorySetting, the parameter of which is void, and the return value type of which is void;

[0054] When the fault state control is performed, the vehicle-mounted audio and video processing application end sends a signal of registering callback to the interface module, the interface module sends a notification signal to the fault state module, the fault state module performs state change notification after receiving the notification signal, and sends a callback notification to the interface module, the interface module sends the callback notification to the vehicle-mounted audio and video processing application end after receiving the callback notification, and the interface exposed by the interface module to the vehicle-mounted audio and video processing application end in the fault state control includes a fault state display interface, the fault state display interface is an interface for displaying the state during photographing and emergency recording, the function name is setOnDvrEventListener, the parameter name is listener, the parameter type is OnDvrEventListene, the parameter description is interface OnDvrEventListener, and the return value type is void.

[0055] Specifically, the method described in the embodiment is based on the hardware devices of the driving domain controller and the cabin domain controller of the whole vehicle, and the vehicle-mounted audio and video are processed, the driving domain controller includes a video processor, a server-side video data transmitter and a server-side user parameter receiver, the cabin domain controller includes a sound processor, a real-time transmission processor, a client and an interface module, the client includes a video data receiver, a video parameter transmitter and an application end, the interface module includes a data storage module and a display module, the data storage module includes a storage strategy unit and a mobile hard disk, and the display module includes an encoder and a vehicle-mounted entertainment information display.

[0056] Specifically, the method utilizes the hardware devices of the vehicle itself on the basis of not increasing hardware devices to process the vehicle-mounted audio and video, so that the vehicle recording function under the new architecture is realized without independent devices, and the processing efficiency of the vehicle-mounted audio and video is improved.

[0057] Specifically, the embodiment is based on the hardware devices of the driving domain controller and the cabin domain controller of the whole vehicle to process the vehicle-mounted audio and video under the SOA architecture, the SOA architecture refers to a service-oriented architecture, the hardware interfaces in the method include a mobile hard disk and a driving domain controller, the interface properties of the mobile hard disk include USB2.0 and USB3.0, the interface property of the driving domain controller is Ethernet, the software interfaces in the application include RTSP, SOA, MediaMuxer and HQ Unified API, RTSP refers to a protocol for transmitting video through Ethernet, SOA refers to an interface for controlling the processing of vehicle-mounted audio and video through Ethernet, MediaMuxer refers to an interface provided by an Android system for combining audio and video into audio and video, and HQ Unified API refers to an interface for providing a vehicle-mounted audio and video processing application end of a client to access a vehicle-mounted audio and video processing process.

[0058] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical solutions after these changes or replacements will all fall within the protection scope of the present application.

Claims

1. A method for in-vehicle audio and video processing based on SOA architecture, characterized in that, include: Step S1: Collect audio and video from around the vehicle, upload the collected audio to the cockpit domain controller, and upload the collected video to the driver domain controller. Step S2: Optimize the uploaded video using a video processor and optimize the uploaded audio using a sound processor. Step S3: The optimized video is transmitted to the server. After receiving the video, the server sends user parameters to the video processor. Step S4: The video is transmitted from the server to the client of the cockpit domain controller; Step S5: The video is transmitted to the in-vehicle infotainment system via the client for real-time display; Step S6: While displaying the video in real time, perform time synchronization processing on the video in the video processor, the video in the client, and the audio in the sound processor. Step S7: After the time synchronization process is completed, the optimized audio and the video in the client are combined to obtain the combined audio and video. Step S8: The synthesized audio and video are transmitted to the interface module for storage and display, and the in-vehicle audio and video processing process is controlled by the application in the client. In step S8, the application terminal controls the in-vehicle audio and video processing process in several ways, including video display control, audio and video recording control, factory reset control, and fault status control. When performing video display control, the vehicle audio and video processing application sends signals to the interface module to display video and hide video. The interface module sends a notification signal to the rendering module. The rendering module sends the rendered video or the hidden video to the client's screen according to the notification signal. When performing video display control, the interfaces exposed by the interface module to the vehicle audio and video processing application include the display video interface and the hidden video interface. When controlling audio and video recording, the vehicle audio and video processing application sends an application interface call signal to the interface module, the interface module sends a notification signal to the rendering module, the rendering module sends the audio and video file to the disk, the disk stores the audio and video file, and after storage, the disk returns the interface call result to the rendering module. After receiving the returned interface call result from the disk, the rendering module returns the interface call result to the interface module, and the interface module then returns the interface call result to the vehicle audio and video processing application. When performing factory reset control, the vehicle audio and video processing application sends a factory reset signal to the interface module, and the interface module sends a notification signal to the factory reset module. During fault status control, the vehicle audio and video processing application sends a registration callback signal to the interface module, the interface module sends a notification signal to the fault status module, the fault status module sends a status change notification after receiving the notification signal, and sends a callback notification to the interface module. After receiving the callback notification, the interface module sends the callback notification to the vehicle audio and video processing application.

2. The in-vehicle audio and video processing method based on SOA architecture according to claim 1, characterized in that, In step S1, during video acquisition, the video of the vehicle's surroundings recorded by the camera is acquired through the video interface. In step S2, the video optimization processing includes stabilizing the video, dehazing, scaling the video, overlaying the video, and encoding the video.

3. The in-vehicle audio and video processing method based on SOA architecture according to claim 2, characterized in that, In step S1, audio from around the vehicle is collected via a microphone interface. In step S2, the audio is optimized using a sound processor, which includes noise suppression and audio format encoding.

4. The in-vehicle audio and video processing method based on SOA architecture according to claim 1, characterized in that, In step S3, the optimized video is transmitted to the server via a video data transmitter, and user parameters are sent to the video processor via a user parameter receiver on the server. The video data transmitter transmits the optimized video to the server via a real-time streaming protocol and audio / video bridging technology.

5. The in-vehicle audio and video processing method based on SOA architecture according to claim 1, characterized in that, In step S4, video is transmitted from the server of the driving domain controller to the client of the cockpit domain controller via Ethernet based on the real-time streaming protocol. The video format is H.

264.

6. The in-vehicle audio and video processing method based on SOA architecture according to claim 1, characterized in that, In step S6, the video in the video processor, the video in the client, and the audio in the sound processor are synchronized in time using a global clock. In step S7, the audio and video are synthesized using a real-time transmission processor.

7. The in-vehicle audio and video processing method based on SOA architecture according to claim 1, characterized in that, In step S8, the synthesized audio and video are stored in the mobile hard drive of the interface module, and when the adaptive automatic braking signal is acquired by the SOA interface, the synthesized audio and video are transmitted to the interface module for storage.

8. The in-vehicle audio and video processing method based on SOA architecture according to claim 1, characterized in that, The driving domain controller includes a video processor, a server-side video data transmitter, and a server-side user parameter receiver. The cockpit domain controller includes a sound processor, a real-time transmission processor, a client, and an interface module. The client includes a video data receiver, a video parameter transmitter, and an application terminal. The interface module includes a data storage module and a display module. The data storage module includes a storage strategy unit and a portable hard drive. The display module includes an encoder and an in-vehicle entertainment information display device.

9. The in-vehicle audio and video processing method based on SOA architecture according to claim 8, characterized in that, The portable hard drive has USB 2.0 and USB 3.0 interfaces, and the driving domain controller has Ethernet interfaces.

Images