Distributed video transmission architecture and method based on swan gap system, camera and equipment

Abstract

The embodiment of the invention discloses a distributed video transmission architecture and method based on a swan gap system, a camera and equipment. In the video transmission framework, a video preview stream supporting acquisition of an MJPEG format is added to a camera service layer interface on the basis of an original camera driving framework of a swan gap system through an improved camera driving framework; processing the PipeLineCore layer and improving the PipeLineCore layer to output a video preview stream in an MJPEG (Multimedia Joint Photographic Experts Group) format; adding an MJPEG format definition in the improved distributed camera; during video transmission, a camera client of an improved distributed controlled end selects a video preview stream in an MJPEG format and sends the video preview stream to a camera server; the surface is directly called in the coding processing of the improved distributed controlled end; and calling an MJPEG decoder to decode the video stream in the decoding processing of the improved distributed master control end, and transmitting the decoded video stream to the virtual camera. The video transmission efficiency can be improved by improving the original video stream processing of the Nan Mongolian system.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a distributed video transmission architecture, method, camera and device based on the Hongmeng system. Background Art

[0002] With the localization of computer technology, OpenHarmony has achieved great development.

[0003] The distributed camera processing flow under the traditional OpenHarmony system is clear and smooth. However, the OpenHarmony distributed camera does not support the motion still image (or frame-by-frame) compression technology (Motion Joint Photographic Experts Group, MJPEG) encoding method. For MJPEG video streams, the distributed camera processing under the traditional OpenHarmony system must first decode and then encode into High Efficiency Video Coding (H265), standard advanced video coding (H.264 / MPEG-4 AVC, H264) or Moving Picture Experts Group 4 (Moving Picture Experts Group 4, MPEG-4) format.

[0004] In this way, the traditional OpenHarmony distributed camera framework will have the following problems for MJPEG cameras: 1) Since the original MJPEG must be decompressed into the three-primary color mode (RGB), then converted to color encoding (yuv), and then encoded into MPEG-4, H264 or H265 for soft bus transmission, it causes excessive occupation of processor resources; 2) Since the data is compressed and then decompressed, the latency is increased; 3) During data processing, the video quality is reduced. Summary of the invention

[0005] The present invention provides a distributed video transmission architecture, method, camera and device based on the Hongmeng system, so that the OpenHarmony distributed camera framework supports MJPEG cameras and improves the transmission efficiency of MJPEG format video streams.

[0006] According to one aspect of the present invention, a distributed video transmission architecture based on the Hongmeng system is provided, and the video transmission architecture includes: an improved camera driving framework, an improved distributed controlled terminal, an improved distributed master terminal and a virtual camera; wherein:

[0007] The improved camera driver framework is based on the original camera driver framework of the Hongmeng system, and adds support for obtaining the video preview stream in MJPEG format in the camera service layer interface; and improves the StreamPipeLineCore processing in the PipeLineCore layer to output the video preview stream in MJPEG format;

[0008] Adding MJPEG format definition in the improved distributed cameras to which the improved distributed controlled terminal and the improved distributed master control terminal belong;

[0009] During video transmission, the camera client of the improved distributed controlled end selects a video preview stream in MJPEG format and sends it to the camera service;

[0010] The improved distributed controlled end directly calls the shared memory surface in the encoding process without creating an encoding service;

[0011] In the decoding process of the improved distributed master control end, the MJPEG decoder is called to decode the video stream transmitted by the improved distributed controlled end, and the decoded video stream is transmitted to the virtual camera.

[0012] Optionally, add support for obtaining the video preview stream in MJPEG format to the camera service layer interface, including:

[0013] Add OHOS_CAMERA_FORMAT_MJPEG definition in extendAvailableConfigurations in camera_host_config.hcs file to enable the camera application layer to obtain the video preview stream in MJPEG format.

[0014] Optionally, improve the StreamPipeLineCore processing in the PipeLineCore layer to output a video preview stream in MJPEG format, including:

[0015] Remove the RKCodecNode processing and RKFaceNode processing of the preview stream in the StreamPipeLineCore processing of the PipeLineCore layer.

[0016] Optionally, adding MJPEG format definition in the improved distributed camera to which the improved distributed controlled terminal and the improved distributed master control terminal belong includes:

[0017] Add OHOS_CAMERA_FORMAT_MJPEG definition to DCameraFormat in distributed_camera_constants.h file.

[0018] Optionally, during video transmission, the camera client of the improved distributed controlled end selects a video preview stream in MJPEG format and sends it to the camera service, including:

[0019] During video transmission, when the camera client of the improved distributed controlled end determines that info->format is equal to OHOS_CAMERA_FORMAT_MJPEG, it turns on the camera to capture the video stream and sends it to the camera service.

[0020] Optionally, the improved encoding process of the distributed controlled terminal directly calls the shared memory surface without creating an encoding service, including:

[0021] The improved distributed controlled end directly calls Surface::createSurface to create a surface in the encoding process, without creating an encoding service;

[0022] The MJPEG video stream generated by the camera is stored in the buffer generated by the surface.

[0023] Optionally, the video transmission architecture further includes:

[0024] In the encoding process of the improved distributed controlled end, the video stream in the buffer generated by the surface is obtained through the GetEncoderOutputBuffer function in the encode_data_process.cpp file, and the video stream is transmitted to the improved distributed master control end.

[0025] According to another aspect of the present invention, a distributed video transmission method based on the Hongmeng system is provided. The video transmission method is applied to a distributed video transmission architecture based on the Hongmeng system provided in any embodiment of the present invention; the method comprises:

[0026] Pulling a video stream in MJPEG format by improving the camera driving framework, and transmitting the video stream in MJPEG format to the improved distributed controlled terminal;

[0027] Processing the video stream in MJPEG format by improving the distributed controlled terminal, and transmitting the processed video stream in MJPEG format to the improved distributed master control terminal;

[0028] Perform MJPEG decoding on the processed MJPEG format video stream by improving the decoding process in the distributed master control end;

[0029] The decoded and processed video stream is transmitted to the virtual camera by improving the distributed master control terminal.

[0030] According to another aspect of the present invention, a distributed camera based on the Hongmeng system is provided, and the distributed camera includes a distributed video transmission architecture based on the Hongmeng system as provided in any embodiment of the present invention.

[0031] According to another aspect of the present invention, an electronic device is provided, comprising a distributed camera based on the Hongmeng system as provided in any embodiment of the present invention.

[0032] The technical solution of the embodiment of the present invention improves the distributed video transmission architecture of the Hongmeng system. Specifically, based on the original camera driver framework of the Hongmeng system, support for obtaining the video preview stream in MJPEG format is added to the camera service layer interface; and the StreamPipeLineCore processing in the PipeLineCore layer is improved to output the video preview stream in MJPEG format; the MJPEG format definition is added to the improved distributed camera to which the improved distributed controlled end and the improved distributed master end belong; during video transmission, the camera client of the improved distributed controlled end selects the video preview stream in MJPEG format and sends it to the camera service; in the encoding process of the improved distributed controlled end, the shared memory surface is directly called without creating an encoding service; in the decoding process of the improved distributed master end, the MJPEG decoder is called to decode the video stream transmitted by the improved distributed controlled end, and the decoded video stream is transmitted to the virtual camera, which solves the problem that the original camera driver framework of the Hongmeng system does not support the MJPEG format video stream, and can improve the resource utilization of the distributed architecture of the Hongmeng system when performing MJPEG format video processing, reduce latency, and improve picture quality.

[0033] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 This is the original camera video stream processing flow chart of Hongmeng system;

[0036] Figure 2 This is the camera framework structure diagram of the Hongmeng system;

[0037] Figure 3This is a schematic diagram of the PipeLine processing flow in the camera driver framework of the Hongmeng system;

[0038] Figure 4 is a schematic diagram of the processing flow of PipeLine in the improved camera driving framework provided according to an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the video processing flow of the distributed camera framework in the Hongmeng system;

[0040] Figure 6 It is a flowchart of a distributed video transmission method based on the Hongmeng system provided according to an embodiment of the present invention;

[0041] Figure 7 is a structural schematic diagram of a distributed camera based on the Hongmeng system provided according to an embodiment of the present invention;

[0042] Figure 8 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] Figure 1 This is the original camera video stream processing flow chart of Hongmeng system. Figure 1As shown in FIG, the distributed camera processing flow under the traditional OpenHarmony system includes the camera driver framework, the distributed camera framework, and the codec service (FFmpeg Audio / Video Codec Library, avcodec). Figure 1 As shown in the figure, the camera driver framework pulls the MJPEG format video stream, decodes the MJPEG to generate RGB, converts the format into yuv, and compresses the video stream to transmit it to the distributed camera framework. The distributed camera framework includes a distributed controlled end and a distributed master control end. The distributed controlled end can perform video decompression, format conversion, and video recompression to generate MPEG-4, H264 or H265 format, which is transmitted to the distributed master control end through the soft bus. The distributed master control end can decode the video and transmit the decoded video stream to the virtual camera.

[0046] In such Figure 1 The original camera video stream processing of the Hongmeng system shown in the figure cannot utilize the MJPEG camera's own MJPEG compression. The distributed controlled end needs to perform video decompression, format conversion, and video recompression, resulting in a large waste of resources. In order to overcome the problem of high resource consumption, the inventor improved the original distributed camera architecture of the Hongmeng system so that the distributed camera of the Hongmeng system supports the MJPEG encoding method.

[0047] Among them, MJPEG is a video encoding format, which is widely used in the field of non-linear editing. It can perform frame-accurate editing and multi-layer image processing, and process moving video sequences as continuous still images. MJPEG compression method compresses each frame separately and completely, and can randomly store each frame during the editing process, so that frame-accurate editing can be performed. In addition, the compression and decompression of MJPEG are symmetrical and can be implemented by the same hardware and software. However, MJPEG only compresses the spatial redundancy within the frame. It does not compress the temporal redundancy between frames. When the MJPEG digital compression format is used, when the compression ratio is 7:1, it can provide images with the quality equivalent to Betacam SP (a video format service package).

[0048] Figure 2 This is the camera framework structure diagram of the Hongmeng system. Figure 2 As shown in the figure, the camera framework structure of the Hongmeng system includes: Camera Application, Camera Service, Hardware Driver Interface (HDI), Pipeline Core (PipeLineCore), Device Manager, Platform Adapter and Kernel Layer.

[0049] Among them, PipeLineCore is one of the core components in the camera software architecture, which is mainly responsible for camera data processing and process control. The main functions of PipeLineCore include: data flow management, image processing, autofocus and exposure control, image format conversion, image data encoding and compression, and pipeline control and configuration.

[0050] The distributed video transmission architecture based on the Hongmeng system provided in the embodiment of the present invention includes: an improved camera driving framework, an improved distributed controlled terminal, an improved distributed master terminal and a virtual camera. In the improved camera driving framework, the PipeLineCore layer in the camera framework structure is mainly improved.

[0051] Specifically, the improved camera driver framework is to add support for obtaining the video preview stream in MJPEG format to the camera service layer interface based on the original camera driver framework of the Hongmeng system; and improve the StreamPipeLineCore processing in the PipeLineCore layer to output the video preview stream in MJPEG format.

[0052] By adding support for obtaining the video preview stream in MJPEG format in the camera service layer interface and outputting the video preview stream in MJPEG format, the distributed cameras of the Hongmeng system can obtain the video preview stream in MJPEG format.

[0053] Optionally, add support for obtaining the video preview stream in MJPEG format to the camera service layer interface, including:

[0054] Add OHOS_CAMERA_FORMAT_MJPEG definition in extendAvailableConfigurations in camera_host_config.hcs file to enable the camera application layer to obtain the video preview stream in MJPEG format.

[0055] By modifying the camera_host_config.hcs file, you can add OHOS_CAMERA_FORMAT_MJPEG (camera service supports MJPEG format) to the video formats supported by the camera service layer interface, which can support video preview streams in MJPEG format and enable the camera application layer to obtain video preview streams in MJPEG format.

[0056] Among them, the camera_host_config.hcs file is the configuration file for the camera driver in the Hongmeng system. The camera_host_config.hcs file is located in the device configuration directory of the Hongmeng system and can realize the bridge between the camera driver and the user-mode application. The camera_host_config.hcs file defines the configuration information required during the startup and operation of the camera driver framework.

[0057] extendAvailableConfigurations is a configuration item in the camera_host_config.hcs file. By adding the OHOS_CAMERA_FORMAT_MJPEG definition in extendAvailableConfigurations, the video stream format supported by the camera can be extended, even if the Hongmeng system distributed camera architecture supports MJPEG format video streams.

[0058] In the original Hongmeng system distributed camera service interface, the supported video formats are:

[0059] typedef enum camera_format{

[0060] OHOS_CAMERA_FORMAT_RGBA_8888=1,

[0061] OHOS_CAMERA_FORMAT_YCBCR_420_888,

[0062] OHOS_CAMERA_FORMAT_YCRCB_420_SP,

[0063] OHOS_CAMERA_FORMAT_JPEG,

[0064] }camera_format_t;

[0065] In the embodiment of the present invention, the MJPEG format is added to the above original format, that is, OHOS_CAMERA_FORMAT_MJPEG is added.

[0066] The video formats supported by the improved distributed cameras of Hongmeng system are:

[0067] typedef enum camera_format{

[0068] OHOS_CAMERA_FORMAT_RGBA_8888=1,

[0069] OHOS_CAMERA_FORMAT_YCBCR_420_888,

[0070] OHOS_CAMERA_FORMAT_YCRCB_420_SP,

[0071] OHOS_CAMERA_FORMAT_MJPEG,

[0072] }camera_format_t;

[0073] On the basis of improving the camera driver framework to support obtaining the video preview stream in MJPEG format, it is also necessary to set the output of the improved camera driver framework to output the video preview stream in MJPEG format.

[0074] Optionally, the StreamPipeLineCore processing in the PipeLineCore layer is improved to output a video preview stream in MJPEG format, including: removing the RKCodecNode processing and RKFaceNode processing of the preview stream in the StreamPipeLineCore processing of the PipeLineCore layer.

[0075] In the PipeLineCore layer, StreamPipeLineCore is one of the core components. StreamPipeLineCore is a core class in the camera module, responsible for creating and managing the PipeLine of the data stream. PipeLine consists of multiple nodes, each of which performs specific processing on the data and then passes it to the next node. Figure 3 This is a schematic diagram of the PipeLine processing flow in the camera driver framework of the Hongmeng system. Figure 3 As shown in the figure, V4L2SourceNode (video source node based on Video4Linux 2 framework) reads the camera stream from the kernel; ForkNode (fork node) copies 1 video stream into 2 video streams, one for video preview and one for video / capture; RKCodecNode0 (decoding node) is the codec of the video; SinkNode (downstream node, data stream end point) mainly sends the video stream to the distributed framework. The original RkCodecNode module receives the MJPEG camera stream for decoding, and the decoded data format is RGB. After converting RGB to YUV, it is sent to the distributed camera framework through the camera service.

[0076] Figure 4 Schematic diagram of the processing flow of PipeLine in the improved camera driving framework provided in an embodiment of the present invention. Figure 4As shown, in this embodiment of the present invention, the RKCodecNode processing and RKFaceNode (face recognition or feature extraction node) processing of the preview stream in the StreamPipeLineCore processing of the PipeLineCore layer are removed. In the preview stream processing, the RkCodecNode0 module directly sends the MJPEG decoding to SinkNode0, and then sends it to the distributed camera framework through the camera service.

[0077] After improving the camera driver framework, it is necessary to improve the distributed camera. Figure 5 This is a schematic diagram of the video processing flow of the distributed camera framework in the Hongmeng system. Figure 5 As shown, in the original process of the Hongmeng system, the EncodecProcess (encoding process) in the distributed controlled end pulls the camera stream from the camera driver framework, encodes and compresses it into an H264 / H264 / MPEG4 video stream, and sends the encoded video stream to the ChannelSink (channel receiver), and the ChannelSink module sends it to the distributed master through the soft bus. The distributed master receives the compressed video stream from the ChannelSource (channel source) and sends it to the DecodeProcess (decoding process). The DecoderProcess decodes the video stream into RGB / YUV, and then sends the RGB / YUV to the VirtualCameraHal (virtual camera) layer. VirtualCameraHal simulates the data as a camera. The camera application can preview the VirtualCameraHal camera like opening a physical camera.

[0078] In the embodiment of the present invention, the main Figure 5 Improvements have been made to EncodecProcess and DecodeProcess in .

[0079] Specifically, in an embodiment of the present invention, the improvement of the distributed camera includes: adding MJPEG format definition in the improved distributed camera to which the improved distributed controlled end and the improved distributed master end belong; during video transmission, the camera client of the improved distributed controlled end selects the video preview stream in MJPEG format and sends it to the camera service; in the encoding process of the improved distributed controlled end, the shared memory surface is directly called without creating an encoding service; in the decoding process of the improved distributed master end, the MJPEG decoder is called to decode the video stream transmitted by the improved distributed controlled end, and the decoded video stream is transmitted to the virtual camera.

[0080] Among them, adding the MJPEG format definition in the improved distributed camera can perform data transmission in the MJPEG format within the distributed camera framework. Optionally, adding the MJPEG format definition in the improved distributed camera to which the improved distributed controlled end and the improved distributed master end belong, including: adding the OHOS_CAMERA_FORMAT_MJPEG (distributed camera supports MJPEG format) definition to DCameraFormat (enumerates the image data formats supported by the distributed camera) in the distributed_camera_constants.h file (a header file used to define some constants related to the distributed camera functions). After adding the MJPEG format definition, when the distributed camera applies for the camera stream, the camera capability obtained supports MJPEG format video streams. For example, in the preview of the distributed camera, you can select a stream with encoding type 5 (MJPEG) and send it to the camera service.

[0081] After adding the MJPEG format definition in the improved distributed camera, the distributed controlled end can select the video stream in MJPEG format for processing. Optionally, during video transmission, the improved distributed controlled end camera client selects the video preview stream in MJPEG format and sends it to the camera service, including: during video transmission, when the improved distributed controlled end camera client determines that info->format is equal to OHOS_CAMERA_FORMAT_MJPEG, opens the camera to capture the video stream and sends it to the camera service; otherwise, skips the video stream in this format.

[0082] In the processing flow of the EncodeProcess of the original distributed controlled end, encode_data_process.cpp (data encoding process file) first creates an encoding service, and returns a Surface (shared memory) when creating the encoding service. EncodeProcess returns this surface to camera_framework (camera management framework), and then the camera framework and the encoding service transmit data through the surface. After the encoding service obtains the camera stream, it performs H265 / H264 / MPEG4 encoding, and then the encoding service sends the data to EncodeProcess. EncodeProcess then sends the data to the distributed master through the soft bus.

[0083] Surface is a shared memory used to manage and transfer graphics and media. Specific scenarios include graphics display and synthesis, media playback and recording, etc. Surface's cross-process transmission uses control structures such as IPC transfer handles (with copies) and uses shared memory to transfer graphics / media data (zero copy).

[0084] In an embodiment of the present invention, the distributed controlled end is improved to directly call the shared memory surface in the encoding process without creating an encoding service. Optionally, the distributed controlled end is improved to directly call the shared memory surface in the encoding process without creating an encoding service, including: improving the distributed controlled end to directly call Surface::createSurface to create a surface in the encoding process without creating an encoding service; the MJPEG format video stream generated by the camera is stored in the cache generated by the surface.

[0085] By directly calling Surface::createSurface (a static method for creating a surface) in the EncodeProcess module encode_data_process.cpp (the source file that implements the distributed camera encoding process), without creating an encoding service, this surface is returned to the camera_framework. The MJPEG frame data generated by the camera will be placed in the buffer generated by this surface.

[0086] Optionally, the video transmission architecture also includes: obtaining the video stream in the buffer generated by the surface through the GetEncoderOutputBuffer function (a function for obtaining data in the encoder output buffer) in the decode_data_process.cpp file (an encoding processing file) in the improved distributed controlled end during encoding processing, and transmitting the video stream to the improved distributed master end. Obtain a frame of data through the RequesetBuffer method (request buffer) of the surface created above, and send the frame of data to the distributed master end through the soft bus.

[0087] After improving the distributed controlled end, the distributed master control end may be improved. Specifically, in the decoding process of the improved distributed master control end, the MJPEG decoder is called to decode the video stream transmitted by the improved distributed controlled end, and the decoded video stream is transmitted to the virtual camera.

[0088] The main function of the original distributed camera framework master control end DecdeProcess module is to decode the video stream (H265 / H264 / MPEG4) and send the decoded YUV / RGB to the VirtualCamera Hal layer. The original module does not support MJPEG format video streams. In the embodiment of the present invention, if the distributed camera framework master control end DecdeProcess finds that the data stream is MJPEG, the MJPEG decoder is called for decoding, and the decoded data is processed according to the standard distributed camera stream. Among them, the MJPEG decoder is avcodec, soft decoding or system built-in decoder.

[0089] The technical solution of the embodiment of the present invention improves the camera driving framework, distributed controlled terminals and distributed master terminals in the distributed architecture of the original Hongmeng system, so that the OpenHarmony system can support the real-time video stream acquisition and transmission technology stack based on MPEGJ camera adaptation, optimizes the real-time video stream acquisition and transmission architecture of the distributed camera of the OpenHarmony system, maximizes the use of the MJPEG camera's own video MJPEG compression, eliminates the decompression, format conversion and video recompression process of the distributed controlled terminal, and reduces the use of resources; MJPEG can be directly decoded and displayed on the controlled terminal, reducing the delay of the distributed camera; the original MJPEG video stream is used to retain higher picture quality, avoiding the quality loss that may occur during the re-compression process of compression standards such as h264 / h264 / mpeg-4. Compared with the native OpenHarmony system, the technical solution of the embodiment of the present invention can significantly improve the resource utilization of distributed cameras, reduce delays, and improve picture quality.

[0090] Figure 6 1 is a flow chart of a distributed video transmission method based on the Hongmeng system provided in accordance with an embodiment of the present invention. The method can be applied to the distributed video transmission architecture based on the Hongmeng system provided in any embodiment of the present invention. Figure 6 As shown, the method comprises:

[0091] By improving the camera driver framework, the video stream in MJPEG format is pulled, and the video stream in MJPEG format is transmitted to the improved distributed controlled terminal; the video stream in MJPEG format is processed by the improved distributed controlled terminal, and the processed video stream in MJPEG format is transmitted to the improved distributed master control terminal; the processed video stream in MJPEG format is decoded by MJPEG by improving the decoding process in the distributed master control terminal; the decoded video stream is transmitted to the virtual camera by the improved distributed master control terminal.

[0092] Through the above-mentioned distributed video transmission method based on the Hongmeng system, the MJPEG format video stream can be directly processed in the distributed camera, eliminating MJPEG decompression, image format conversion, and image recompression at the distributed controlled end, thereby improving resource utilization; by directly decoding and displaying the MJPEG format video stream at the distributed controlled end, the efficiency and real-time performance of video processing are improved; and by reducing the secondary lossy encoding, the picture quality can be improved.

[0093] Figure 7 It is a structural schematic diagram of a distributed camera based on the Hongmeng system provided according to an embodiment of the present invention. The distributed camera includes a distributed video transmission architecture based on the Hongmeng system provided in any embodiment of the present invention.

[0094] Figure 8 It is a structural schematic diagram of an electronic device provided according to an embodiment of the present invention, and the electronic device includes a distributed camera based on the Hongmeng system as provided in any embodiment of the present invention.

[0095] The distributed cameras based on the Harmony system and the electronic devices in the embodiments of the present invention can adopt the distributed video transmission architecture based on the Harmony system provided in any embodiment of the present invention to implement the distributed video transmission method based on the Harmony system, so as to achieve the technical effects of improving resource utilization, improving the efficiency and real-time performance of video processing, and improving picture quality.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0097] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed video transmission architecture based on Hongmeng system, characterized in that: The video transmission architecture includes: an improved camera driving framework, an improved distributed controlled terminal, an improved distributed master control terminal and a virtual camera; wherein: The improved camera driver framework is based on the original camera driver framework of the Hongmeng system, and adds support for obtaining the video preview stream in MJPEG format in the camera service layer interface; and improves the StreamPipeLineCore processing in the PipeLineCore layer to output the video preview stream in MJPEG format; Adding MJPEG format definition in the improved distributed cameras to which the improved distributed controlled terminal and the improved distributed master control terminal belong; During video transmission, the camera client of the improved distributed controlled end selects a video preview stream in MJPEG format and sends it to the camera service; The improved distributed controlled end directly calls the shared memory surface in the encoding process without creating an encoding service; In the decoding process of the improved distributed master control end, the MJPEG decoder is called to decode the video stream transmitted by the improved distributed controlled end, and the decoded video stream is transmitted to the virtual camera.

2. The video transmission architecture according to claim 1, characterized in that: Add support for obtaining the video preview stream in MJPEG format in the camera service layer interface, including: Add OHOS_CAMERA_FORMAT_MJPEG definition in extendAvailableConfigurations in camera_host_config.hcs file to enable the camera application layer to obtain the video preview stream in MJPEG format.

3. The video transmission architecture according to claim 1, characterized in that: Improve the StreamPipeLineCore processing in the PipeLineCore layer to output the video preview stream in MJPEG format, including: Remove the RKCodecNode processing and RKFaceNode processing of the preview stream in the StreamPipeLineCore processing of the PipeLineCore layer.

4. The video transmission architecture according to claim 1, characterized in that: Adding MJPEG format definition in the improved distributed camera to which the improved distributed controlled terminal and the improved distributed master control terminal belong includes: Add OHOS_CAMERA_FORMAT_MJPEG definition to DCameraFormat in distributed_camera_constants.h file.

5. The video transmission architecture according to claim 1, characterized in that: During video transmission, the camera client of the improved distributed controlled end selects a video preview stream in MJPEG format and sends it to the camera service, including: During video transmission, when the camera client of the improved distributed controlled end determines that info->format is equal to OHOS_CAMERA_FORMAT_MJPEG, it turns on the camera to capture the video stream and sends it to the camera service.

6. The video transmission architecture according to claim 1, characterized in that: The improved distributed controlled end encoding process directly calls the shared memory surface without creating an encoding service, including: The improved distributed controlled end directly calls Surface::createSurface to create a surface in the encoding process, without creating an encoding service; The MJPEG video stream generated by the camera is stored in the buffer generated by the surface.

7. The video transmission architecture according to claim 6, characterized in that: Also includes: In the encoding process of the improved distributed controlled end, the video stream in the buffer generated by the surface is obtained through the GetEncoderOutputBuffer function in the encode_data_process.cpp file, and the video stream is transmitted to the improved distributed master control end.

8. A distributed video transmission method based on Hongmeng system, characterized in that: The video transmission method is applied to the distributed video transmission architecture based on the Hongmeng system as described in any one of claims 1 to 7; the method comprises: Pulling a video stream in MJPEG format by improving the camera driving framework, and transmitting the video stream in MJPEG format to the improved distributed controlled terminal; Processing the video stream in MJPEG format by improving the distributed controlled terminal, and transmitting the processed video stream in MJPEG format to the improved distributed master control terminal; Perform MJPEG decoding on the processed MJPEG format video stream by improving the decoding process in the distributed master control end; The decoded and processed video stream is transmitted to the virtual camera by improving the distributed master control terminal.

9. A distributed camera based on Hongmeng system, characterized in that: The distributed camera comprises a distributed video transmission architecture based on the Hongmeng system as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device includes a distributed camera based on the Hongmeng system as described in claim 9.

Images