SOC-based camera data preview method and device, medium and chip

Abstract

The invention discloses an SOC-based camera data preview method and device, a medium and a chip, and belongs to the technical field of chips. Creating a streaming media processing pipeline in the Linux system; acquiring camera data acquired by the Ethernet camera by using the data receiving plug-in; the data processing plug-in is used for processing camera data, and obtained video coding data is sent to the decoding plug-in; decoding the video coding data by using a decoding plug-in, and sharing the obtained preview image to a display plug-in; and sharing the preview image to a graphic pipeline GPIPE layer in a display controller DC by using a display plug-in, so that the DC displays the preview image on a screen as an upper layer picture, and the DC is controlled by the RTOS. According to the application, the streaming media processing pipeline is used for interacting with the RTOS system to decode the camera data, and the streaming media processing pipeline is used for interacting with the RTOS system to display the preview image on the screen, so that the camera data can be quickly previewed.

Description

Technical Field

[0001] This application relates to the field of chip technology, and particularly to a method, device, medium, and chip for previewing camera data based on an SOC. Background Art

[0002] A multi-core heterogeneous system on chip (SOC) includes multiple cores, and each core is configured to run an operating system. For example, one core is configured to run the Linux system, and one core is configured to run a real-time operating system (RTOS).

[0003] How to quickly preview the data captured by an Ethernet camera in an intelligent cockpit through the Linux system and the RTOS system has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method, device, medium, and chip for previewing camera data based on an SOC, which is used to solve the problem that the data captured by an Ethernet camera cannot be quickly previewed through the Linux system and the RTOS system. The technical solutions are as follows:

[0005] According to the first aspect of this application, a method for previewing camera data based on an SOC is provided. The first group of cores in the SOC is configured to run the Linux system, the second group of cores is configured to run the real-time operating system RTOS, and the first group of cores and the second group of cores are different from each other. The method includes:

[0006] Create a streaming media processing pipeline in the Linux system. The streaming media processing pipeline is obtained by combining multiple streaming media plugins using the open-source framework Gstreamer. The streaming media plugins at least include a data reception plugin, a data processing plugin, a decoding plugin, and a display plugin;

[0007] Use the data reception plugin to obtain the camera data collected by the Ethernet camera, and send the camera data to the data processing plugin;

[0008] Use the data processing plugin to process the camera data, and send the obtained video encoded data to the decoding plugin;

[0009] Use the decoding plugin to decode the video encoded data through shared memory, and share the obtained preview image to the display plugin;

[0010] Use the display plugin to share the preview image with the GPIPE layer in the display controller DC, so that the DC displays the preview image as the upper layer on the screen, and the DC is controlled by the RTOS.

[0011] In a possible implementation, the use of the decoding plugin to decode the video encoding data through shared memory and share the obtained preview image with the display plugin includes:

[0012] Use the decoding plugin to store the video encoding data in the first shared memory, and send the first memory address of the first shared memory to the VPU driver in the RTOS system;

[0013] Use the VPU driver to read the video encoding data from the first shared memory according to the first memory address, decode the video encoding data into a preview image and store it in the second shared memory, and send the second memory address of the second shared memory to the decoding plugin;

[0014] Use the decoding plugin to send the second memory address to the display plugin.

[0015] In a possible implementation, the sending of the first memory address of the first shared memory to the VPU driver in the RTOS system includes:

[0016] Use the decoding plugin to send the first memory address of the first shared memory to the VPU client in the Linux system;

[0017] Use the VPU client to send the first memory address to the VPU server in the RTOS system;

[0018] Use the VPU server to send the first memory address to the VPU driver.

[0019] In a possible implementation, the sending of the second memory address of the second shared memory to the decoding plugin includes:

[0020] Use the VPU driver to send the second memory address of the second shared memory to the VPU server;

[0021] Use the VPU server to send the second memory address to the VPU client in the Linux system;

[0022] Use the VPU client to send the second memory address to the decoding plugin.

[0023] In a possible implementation, sharing the preview image with the GPIPE layer in the DC by using the display sending plugin includes:

[0024] Sending the second memory address to the display rendering manager DRM library file driver by using the display sending plugin;

[0025] Sending the second memory address to the GPIPE layer in the DC by using the DRM library file driver, so that the GPIPE layer reads the preview image from the second shared memory according to the second memory address.

[0026] In a possible implementation, creating a streaming media processing pipeline in the Linux system includes:

[0027] When starting the Linux system, creating a streaming media processing pipeline in the Linux system.

[0028] In a possible implementation, processing the camera data by using the data processing plugin and sending the obtained video encoding data to the decoding plugin includes:

[0029] When the data processing plugin includes a protocol header parsing plugin, a data stream de-encapsulation plugin, and a data stream parsing plugin, using the protocol header parsing plugin to perform protocol header parsing on the camera data and sending the obtained transport stream TS stream to the data stream de-encapsulation plugin;

[0030] Using the data stream de-encapsulation plugin to de-encapsulate the TS stream and sending the obtained encoded data stream to the data stream parsing plugin;

[0031] Using the data stream parsing plugin to parse the encoded data stream and sending the obtained video encoding data to the decoding plugin.

[0032] According to the second aspect of the present application, a camera data preview device based on an SOC is provided for use in an SOC. The first group of cores in the SOC is configured to run a Linux system, the second group of cores is configured to run a real-time operating system RTOS, and the first group of cores and the second group of cores are different from each other;

[0033] A creation module, configured to create a streaming media processing pipeline in the Linux system. The streaming media processing pipeline is obtained by combining multiple streaming media plugins by using an open-source framework Gstreamer. The streaming media plugins at least include a data receiving plugin, a data processing plugin, a decoding plugin, and a display sending plugin;

[0034] An acquisition module, configured to use the data reception plug-in to acquire camera data collected by an Ethernet camera and send the camera data to the data processing plug-in;

[0035] A processing module, configured to use the data processing plug-in to process the camera data and send the obtained video encoded data to the decoding plug-in;

[0036] A decoding module, configured to use the decoding plug-in to decode the video encoded data through shared memory and share the obtained preview image to the display sending plug-in;

[0037] A display sending module, configured to use the display sending plug-in to share the preview image to the GPIPE layer in the graphics pipeline of the display controller DC, so that the DC displays the preview image as the upper layer picture on the screen, and the DC is controlled by the RTOS.

[0038] According to a third aspect of the present application, there is provided a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned SOC-based camera data preview method.

[0039] According to a fourth aspect of the present application, there is provided a chip, and the chip includes the above-mentioned SOC-based camera data preview device.

[0040] The beneficial effects of the technical solution provided by the present application at least include:

[0041] By creating a streaming media processing pipeline in the Linux system, then using the streaming media processing pipeline to receive the camera data sent by the Ethernet camera, using the streaming media processing pipeline to interact with the RTOS system to decode the camera data, and using the streaming media processing pipeline to interact with the RTOS system to display the preview image on the screen, the camera data can be previewed quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 is a schematic structural diagram of an SOC shown according to some exemplary embodiments;

[0044] Figure 2 is a flowchart of an SOC-based camera data preview method provided by an embodiment of the present application;

[0045] Figure 3 It is a flowchart of a method for previewing camera data based on SOC provided by an embodiment of the present application;

[0046] Figure 4 It is a flowchart of a device for previewing camera data based on SOC provided by an embodiment of the present application. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0048] After the vehicle is cold-started or hot-started, if a quick reverse is required, it is necessary to quickly display the reverse image data or 360° panoramic data captured by the Ethernet camera in the intelligent cockpit, such as previewing the camera data within 3 seconds. For SOC, the network card is usually deployed in the Linux system, and the display driver is usually deployed in the RTOS system (safety). Therefore, the quick preview of camera data involves the integration of multiple systems.

[0049] The Ethernet camera integrates an H.264 (High Efficiency Video Coding) encoder inside. The H.264 encoder is used to perform H.264 encoding on the camera data and encapsulate it into a Transport Stream (TS). Then, after encapsulating the TS stream into data packets through the network protocol, the data packets are transmitted to the SOC through a 100M network interface. After the SOC's multi-system parses and decodes the data packets, they are previewed on the screen. The structure of the SOC will be described below.

[0050] The SOC includes multiple groups of cores. The first group of cores is configured to run the Linux system, and the second group of cores is configured to run the RTOS, and the first group of cores and the second group of cores are different from each other. As Figure 1 shown, the Linux system includes a User Space and a Kernel Space. The User Space includes a Gstreamer Pipeline, which is obtained by combining multiple streaming media plugins using the open-source framework Gstreamer. The streaming media plugins at least include a data reception plugin, a data processing plugin, a decoding plugin, and a display plugin.

[0051] The data receiving plugin can be UDPSrc, that is, User Datagram Protocol Source (UDPSrc). The data receiving plugin receives the data sent by the Ethernet camera from the physical (PHY) network port through the Ethernet Driver in the kernel space.

[0052] The data processing plugin can include a protocol header parsing plugin, a data stream unpacking plugin, and a data stream parsing plugin. Among them, the protocol header parsing plugin can be RTPMP2TDepay (RTP MPEG2-TS de-packager). RTP is the Real-time Transport Protocol, MPEG2 is the Moving Pictures Experts Group 2, TS is the Transport Stream, and the de-packager is Depay. The data stream unpacking plugin can be TSDemux, that is, the TS demultiplexer. The data stream parsing plugin can be H264Parse, that is, the H.264 parser.

[0053] The decoding plugin can be a decoding plugin for H.264 created based on the Open Media Acceleration (OpenMAX) framework, which can be named OMXH264Dec. The user space also includes the OpenMAX framework and VPUProto HAL. VPU is the Video Process Unit, Proto is the Protocol, and HAL is the Hardware Abstraction Layer. The kernel space also includes the VPU Dummy Driver and VPU Client. OMXH264Dec communicates with the VPU Dummy Driver and VPU Client in the kernel space through the OpenMAX framework and VPU Proto HAL in the user space.

[0054] The display sending plugin can be APPSink. The user space also includes DRM Display. DRM is the Display Rendering Manager. The kernel space also includes the LibDRM Driver. LibDRM is the DRM library file. APPSink communicates with the LibDRM Driver in the kernel space through DRM Display in the user space.

[0055] The RTOS system in the R5 core includes a VPU Server (server), a VPU HAL, and a VPU Driver. The VPU Driver communicates with the VPU Client in the Linux system through the VPU HAL and the VPU Server. The communication methods include shared memory and Remote Processor Messaging (RPMSG). The RTOS system also includes a DRM Driver and a DC. The DC is a Display Control. The DRM Driver is used to drive the DC, and the DC communicates with the LibDRM Driver in the Linux system. The communication methods include shared memory and Remote Procedure Call (RPCall).

[0056] The DC includes four layers, namely the Graphic Pipeline (GPIPE) layer, the Smart Pipeline (SPIPE) layer, the Display Processor (DP) 2 layer, and the DC1 layer. The display data in each layer is superimposed and then displayed on the screen. Among them, the display data in the GPIPE layer is the topmost picture.

[0057] As Figure 2 shown, it shows the method flow chart of the camera data preview method based on SOC provided by an embodiment of the present application. The camera data preview method based on SOC can be applied to Figure 1 the SOC shown. The camera data preview method based on SOC may include:

[0058] Step 201, create a streaming media processing pipeline in the Linux system.

[0059] Among them, the streaming media processing channel is Figure 1 the GstreamerPipeline shown.

[0060] Step 202, use the data receiving plugin to obtain the camera data collected by the Ethernet camera, and send the camera data to the data processing plugin.

[0061] In this embodiment, the port of the data receiving plugin and the port of the Ethernet camera can be specified so that the data receiving plugin can communicate with the Ethernet camera, thereby receiving the camera data sent by the Ethernet camera.

[0062] Step 203, use the data processing plugin to process the camera data, and send the obtained video encoded data to the decoding plugin.

[0063] The processing of camera data by the data processing plug-in includes but is not limited to: protocol header parsing, data stream unpacking and data stream parsing.

[0064] After the above series of processing operations, the data processing plug-in can obtain the video encoding data, and then send the video encoding data to the decoding plug-in.

[0065] Step 204: Use the decoding plug-in to decode the video encoding data through the shared memory, and share the obtained preview image with the display plug-in.

[0066] The decoding plug-in needs to communicate with the VPU driver in the RTOS system through the shared memory, so as to decode the video encoding data through the VPU driver to obtain a preview image, wherein the preview image can be a YUV image.

[0067] Step 205, using the display plug-in to share the preview image to the GPIPE layer in the DC, so that the DC displays the preview image as the upper layer image on the screen, and the DC is controlled by the RTOS.

[0068] The display plug-in needs to communicate with the DC in the RTOS system through shared memory to display the preview image on the screen.

[0069] To summarize, the SOC-based camera data preview method provided in the embodiment of the present application creates a streaming media processing pipeline in the Linux system, then uses the streaming media processing pipeline to receive the camera data sent by the Ethernet camera, uses the streaming media processing pipeline to interact with the RTOS system to decode the camera data, and uses the streaming media processing pipeline to interact with the RTOS system to display the preview image on the screen, thereby enabling quick preview of the camera data.

[0070] like Figure 3 FIG. 1 is a flowchart of a camera data preview method based on SOC provided by an embodiment of the present application. The camera data preview method based on SOC can be applied to Figure 1 The SOC-based camera data preview method may include:

[0071] Step 301, creating a streaming media processing pipeline in the Linux system.

[0072] If the present application is applied to a cold start application scenario, a streaming media processing pipeline is created in the Linux system when the Linux system is started. Specifically, the script of the present application can be placed in the init.rc script of the Linux system startup script, so that the script is run when the power is turned on, so as to realize a quick preview of the camera data when the power is turned on.

[0073] If this application is applied to a hot start application scenario, the script of this application can be executed when previewing camera data is required.

[0074] Among them, the streaming media processing channel is Figure 1 the GstreamerPipeline shown.

[0075] Step 302, use the data receiving plug-in to obtain the camera data collected by the Ethernet camera, and send the camera data to the data processing plug-in.

[0076] In this embodiment, the port of the data receiving plug-in and the port of the Ethernet camera can be specified so that the data receiving plug-in can communicate with the Ethernet camera, thereby receiving the camera data sent by the Ethernet camera.

[0077] The data receiving plug-in also needs to send the camera data to the data processing plug-in. When the data processing plug-in includes a protocol header parsing plug-in, a data stream de-encapsulation plug-in, and a data stream parsing plug-in, the data receiving plug-in specifically sends the camera data to the protocol header parsing plug-in.

[0078] Such as Figure 1 shown, UDPSrc sends the camera data to RTPMP2TDepay.

[0079] Step 303, use the data processing plug-in to process the camera data, and send the obtained video encoded data to the decoding plug-in.

[0080] Specifically, using the data processing plug-in to process the camera data and sending the obtained video encoded data to the decoding plug-in may include:

[0081] (1) When the data processing plug-in includes a protocol header parsing plug-in, a data stream de-encapsulation plug-in, and a data stream parsing plug-in, use the protocol header parsing plug-in to perform protocol header parsing on the camera data, and send the obtained TS stream to the data stream de-encapsulation plug-in.

[0082] The protocol header parsing plug-in can parse the Real-Time Transport Protocol (RTP) protocol header of the camera data to obtain the TS stream.

[0083] Such as Figure 1 shown, RTPMP2TDepay sends the TS stream to TSDemux.

[0084] (2) Use the data stream de-encapsulation plug-in to de-encapsulate the TS stream, and send the obtained encoded data stream to the data stream parsing plug-in.

[0085] The data stream de-encapsulation plug-in de-encapsulates the TS stream to obtain H.264 encoded data.

[0086] As Figure 1 shown, the TSDemux sends the H.264 encoded data to the H264Parse.

[0087] (3) Use the data stream parsing plug-in to parse the encoded data stream, and send the obtained video encoded data to the decoding plug-in.

[0088] The data stream parsing plug-in parses the H.264 encoded data to obtain video encoded data, which includes key frames and other metadata.

[0089] As Figure 1 shown, the H264Parse sends the video encoded data to the OMXH264Dec.

[0090] Step 304, use the decoding plug-in to store the video encoded data in the first shared memory, and send the first memory address of the first shared memory to the VPU driver in the RTOS system.

[0091] Specifically, sending the first memory address of the first shared memory to the VPU driver in the RTOS system may include:

[0092] (1) Use the decoding plug-in to send the first memory address of the first shared memory to the VPU client in the Linux system.

[0093] As Figure 1 shown, the OMXH264Dec sends the first memory address to the VPU Client through the OpenMAX framework, the VPU Proto HAL, and the VPU DummyDriver in sequence.

[0094] (2) Use the VPU client to send the first memory address to the VPU server in the RTOS system.

[0095] As Figure 1 shown, the VPU Client sends the first memory address to the VPU Server through RPMSG communication.

[0096] (3) Use the VPU server to send the first memory address to the VPU driver.

[0097] As Figure 1 shown, the VPU Server sends the first memory address to the VPU Driver through the VPU HAL.

[0098] Step 305: Use the VPU driver to read the video encoding data from the first shared memory according to the first memory address, decode the video encoding data into a preview image and store it in the second shared memory, and send the second memory address of the second shared memory to the decoding plugin.

[0099] Specifically, sending the second memory address of the second shared memory to the decoding plugin may include:

[0100] (1) Use the VPU driver to send the second memory address of the second shared memory to the VPU server.

[0101] As Figure 1 shown, the VPU Driver sends the second memory address to the VPU Server through the VPU HAL.

[0102] (2) Use the VPU server to send the second memory address to the VPU client in the Linux system.

[0103] As Figure 1 shown, the VPU Server sends the second memory address to the VPU Client through RPMSG communication.

[0104] (3) Use the VPU client to send the second memory address to the decoding plugin.

[0105] As Figure 1 shown, the VPU Client sequentially sends the first memory address to the OMXH264Dec through the VPU Dummy Driver, the VPU Proto HAL, and the OpenMAX framework.

[0106] Step 306: Use the decoding plugin to send the second memory address to the display plugin.

[0107] As Figure 1 shown, the OMXH264Dec sends the second memory address to the APPSink.

[0108] Step 307: Use the display plugin to share the preview image with the GPIPE layer in the DC, so that the DC displays the preview image as the upper-layer picture on the screen, and the DC is controlled by the RTOS.

[0109] Specifically, using the display plugin to share the preview image with the GPIPE layer in the DC may include:

[0110] (1) Use the display plugin to send the second memory address to the DRM library file driver.

[0111] As Figure 1As shown, APPSink sends the second memory address to the LibDRM Driver through the DRM Display.

[0112] (2) Use the DRM library file driver to send the second memory address to the GPIPE layer in the DC, so that the GPIPE layer reads the preview image from the second shared memory according to the second memory address.

[0113] As Figure 1 shown, the LibDRM Driver sends the second memory address to the GPIPE layer through RPCall communication. The GPIPE layer reads the preview image from the second shared memory according to the second memory address and displays the preview image as the upper layer screen on the screen. That is, the preview image is displayed on the top layer of the screen and will not be covered by other screens.

[0114] In summary, the SOC-based camera data preview method provided by the embodiments of the present application creates a streaming media processing pipeline in the Linux system, then uses the streaming media processing pipeline to receive the camera data sent by the Ethernet camera, uses the streaming media processing pipeline to interact with the RTOS system to decode the camera data, and uses the streaming media processing pipeline to interact with the RTOS system to display the preview image on the screen, so as to quickly preview the camera data.

[0115] As Figure 4 shown, it shows the structural block diagram of the SOC-based camera data preview device provided by an embodiment of the present application. The SOC-based camera data preview device can be applied to Figure 1 the SOC shown. The camera data preview device may include:

[0116] A creation module 410, configured to create a streaming media processing pipeline in the Linux system. The streaming media processing pipeline is obtained by combining multiple streaming media plugins using Gstreamer. The streaming media plugins at least include a data reception plugin, a data processing plugin, a decoding plugin, and a display sending plugin;

[0117] An acquisition module 420, configured to use the data reception plugin to acquire the camera data collected by the Ethernet camera and send the camera data to the data processing plugin;

[0118] A processing module 430, configured to use the data processing plugin to process the camera data and send the obtained video encoding data to the decoding plugin;

[0119] A decoding module 440, configured to use the decoding plugin to decode the video encoding data through shared memory and share the obtained preview image to the display sending plugin;

[0120] The display sending module 450 is used to share the preview image with the GPIPE layer in the DC by using the display sending plug-in, so that the DC displays the preview image as the upper-layer picture on the screen, and the DC is controlled by the RTOS.

[0121] In an optional embodiment, the decoding module 440 is further used for:

[0122] Storing the video encoding data into the first shared memory by using the decoding plug-in, and sending the first memory address of the first shared memory to the VPU driver in the RTOS system;

[0123] Reading the video encoding data from the first shared memory according to the first memory address by using the VPU driver, decoding the video encoding data into a preview image and storing it into the second shared memory, and sending the second memory address of the second shared memory to the decoding plug-in;

[0124] Sending the second memory address to the display sending plug-in by using the decoding plug-in.

[0125] In an optional embodiment, the decoding module 440 is further used for:

[0126] Sending the first memory address of the first shared memory to the VPU client in the Linux system by using the decoding plug-in;

[0127] Sending the first memory address to the VPU server in the RTOS system by using the VPU client;

[0128] Sending the first memory address to the VPU driver by using the VPU server.

[0129] In an optional embodiment, the decoding module 440 is further used for:

[0130] Sending the second memory address of the second shared memory to the VPU server by using the VPU driver;

[0131] Sending the second memory address to the VPU client in the Linux system by using the VPU server;

[0132] Sending the second memory address to the decoding plug-in by using the VPU client.

[0133] In an optional embodiment, the display sending module 450 is further used for:

[0134] Sending the second memory address to the DRM library file driver by using the display sending plug-in;

[0135] Using the DRM library file driver to send the second memory address to the GPIPE layer in the DC, so that the GPIPE layer reads the preview image from the second shared memory according to the second memory address.

[0136] In an optional embodiment, the creation module 410 is further configured to:

[0137] When starting the Linux system, create a streaming media processing pipeline in the Linux system.

[0138] In an optional embodiment, the processing module 430 is further configured to:

[0139] When the data processing plugin includes a protocol header parsing plugin, a data stream decapsulation plugin, and a data stream parsing plugin, use the protocol header parsing plugin to perform protocol header parsing on the camera data, and send the obtained TS stream to the data stream decapsulation plugin;

[0140] Use the data stream decapsulation plugin to perform decapsulation on the TS stream, and send the obtained encoded data stream to the data stream parsing plugin;

[0141] Use the data stream parsing plugin to parse the encoded data stream, and send the obtained video encoded data to the decoding plugin.

[0142] In summary, the camera data preview device based on SOC provided by the embodiments of the present application creates a streaming media processing pipeline in the Linux system, then uses the streaming media processing pipeline to receive the camera data sent by the Ethernet camera, uses the streaming media processing pipeline to interact with the RTOS system to decode the camera data, and uses the streaming media processing pipeline to interact with the RTOS system to display the preview image on the screen, so as to be able to quickly preview the camera data.

[0143] An embodiment of the present application provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned method for previewing camera data based on SOC.

[0144] An embodiment of the present application provides a chip, and the chip includes any of the above-mentioned camera data preview devices based on SOC.

[0145] It should be noted that: when the above-mentioned camera data preview device based on SOC performs preview of camera data based on SOC, only the above-mentioned division of each functional module is used for illustration. In actual application, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the camera data preview device based on SOC is divided into different functional modules to complete all or part of the above-described functions. In addition, the above-mentioned camera data preview device based on SOC and the embodiment of the method for previewing camera data based on SOC belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0146] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk, an optical disc, etc.

[0147] The above does not intend to limit the embodiments of the present application. Any adjustment, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.

Claims

1. A camera data preview method based on SOC, characterized in that: The first group of kernels in the SOC is configured to run a Linux system, and the second group of kernels is configured to run a real-time operating system RTOS, and the first group of kernels and the second group of kernels are different from each other, and the method includes: Creating a streaming media processing pipeline in the Linux system, wherein the streaming media processing pipeline is obtained by combining multiple streaming media plug-ins using the open source framework Gstreamer, and the streaming media plug-ins at least include a data receiving plug-in, a data processing plug-in, a decoding plug-in, and a display sending plug-in; Using the data receiving plug-in to obtain the camera data collected by the Ethernet camera, and sending the camera data to the data processing plug-in; Processing the camera data using the data processing plug-in, and sending the obtained video encoding data to the decoding plug-in; Decoding the video encoding data by using the decoding plug-in through the shared memory, and sharing the obtained preview image with the display sending plug-in; The preview image is shared with the graphics pipeline GPIPE layer in the display controller DC by using the display plug-in, so that the DC displays the preview image as the upper layer image on the screen, and the DC is controlled by the RTOS.

2. The SOC-based camera data preview method according to claim 1, characterized in that: The method of decoding the video encoding data by using the decoding plug-in through the shared memory and sharing the obtained preview image with the display plug-in includes: Using the decoding plug-in to store the video encoding data in a first shared memory, and sending a first memory address of the first shared memory to a video processing unit VPU driver in the RTOS system; Using the VPU driver to read the video encoding data from the first shared memory according to the first memory address, decoding the video encoding data into a preview image and storing it in a second shared memory, and sending a second memory address of the second shared memory to the decoding plug-in; The second memory address is sent to the display plug-in by using the decoding plug-in.

3. The SOC-based camera data preview method according to claim 2, characterized in that: The sending the first memory address of the first shared memory to the video processing unit VPU driver in the RTOS system includes: Using the decoding plug-in, the first memory address of the first shared memory is sent to the VPU client in the Linux system; Using the VPU client to send the first memory address to the VPU server in the RTOS system; The VPU server is used to send the first memory address to the VPU driver.

4. The SOC-based camera data preview method according to claim 2, characterized in that: The sending the second memory address of the second shared memory to the decoding plug-in includes: Using the VPU driver, the second memory address of the second shared memory is sent to the VPU server; Using the VPU server to send the second memory address to the VPU client in the Linux system; The second memory address is sent to the decoding plug-in using the VPU client.

5. The SOC-based camera data preview method according to claim 2, characterized in that: The step of using the display plug-in to share the preview image with the GPIPE layer in the DC includes: Using the display sending plug-in, the second memory address is sent to a display rendering manager DRM library file driver; The DRM library file driver sends the second memory address to the GPIPE layer in the DC, so that the GPIPE layer reads the preview image from the second shared memory according to the second memory address.

6. The SOC-based camera data preview method according to claim 1, characterized in that: The creating a streaming media processing pipeline in the Linux system includes: When the Linux system is started, a streaming media processing pipeline is created in the Linux system.

7. The SOC-based camera data preview method according to any one of claims 1 to 6, characterized in that: The processing of the camera data by the data processing plug-in and sending the obtained video encoding data to the decoding plug-in includes: When the data processing plug-in includes a protocol header parsing plug-in, a data stream decapsulation plug-in and a data stream parsing plug-in, the protocol header parsing plug-in is used to parse the protocol header of the camera data, and the obtained transport stream TS stream is sent to the data stream decapsulation plug-in; Decapsulating the TS stream using the data stream decapsulation plug-in, and sending the obtained encoded data stream to the data stream parsing plug-in; The coded data stream is parsed using the data stream parsing plug-in, and the obtained video coded data is sent to the decoding plug-in.

8. A camera data preview device based on SOC, characterized in that: Used in a SOC, wherein a first group of kernels in the SOC is configured to run a Linux system, a second group of kernels is configured to run a real-time operating system RTOS, and the first group of kernels and the second group of kernels are different; A creation module is used to create a streaming media processing pipeline in the Linux system, wherein the streaming media processing pipeline is obtained by combining multiple streaming media plug-ins using the open source framework Gstreamer, and the streaming media plug-ins at least include a data receiving plug-in, a data processing plug-in, a decoding plug-in, and a sending and displaying plug-in; An acquisition module, used for acquiring camera data collected by an Ethernet camera using the data receiving plug-in, and sending the camera data to the data processing plug-in; A processing module, used to process the camera data using the data processing plug-in, and send the obtained video encoding data to the decoding plug-in; A decoding module, used to use the decoding plug-in to decode the video encoding data through the shared memory, and share the obtained preview image with the display plug-in; The display sending module is used to share the preview image with the graphics pipeline GPIPE layer in the display controller DC by using the display sending plug-in, so that the DC displays the preview image as the upper layer picture on the screen, and the DC is controlled by the RTOS.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the SOC-based camera data preview method as described in any one of claims 1 to 7.

10. A chip, characterized in that: The chip includes the SOC-based camera data preview device as described in claim 8.

Images