Video processing system and method

Abstract

A video processing system includes a system processing circuit and a video processing circuit. The system processing circuit includes an image buffer and an OpenMAX layer. The image buffer is used to store video data from a camera. The OpenMAX layer is used to extract at least one data parameter related to the video data. The video processing circuit is used to receive at least one data parameter, receive video data from the image buffer according to the at least one data parameter, encode the video data according to the at least one data parameter to generate encoded data, and transmit the encoded data to the system processing circuit.

Description

Technical Field

[0001] The embodiments described in this disclosure relate to a video processing system and method, and particularly to a video processing system and method that can reduce the workload of the system processing circuit (loading). Background Art

[0002] With the development of technology, more and more electronic devices are equipped with cameras for users to take pictures or record videos and other functions. Taking video recording as an example, in the prior art, when the resolution of video data is higher, the workload of the system processor in these electronic devices will be greater. This will cause the operating system run by the system processor to run slowly or even crash. Summary of the Invention

[0003] Some embodiments of this disclosure relate to a video processing system. The video processing system includes a system processing circuit and a video processing circuit. The system processing circuit includes an image buffer and an OpenMAX layer. The image buffer is used to store video data from a camera. The OpenMAX layer is used to extract at least one data parameter related to the video data. The video processing circuit is used to receive at least one data parameter, receive video data from the image buffer according to at least one data parameter, encode the video data according to at least one data parameter to generate encoded data, and transmit the encoded data to the system processing circuit.

[0004] Some embodiments of this disclosure relate to a video processing method. The video processing method includes the following operations: storing video data from a camera through the image buffer of the system processing circuit; extracting at least one data parameter related to the video data through the OpenMAX layer of the system processing circuit; receiving at least one data parameter through the video processing circuit; receiving video data from the image buffer by the video processing circuit according to at least one data parameter; encoding the video data by the video processing circuit according to at least one data parameter to generate encoded data; and transmitting the encoded data to the system processing circuit by the video processing circuit.

[0005] In summary, in this disclosure, the system processing circuit can extract the data parameters of the video data and transmit the data parameters to the video processing circuit. Then, the video processing circuit can directly receive the video data from the image buffer according to the data parameters for subsequent encoding. Accordingly, the system processing circuit does not need to copy the video data multiple times, so the workload of the system processing circuit can be reduced, and thus the operating system can run smoothly. Brief Description of the Drawings

[0006] To make the above and other purposes, features, advantages and embodiments of this disclosure more obvious and understandable, the drawings are described as follows:

[0007] Figure 1 is a schematic diagram of a video processing system depicted according to some embodiments of the present disclosure; and

[0008] Figure 2 is a flowchart of a video processing method depicted according to some embodiments of the present disclosure. Detailed Embodiments

[0009] As used herein, the term "coupled" may also refer to "electrically coupled", and the term "connected" may also refer to "electrically connected". "Coupled" and "connected" may also refer to two or more elements cooperating or interacting with each other.

[0010] Refer to Figure 1 . Figure 1 is a schematic diagram of a video processing system 100 depicted according to some embodiments of the present disclosure.

[0011] For Figure 1 example, the video processing system 100 includes a system processing circuit 110 and a video processing circuit 120. The system processing circuit 110 is coupled to the video processing circuit 120 and the camera CM.

[0012] The video processing system 100 may be disposed on a system on chip (SoC) of an electronic device. In some embodiments, if the above-mentioned electronic device is a smart TV, the camera CM may be disposed outside the system chip of the smart TV. In some other embodiments, if the above-mentioned electronic device is a smart phone, the camera CM may be integrated with the video processing system 100 and disposed inside the smart phone.

[0013] The camera CM can be used to perform functions such as taking pictures or recording videos. The following will take the video recording function as an example for illustration.

[0014] Functionally, the system processing circuit 110 mainly runs application programs related to the operating system of the electronic device. For example, if the electronic device runs the Android operating system, the system processing circuit 110 can run application programs related to the Android operating system. The video processing circuit 120 is mainly used to process the video data VD from the camera CM. For example, when the camera CM executes the video recording function, the camera CM generates the video data VD. The video processing circuit 120 can encode the video data VD from the camera CM and send the encoded data back to the system processing circuit 110.

[0015] For Figure 1For example, the system processing circuit 110 includes a frame buffer 111, a graphic buffer 112, a system framework 113, an open media acceleration (OpenMax) layer 114, and a decoding circuit 115. In some embodiments, the system framework 113 and the open media acceleration layer 114 are implemented in software and stored in a storage element of the system processing circuit 110.

[0016] The frame buffer 111 is coupled to the camera CM and the graphic buffer 112. The graphic buffer 112 is coupled to the system framework 113. The open media acceleration layer 114 is coupled between the system framework 113 and the video processing circuit 120. The system framework 113 is also coupled to the decoding circuit 115.

[0017] For Figure 1 example, the video processing circuit 120 includes a ring buffer 121, a reading circuit 122, a memory 123, an encoding circuit 124, and a ring buffer 125.

[0018] The ring buffer 121 is coupled to the open media acceleration layer 114 and the reading circuit 122. The reading circuit 122 is coupled to the memory 123 and the encoding circuit 124. The encoding circuit 124 is coupled to the memory 123 and the ring buffer 125. The ring buffer 125 is coupled to the open media acceleration layer 114.

[0019] Hereinafter, the video processing system 100 disposed in a smart TV equipped with an Android operating system will be taken as an example for illustration, but the present disclosure is not limited thereto.

[0020] In operation, when the camera CM executes the video recording function, the camera CM generates original video data VD. The video data VD is transmitted to the frame buffer 111. For example, the video data VD in the frame buffer 111 is stored in a frame-by-frame form. Then, the video data VD can be converted and stored in the graphic buffer 112 through an application programming interface (e.g., CameraHal). The application programming interface can be implemented in software. For example, the video data VD is stored in the graphic buffer 112 according to the specifications of an operating system (e.g., the Android operating system). In some embodiments, the graphic buffer 112 can be configured in the dynamic random-access memory (DRAM) of the system processing circuit 110.

[0021] Video data VD can be transmitted to the open multimedia acceleration layer 114 through the system framework 113 of the operating system. The open multimedia acceleration layer 114 can extract at least one data parameter P related to the video data VD. Then, the open multimedia acceleration layer 114 can transmit the data parameter P to the video processing circuit 120. In some embodiments, the data parameter P may include at least one of the physical address, data size, and encoding format of the video data VD in the image buffer 112. The data size is, for example, the data length. The encoding format is, for example, YUV or RGB. In some embodiments, the foregoing physical address, data size, and encoding format may be encapsulated in a data structure for transmission to the video processing circuit 120.

[0022] The video processing circuit 120 can receive the data parameter P from the system processing circuit 110 and receive the video data VD from the image buffer 112 according to the data parameter P. Specifically, the circular buffer 121 can receive the data parameter P from the open multimedia acceleration layer 114 and transmit the data parameter P to the reading circuit 122. The reading circuit 122 can receive the data parameter P and request the video processing buffer 1231 from the memory 123 according to the data parameter P. On the other hand, the reading circuit 122 can read out the video data VD stored in the image buffer 112 according to the data parameter P. For example, the reading circuit 122 can determine the corresponding position in the image buffer 112 where the video data VD to be read is stored according to the physical address carried by the data parameter P, and can determine the data volume of the video data VD to be read according to the data size carried by the data parameter P. Accordingly, the reading circuit 122 can read out the video data VD stored in the image buffer 112 according to the above physical address and data size. Then, the reading circuit 122 can store the read video data VD into the video processing buffer 1231. The capacity of the video processing buffer 1231 can be determined according to the data volume of the video data VD read from the image buffer 112.

[0023] The encoding circuit 124 can receive the data parameter P from the reading circuit 122 and receive the video data VD from the video processing buffer 1231. As mentioned above, the data parameter P may further include the encoding format of the video data VD. That is, the encoding circuit 124 can encode the video data VD according to the encoding format (for example: YUV or RGB) in the data parameter P to generate encoded data ED.

[0024] Next, the open multimedia acceleration layer 114 can receive the encoded data ED from the circular buffer 125. The open multimedia acceleration layer 114 can transmit the encoded data ED to the system framework 113. And the system framework 113 can transmit the encoded data ED to the decoding circuit 115. And the decoding circuit 115 can decode the encoded data ED to generate the decoded data DD.

[0025] In some embodiments, the decoded data DD can be used by a display to display a corresponding image. In some other embodiments, the decoded data DD can also be stored.

[0026] In some related technologies, the video data generated by a camera is first copied to the buffer of the open multimedia acceleration layer. Then, it is copied from the buffer of the open multimedia acceleration layer to the shared memory in the video processing circuit. In other words, the video data is copied multiple times in the system processing circuit. However, multiple copies will increase the workload of the system processing circuit. In particular, the amount of video data before encoding is very large, and as the resolution of the video data is higher, the workload of the system processing circuit will be greater. This can cause the operating system run by the system processor to run slowly or even crash.

[0027] Compared with these related technologies above, in the present disclosure, the system processing circuit 110 can extract the data parameter P of the video data VD and transmit the data parameter P to the video processing circuit 120. Then, the video processing circuit 120 can directly receive (copy) the video data VD from the image buffer 112 according to the data parameter P for subsequent encoding. Accordingly, the video data VD does not need to be copied multiple times in the system processing circuit 110, so the workload of the system processing circuit 110 can be reduced, and thus the operating system can run smoothly.

[0028] In addition, the video processing circuit 120 is mainly implemented by hardware. Accordingly, compared with these related technologies above that use software (system framework 113, open multimedia acceleration layer 114) for data copying, the data copying speed of the video processing circuit 120 implemented by hardware is faster, so the present disclosure can improve the processing efficiency of the video data VD.

[0029] Furthermore, by the method of the present disclosure, it is not necessary to reduce the resolution or frame rate of the video data VD, nor to close other application programs run by the system processing circuit 110.

[0030] Reference Figure 2 . Figure 2 is a flowchart of the video processing method 200 depicted according to some embodiments of the present disclosure. Figure 2For example, the video processing method 200 includes operation S210, operation S220, operation S230, operation S240, operation S250, and operation S260. It should be understood that, among the operations mentioned in this implementation method, unless the order is specifically stated, the order before and after can be adjusted according to actual needs, and they can even be executed simultaneously or partially simultaneously.

[0031] In some embodiments, the video processing method 200 can be applied to Figure 1 the video processing system 100 in, but the present disclosure is not limited thereto. For ease of understanding, the following paragraphs will be described in conjunction with Figure 1 the video processing system 100.

[0032] In operation S210, the video data VD from the camera CM is stored through the image buffer 112 of the system processing circuit 110. In some embodiments, the camera CM can be disposed outside the system chip of the electronic device. In some embodiments, the camera CM can be disposed on the system chip of the electronic device.

[0033] In operation S220, the data parameter P related to the video data VD is extracted through the OpenMAX layer 114 of the system processing circuit 110. In some embodiments, the data parameter P includes at least one of the physical address of the video data VD in the image buffer 112, the data size of the video data VD, and the encoding format of the video data VD.

[0034] In operation S230, the data parameter P is received through the video processing circuit 120. In some embodiments, the data parameter P can be transmitted from the OpenMAX layer 114 to the ring buffer 121.

[0035] In operation S240, the video data VD is received from the image buffer 112 through the video processing circuit 120 according to the data parameter P. In some embodiments, the reading circuit 122 can read out the video data VD stored in the image buffer 112 according to the above physical address and data size.

[0036] In operation S250, the video data VD is encoded through the video processing circuit 120 according to the data parameter P to generate encoded data ED. In some embodiments, the encoding circuit 124 can encode the video data VD according to the encoding format (e.g., YUV or RGB) in the data parameter P to generate encoded data ED.

[0037] In operation S260, the encoded data ED is transmitted to the system processing circuit 110 through the video processing circuit 120. In some embodiments, the encoded data ED can be transmitted from the ring buffer 125 to the OpenMAX layer 114 and transmitted to the decoding circuit 115 through the system framework 113.

[0038] In summary, in the present disclosure, the system processing circuit can extract the data parameters of the video data and transmit the data parameters to the video processing circuit. Then, the video processing circuit can directly receive the video data from the image buffer according to the data parameters for subsequent encoding. Accordingly, the system processing circuit does not need to copy the video data multiple times, so the workload of the system processing circuit can be reduced, and thus the operating system can run smoothly.

[0039] Various functional elements and modules have been disclosed herein. For those of ordinary skill in the art, the functional modules can be implemented by circuits (either dedicated circuits or general-purpose circuits operating under the control of one or more processors and encoded instructions), which generally include transistors or other circuit elements for controlling the operation of electrical circuits corresponding to the functions and operations described herein. Further understanding, generally, the specific structure and interconnection of circuit elements can be determined by a compiler, such as a Register Transfer Language (RTL) compiler. The Register Transfer Language compiler operates on a script that is quite similar to assembly language code and compiles the script into a form for layout or fabrication of the final circuit. Indeed, the Register Transfer Language is well-known for its role and use in facilitating the design process of electronic and digital systems.

[0040] Although the present disclosure has been disclosed as above in embodiments, it is not intended to limit the present disclosure. Any person of ordinary skill in the art can make various changes and improvements without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be defined by the scope of the appended claims.

[0041] Description of the reference numerals:

[0042] 100: Video processing system

[0043] 110: System processing circuit

[0044] 111: Frame buffer

[0045] 112: Image buffer

[0046] 113: System framework

[0047] 114: Open Multimedia Acceleration Layer

[0048] 115: Decoding circuit

[0049] 120: Video processing circuit

[0050] 121: Circular buffer

[0051] 122: Read circuit

[0052] 123: Memory

[0053] 1231: Video processing buffer

[0054] 124: Encoding circuit

[0055] 125: Circular buffer

[0056] 200: Video processing method

[0057] CM: Camera

[0058] VD: Video data

[0059] ED: Encoded data

[0060] DD: Decoded data

[0061] P: Data parameter

[0062] S210, S220, S230, S240, S250, S260: Operations

Claims

1. A video processing system, comprising: A system processing circuit and a video processing circuit, the system processing circuit comprising: An image buffer for storing video data from a camera; And An open multimedia acceleration layer for extracting at least one data parameter related to the video data; And The video processing circuit for receiving the at least one data parameter, receiving the video data from the image buffer according to the at least one data parameter, encoding the video data according to the at least one data parameter to generate encoded data, and transmitting the encoded data to the system processing circuit.

2. The video processing system according to claim 1, wherein the video processing circuit comprises: A first circular buffer for receiving the at least one data parameter.

3. The video processing system according to claim 2, wherein the video processing circuit further comprises: A reading circuit for reading the at least one data parameter received by the first circular buffer and requesting a video processing buffer according to the at least one data parameter.

4. The video processing system according to claim 3, wherein the reading circuit is further configured to read the video data in the image buffer according to the at least one data parameter and store the video data in the video processing buffer.

5. The video processing system according to claim 3, wherein the at least one data parameter comprises a physical address and a data size, and the reading circuit is further configured to read the video data in the image buffer according to the physical address and the data size.

6. The video processing system according to claim 5, wherein the video processing circuit further comprises: An encoding circuit, wherein the at least one data parameter comprises an encoding format, and the encoding circuit is configured to encode the video data according to the encoding format to generate the encoded data.

7. The video processing system according to claim 6, wherein the video processing circuit further comprises: A second circular buffer for receiving the encoded data, and the open multimedia acceleration layer is further configured to receive the encoded data.

8. The video processing system according to claim 1, wherein the system processing circuit further comprises: A system framework for receiving the encoded data from the video processing circuit.

9. The video processing system according to claim 8, wherein the system processing circuit further comprises: A decoding circuit for decoding the encoded data to generate decoded data, and the decoded data is for display by a display.

10. A video processing method, comprising: Storing video data from a camera through an image buffer of a system processing circuit; Extracting at least one data parameter related to the video data through an open multimedia acceleration layer of the system processing circuit; Receiving the at least one data parameter through a video processing circuit; Receiving the video data from the image buffer through the video processing circuit according to the at least one data parameter; Encoding the video data according to the at least one data parameter through the video processing circuit to generate encoded data; And The encoded data is transmitted to the system processing circuit through the video processing circuit.

Images